Text tools: grep, sed, awk, find¶
Status: AI draft, 2026-11-16. Not yet verified by me. Commands and output were run in a throwaway container; addresses are rewritten to example ranges.
In one line¶
grep picks lines, cut and awk pick fields, sort and uniq -c count, sed edits text as it streams past, and find picks files. Chained with pipes, they answer most "what is in this log?" questions in one line.
The plain idea¶
Logs, configs and command output are just lines of text. Most questions about them are one of a few shapes: "which lines mention X?", "how many of each?", "what is in column 9?", "what are the slowest?". Each shape has a small tool, and you join the tools with pipes (Streams, pipes and redirection).
The habit that makes this work is building a pipeline one step at a time. Run the first command, look at the output, add the next stage, look again. You never write the whole thing in one go.
How it works¶
flowchart TD
A["access.log"] --> B["grep: keep matching lines"]
B --> C["awk or cut: keep one field"]
C --> D["sort: group equal values together"]
D --> E["uniq -c: count each group"]
E --> F["sort -rn: biggest first"]
F --> G["head: top N"]
The tools and their one job¶
| Tool | Job | Works on |
|---|---|---|
grep |
Keep lines that match a pattern (or, with -v, that do not) |
Lines |
cut |
Keep fields split by one fixed delimiter | Fields |
awk |
Split each line into fields, test conditions, print, sum, count | Fields, with logic |
sed |
Edit the stream: substitute, delete or print lines | Lines |
sort |
Order lines; -n numeric, -r reverse, -k by field |
Whole input |
uniq |
Collapse repeated adjacent lines; -c counts them |
Sorted input |
wc -l |
Count lines | Whole input |
head, tail |
First or last N lines; tail -f follows a growing file |
Lines |
find |
Find files by name, age, size or type, and act on them | The filesystem |
uniq only merges lines that are next to each other, which is why sort almost always comes before it.
grep and regular expressions¶
grep matches a regular expression, a small pattern language. With -E (extended), the common pieces are:
| Pattern | Matches |
|---|---|
. |
Any one character |
^ and $ |
Start and end of the line |
[0-9], [a-z] |
One character from a set |
x*, x+, x? |
Zero or more, one or more, zero or one of x |
\. |
A literal dot |
A vertical bar separates alternatives: (500|502) matches either code.
awk in one paragraph¶
awk reads a line, splits it on whitespace into $1, $2 and so on ($0 is the whole line, NF the number of fields, NR the line number), then runs your program on it. A program is condition { action }. Either part may be left out. END { ... } runs after the last line, which is where totals are printed. Variables need no declaring and start at zero.
sed in one paragraph¶
sed applies an edit to each line as it passes. The one you will use most is substitute, s/old/new/, where old is a regular expression; add g to replace every match on the line, not just the first. sed -n '3,4p' prints only lines 3 to 4. sed -i edits a file in place, so try the command without -i first.
A concrete example¶
Users report errors on the orders API. You have the access log. Each line looks like this; field 1 is the client address, field 9 the status code, and the last field (11) the response time in seconds:
203.0.113.10 - - [16/Nov/2026:09:00:02 +0000] "POST /api/orders HTTP/1.1" 500 87 1.204
- How many 5xx responses?
grep -c " 50[0-9] " access.logprints4. - Which status codes, and how many of each?
awk '{print $9}' access.log | sort | uniq -c | sort -rn. - What share are errors?
awk '$9 >= 500 {e++} END {printf "errors %d of %d\n", e, NR}' access.log. - Which requests were slowest?
sort -k11 -rn access.log | head -3. One took 30 seconds and returned 502: a sign of a timeout behind a proxy (Load balancers and proxies). - Who is sending the errors?
grep -E " (500|502|503) " access.log | cut -d" " -f1,9.
Five commands, and you have the error rate, the slowest calls and the clients affected, before opening any dashboard.
Why not just...?¶
- ...open the log in an editor? A production log can be gigabytes. Editors load it into memory;
grepandawkstream it and use almost none. - ...use
cutfor everything?cutsplits on exactly one delimiter character, so two spaces in a row create an empty field.awksplits on any run of whitespace, which suits most logs. - ...use
sed -istraight away? It changes the file with no undo. Run the samesedwithout-ifirst and read the output, or use-i.bakto keep a copy.
Commands to know¶
Count, invert and number:
# grep -c " 50[0-9] " access.log
4
# grep -v "/health" access.log | wc -l
8
# grep -n "POST" access.log
3:203.0.113.10 - - [16/Nov/2026:09:00:02 +0000] "POST /api/orders HTTP/1.1" 500 87 1.204
7:203.0.113.11 - - [16/Nov/2026:09:00:06 +0000] "POST /api/orders HTTP/1.1" 201 77 0.233
-c counts matching lines, -v inverts the match, -n adds line numbers. Also: -i ignores case, -r searches a directory tree, -E uses extended patterns, and -A 2, -B 2, -C 2 show 2 lines after, before, or around each match.
Top talkers and a status-code histogram:
# awk '{print $1}' access.log | sort | uniq -c | sort -rn
4 203.0.113.10
2 203.0.113.11
2 198.51.100.7
1 203.0.113.13
1 203.0.113.12
# awk '{print $9}' access.log | sort | uniq -c | sort -rn
5 200
1 503
1 502
1 500
1 404
1 201
A rate and the slowest requests:
# awk '$9 >= 500 {e++} END {printf "errors %d of %d (%.0f%%)\n", e, NR, 100*e/NR}' access.log
errors 3 of 10 (30%)
# sort -k11 -rn access.log | head -3 | awk '{print $11, $7, $9}'
30.001 /api/orders 502
1.204 /api/orders 500
0.233 /api/orders 201
sort -k11 -rn sorts by field 11, numerically, largest first.
Pick fields and lines, and edit:
# grep -E " (500|502|503) " access.log | cut -d" " -f1,9
203.0.113.10 500
203.0.113.10 502
203.0.113.13 503
# sed -n "3,4p" access.log
203.0.113.10 - - [16/Nov/2026:09:00:02 +0000] "POST /api/orders HTTP/1.1" 500 87 1.204
198.51.100.7 - - [16/Nov/2026:09:00:03 +0000] "GET /health HTTP/1.1" 200 2 0.001
# echo "level=INFO user=alice" | sed -E "s/user=[a-z]+/user=REDACTED/"
level=INFO user=REDACTED
Find old files, look first, then act:
# find /var/log/app -name "*.log" -mtime +7
/var/log/app/old.log
# find /var/log/app -name "*.log" -mtime +7 -exec rm -v {} \;
removed '/var/log/app/old.log'
-mtime +7 means modified more than 7 days ago. -exec ... {} \; runs the command once per file, with {} replaced by the name. Other useful tests: -type f (files only), -size +100M, -user app, -maxdepth 1.
How it breaks¶
| Symptom | Layer | Check | Likely cause |
|---|---|---|---|
uniq -c shows the same value several times |
Input order | Was the input sorted? | uniq only merges adjacent lines |
sort puts 10 before 9 |
Sort type | Add -n |
Text sort compares character by character |
cut gives empty or wrong fields |
Delimiter | cat -A shows tabs as ^I and line ends as $ |
Several spaces in a row, or tabs; use awk |
grep misses lines you can see |
Pattern | Try -F for a fixed string, or -i |
Special characters such as . or [ treated as pattern syntax, or case differences |
sed -i damaged a config file |
In-place edit | Compare with a backup | The pattern matched more than intended; test without -i first |
find -exec rm deleted the wrong files |
Selection | Run the same find without -exec |
The test was too broad, for example a missing -name or -type f |
Interview questions¶
1. How do you find the 5 IP addresses that made the most requests in an access log?
Answer
- Extract the field:
awk '{print $1}' access.log. - Sort so equal values are adjacent, then count:
sortand thenuniq -c. - Sort the counts numerically, largest first, and take the top:
sort -rnand thenhead -5.
2. Why does uniq need sorted input?
Answer
uniqonly compares each line with the one before it.- Equal values that are not next to each other are counted separately.
- Sorting first brings equal lines together;
sort -uis an alternative when you only need distinct values.
3. When would you use awk instead of cut?
Answer
cutsplits on one fixed character, so runs of spaces break it.awksplits on any whitespace by default and can test conditions, compute and sum.- Use
cutfor simple, consistently delimited data such as CSV without quotes;awkfor logs and anything with logic.
4. How do you show 3 lines of context around every "Exception" in a log, case-insensitively?
Answer
grep -i -C 3 exception app.log.-Aand-Bgive lines after or before only.- Add
-nfor line numbers.
5. How do you replace a value in a config file safely with sed?
Answer
- Run the substitution without
-ifirst and read the result. - Anchor the pattern so it only matches the intended line, for example
s/^port=.*/port=8080/. - Use
-i.bakto keep a copy, then compare withdiff.
6. A disk is filling with old logs. How do you find and delete logs older than 14 days under /var/log/app safely?
Answer
- List first:
find /var/log/app -type f -name "*.log" -mtime +14. - Check the list, then add
-deleteor-exec rm {} \;. - Check that no process still holds them open; deleted open files keep using space (Disks and filesystems).
- The lasting fix is log rotation, not manual deletion.
7. Using only the access log, how would you work out whether a spike of 502s came from one client or from everyone?
Answer
- Filter the 502 lines:
awk '$9 == 502'. - Count by client: print field 1, then
sortanduniq -c. - Count by path the same way, and by minute (part of field 4).
- One client points at that client; all clients on one path at one time points at the backend.
Try it¶
Step 1. Start a throwaway container and make a small log.
docker run --rm -it ubuntu:24.04 bash
cd /tmp
printf '%s\n' \
'203.0.113.10 GET /api/orders 200 0.041' \
'203.0.113.11 GET /api/orders 500 1.204' \
'203.0.113.10 GET /health 200 0.001' \
'203.0.113.12 POST /api/orders 502 30.001' \
'203.0.113.10 GET /api/orders 200 0.044' > app.log
Step 2. Before running each line, predict its output.
grep -c " 5[0-9][0-9] " app.log
awk '{print $1}' app.log | sort | uniq -c | sort -rn
awk '$4 >= 500 {e++} END {print e, "of", NR}' app.log
sort -k5 -rn app.log | head -1
Step 3. Type exit. The container is deleted.
Zoom out¶
These tools are the first pass in almost every investigation, before metrics and dashboards: Logs tells you where the text lives, and jq and yq do the same job for JSON and YAML. Without them you are scrolling. The principle to carry: build a pipeline one stage at a time, and look at the output after every stage.