Pipes, Redirection and Shell Scripts

Chain commands with pipes, redirect stdout and stderr, use exit codes and write your first safe Bash shell script with variables, if and for loops.

• Beginner
• 16 min read
• 4 objectives

Before this lessonLesson 1: Command Line Essentials

In the command line lesson you met | and > briefly. This lesson goes deeper. The real power of the terminal is not any single command, it is combining small commands into a pipeline, and then saving that pipeline as a script so you never have to type it again.

Everything here works in Bash and Zsh on macOS and Linux, and in Git Bash or WSL on Windows. Git hooks, CI jobs and deploy steps are all shell scripts, so this is the glue for the rest of the track.

Three streams: stdin, stdout, stderr

Every program starts with three open channels. stdin (0) is where it reads input, stdout (1) is where normal output goes and stderr (2) is where error messages go. By default all three are your terminal, which is why errors and output look mixed together. Keeping errors on a separate stream is what lets you save clean output to a file while still seeing problems on screen.

Pipes: one command's output is the next one's input

The pipe | connects stdout of the left command to stdin of the right one. Each tool does one small job well. Suppose app.log looks like this:

2026-09-01 INFO start
2026-09-01 ERROR db timeout
2026-09-02 INFO ok
2026-09-02 ERROR db timeout
2026-09-03 ERROR disk full

How many errors are there, and which one is most common?

grep ERROR app.log | wc -l
grep ERROR app.log | cut -d' ' -f3- | sort | uniq -c | sort -rn

       3
   2 db timeout
   1 disk full

Read the second line left to right: keep error lines, cut away the date and level (fields 3 onward, split on spaces), sort so duplicates sit together, count duplicates with uniq -c, then sort numerically in reverse. Five tiny tools, one useful report.

Redirection: sending streams to files

• cmd > out.txt: write stdout to a file, overwriting it.
• cmd >> out.txt: append stdout to the end of the file.
• cmd 2> err.txt: write stderr to a file.
• cmd > all.txt 2>&1: send both streams to the same file (order matters: redirect stdout first).
• cmd 2>/dev/null: throw error messages away.
• cmd < in.txt: feed a file to stdin.

grep ERROR app.log > errors.txt        # save the report
date >> build.log                      # add a line, keep old ones
npm test > test.log 2>&1               # capture everything
find / -name '*.conf' 2>/dev/null      # hide 'Permission denied' noise

Exit codes, && and ||

Every command finishes with an exit code: 0 means success and anything else means failure. The special variable $? holds the code of the last command.

ls nope.txt 2>/dev/null; echo "exit: $?"

exit: 1

You rarely read $? by hand. Instead you chain commands on success or failure:

npm test && git push                 # push only if tests pass
mkdir -p build || echo "could not create build/"
cd project && git pull && npm install

; runs the next command no matter what, && runs it only on success, and || runs it only on failure. CI systems use exactly this rule: a step fails when its script exits non-zero.

Variables and command substitution

name="stackcone"            # no spaces around =
echo "Hello, $name"
today=$(date +%F)          # capture a command's output
echo "Backup for $today"
count=$(grep -c ERROR app.log)
echo "Errors: $count"

Always wrap variables in double quotes, like "$file". Without quotes, a filename with a space in it is split into two words and your script breaks in confusing ways. Single quotes do not expand variables at all: '$name' prints the literal text.

Your first script

A script is just a text file of commands. Save this as count.sh:

#!/usr/bin/env bash
set -euo pipefail

file="${1:-app.log}"

if [[ ! -f "$file" ]]; then
  echo "No such file: $file" >&2
  exit 1
fi

for level in INFO ERROR; do
  n=$(grep -c "$level" "$file" || true)
  echo "$level: $n"
done

• #!/usr/bin/env bash (the shebang) tells the system which interpreter runs the file.
• set -euo pipefail makes the script stop on any failing command, on unset variables and on failures inside pipelines. It turns silent bugs into loud ones.
• $1 is the first argument; ${1:-app.log} means "use app.log if none was given".
• >&2 prints the error message on stderr, and exit 1 reports failure to the caller.
• || true is needed because grep -c exits 1 when it finds zero matches, which set -e would treat as a crash.

Make it executable once, then run it:

chmod +x count.sh
./count.sh
./count.sh missing.log; echo "exit: $?"

INFO: 2
ERROR: 3
No such file: missing.log
exit: 1

Loops over files

for f in *.log; do
  echo "$f has $(wc -l < "$f") lines"
done

# read a file line by line
while IFS= read -r user; do
  echo "Creating home for $user"
done < users.txt

Use a glob like *.log rather than parsing the output of ls; globs handle spaces and odd characters correctly.

Recap

• Programs have stdin, stdout and stderr; pipes connect stdout to the next stdin.
• > overwrites, >> appends, 2>&1 merges errors into output.
• Exit code 0 is success; chain with && and ||.
• Quote your variables and start scripts with a shebang and set -euo pipefail.
• Run chmod +x once, then ./script.sh.

# Write your solution here