2.5 Wildcards, Pipes, and Redirection

Prerequisites: Environment Variables and PATH

Estimated time: 12 minutes

Exercise environment: Bash 5 on GNU/Linux

You are ready to continue when: you can explain who expands wildcards and which stream of data is passed through pipes

As logs and reports accumulate, typing every filename becomes tedious. Globs build several path arguments at once; pipes connect one program's standard output to another's standard input. Both can appear on the same line, but they operate on different things.

How the Shell Expands Wildcards

workdir=$(mktemp -d)
cd "$workdir"
touch report-{01,02,03}.txt photo-{a,b}.jpg notes.md

printf '%s\n' report-*.txt
printf '%s\n' photo-?.jpg
printf '%s\n' report-0[13].txt

Pattern	Meaning
*	Any string of arbitrary length
?	A single character
[13]	A character from a set
[a-z]	A character in a range, affected by locale settings

During filename expansion, * and ? normally do not cross / or match a leading . in a path component. Consequently, *.txt does not automatically include hidden files or files in subdirectories.

{01,02,03} is brace expansion in Bash and Zsh, not POSIX globbing. PowerShell has different expansion rules.

The behavior of unmatched patterns varies depending on the shell and options: Bash defaults to preserving literal text, while Zsh typically reports an error by default. Bash's nullglob or failglob can change the result again. Therefore, loops should avoid treating unmatched patterns as filenames:

for file in ./*.txt; do
  [ -e "$file" ] || continue
  printf 'would rename: %q -> %q\n' "$file" "${file%.txt}.bak"
done

After confirmation, you can change the preview loop into an execution loop:

for file in ./*.txt; do
  [ -e "$file" ] || continue
  mv -- "$file" "${file%.txt}.bak"
done

Here, only the name is changed. Changing .png to .jpg will not convert the image format; you must use a tool that can decode and re-encode images.

Standard Input and Output in Pipelines

printf '%s\n' alpha beta gamma | grep 'a' | wc -l
printf '%s\n' one two three | head -n 2

Pipelines default to passing only standard output, not standard error. Start by testing each segment with small inputs, then connect them step by step. This approach is easier for debugging than starting with a long command chain.

Do not use ps | grep name to check processes, as it may match grep itself. This is available in the GNU/Linux procps toolset:

pgrep -a bash || true

macOS's pgrep does not have the same -a semantics. Consider using pgrep -fl bash instead. Process names can often differ from expectations across platforms, and an empty result does not necessarily indicate a system error.

Do not parse ls to handle arbitrary filenames, and do not treat ls | wc -l as a reliable file count. Hidden files, line breaks, locale settings, and display formats can all introduce errors. When iterating through files, let find use NUL as a delimiter for filenames, then have Bash read them one by one:

count=0
while IFS= read -r -d '' file; do
  ((count += 1))
done < <(find . -type f -print0)
printf 'files=%d\n' "$count"

This example should count 6 files. Finishing the loop does not prove that find succeeded: a failure inside process substitution does not automatically become the loop's exit status. Check permission errors when processing real directories. Process substitution < <(...) and read -d are Bash syntax. NUL cannot appear in Unix filenames, so it safely distinguishes paths with spaces or line breaks. If writing cross-shell scripts, consider using tools or languages explicitly supported on the target platform.

Redirect Order and Pipeline State

Here, command is a placeholder for an actual program. These lines illustrate redirection rules rather than a complete experiment:

command >out.log 2>err.log
command >all.log 2>&1
command 2>&1 >out.log

Redirects are processed from left to right. The second line duplicates standard output after it points to all.log, sending standard error there too. The third line duplicates the original standard output first, then redirects only standard output to the file.

Bash defaults to using the exit status of the last command in the pipeline. Scripts that wish to see failures at any stage can enable this:

set -o pipefail

However, head may exit early, potentially causing upstream commands to receive SIGPIPE. Under pipefail, this should be considered when determining whether the failure is truly business-related.

cd
rm -r -- "$workdir"

→ Next: Terminal Practical Exercise