103.4 Use streams, pipes and redirects

Introduction

This lesson is about moving command input and output around.

The main question is:

How do you send output to a file, connect one command to another, or separate normal output from error messages?

What you should be able to do after this lesson:

• Redirect stdout, stderr, and stdin.
• Use pipes.
• Use tee.
• Use xargs.

The Three Standard Streams

Every command usually works with:

• stdin = standard input
• stdout = standard output
• stderr = standard error

Think of them as three data channels.

Redirecting Output

Write stdout to a file

ls > files.txt

Append stdout to a file

date >> log.txt

Redirect stderr

find /root 2> errors.txt

Ignore errors completely

command 2>/dev/null

Redirect stdout and stderr together

command > all.txt 2>&1

Redirecting Input

wc -l < file.txt

This tells the command to read from a file as input.

Pipes

Pipes connect stdout from one command to stdin of another.

ls -l | less
cut -d: -f1 /etc/passwd | sort

Example output from cut -d: -f1 /etc/passwd | sort | uniq | head -n 8:

_talkd
alpm
avahi
bin
brltty
clickhouse
daemon
dbus

What this shows:

• the first command produces text
• the next commands sort it and remove duplicates
• this is the main idea behind shell pipelines

This is one of the most important command-line patterns in Linux.

tee: See Output and Save It Too

Normally, redirection sends output only to a file. tee lets you see it on screen and save it.

ls -l | tee listing.txt

What this looks like in practice:

• the same output appears on screen
• and a copy is written to listing.txt

Append mode:

date | tee -a build.log

xargs: Turn Input into Command Arguments

Sometimes one command prints a list and another command expects arguments. xargs bridges that gap.

Example:

find . -name "*.log" | xargs -r ls -l

Important beginner warning:

• xargs becomes dangerous when the receiving command is destructive
• inspect the list first before removing or changing anything

Destructive form you may see later:

find . -name "*.log" | xargs rm

When file names may contain spaces, a safer advanced pattern is:

find . -name "*.log" -print0 | xargs -0 -r ls -l

Common Patterns

Errors only

command 2> errors.txt

Normal output only

command > output.txt

Both normal output and errors

command > all.txt 2>&1

Build a pipeline

cat file.txt | sort | uniq

Append while keeping screen output

command | tee -a output.log

Practice Step by Step

1) Save a command result to a file

1. Run: ls -la > listing.txt
2. Open the file: cat listing.txt

2) Show and save output at the same time

1. Run: ls -la | tee listing.txt
2. Notice that you see the output and also keep a copy.

3) Build a simple pipeline

1. Run: cut -d: -f1 /etc/passwd | sort | uniq
2. Read it as: extract usernames -> sort them -> collapse duplicates.

Cheat Sheet

• > = write stdout to file
• >> = append stdout to file
• 2> = write stderr to file
• 2>&1 = merge stderr into stdout
• < = read stdin from file
• | = pipe one command into another
• tee = show output and save it
• xargs = turn stdin into command arguments