/
e) Input and Output Redirection, Pipes, and Filters

e) Input and Output Redirection, Pipes, and Filters

27 Oct, 2020

Input and Output Redirection

We can change the behaviour of programs to redirect input from a file instead of the keyboard and write to a file instead of the screen. The '>' character is used to redirect output to a file and '<' to redirect input.

Redirection of program I/O
-bash-4.1$ echo Hello World > afile -bash-4.1$ cat afile Hello World -bash-4.1$ wc -l < wordsworth 25 -bash-4.1$

The wc -l command counts the number of lines typed. In this example we have redirected the input from a file.

Pipes

Pipes allow the output of one program to be fed into the input of another. The '|' is the pipe symbol.

This example counts the number of lines in a set of files. We write the output to a file. The file is then sorted, using the sort command, into ascending order to give us the file order by number of lines.

Count lines in a file
-bash-4.1$ ls carroll keats milton tennyson thomas wordsworth -bash-4.1$ wc -l * > linecount -bash-4.1$ cat linecount 733 carroll 423 keats 156 milton 11 tennyson 28 thomas 25 wordsworth 1376 total -bash-4.1$ sort -n -k 1 linecount 11 tennyson 25 wordsworth 28 thomas 156 milton 423 keats 733 carroll 1376 total -bash-4.1$

First we count the lines in each of the six file and redirect the output to a file. We then sort the file by the first column in numerical order.

A quicker and more efficient way without using the intermediary file is to use a pipe.

Line counting and sorting using a pipe
-bash-4.1$ wc -l * | sort -n -k 1 11 tennyson 25 wordsworth 28 thomas 156 milton 423 keats 733 carroll 1376 total -bash-4.1$

Filters

A filter is a program that transforms an input stream into an output stream. Almost all Linux programs do this. The pipe is used to connect the filters. Here is an example of finding all the user-names of people logged into the computer.

Find a list of users who has logged in
-bash-4.1$ last |more abs4 pts/14 gallifrey.york.a Thu Sep 11 08:47 still logged in jg757 pts/3 elecpc111.ohm.yo Thu Sep 11 01:51 still logged in dl792 pts/3 host-172-18-1-89 Wed Sep 10 23:10 - 23:34 (00:24) yx664 pts/9 :1001.0 Wed Sep 10 17:55 still logged in yx664 pts/7 :1001.0 Wed Sep 10 17:54 still logged in yx664 pts/5 :1001.0 Wed Sep 10 17:40 still logged in yx664 pts/13 :1001.0 Wed Sep 10 16:07 still logged in yx664 pts/10 :1001.0 Wed Sep 10 16:05 still logged in rm591 pts/14 mandle.york.ac.u Wed Sep 10 16:01 - 16:01 (00:00) yx664 pts/13 :1002.0 Wed Sep 10 15:56 - 16:03 (00:07) yx664 :1002 :1002 Wed Sep 10 15:54 - 16:04 (00:09) jdr500 pts/9 10.240.171.184 Wed Sep 10 15:53 - 17:14 (01:20) yx664 pts/7 :1001.0 Wed Sep 10 15:51 - 17:38 (01:47) yx664 pts/6 :1001.0 Wed Sep 10 15:47 - 16:05 (00:17) yx664 :1001 :1001 Wed Sep 10 15:45 still logged in rm591 pts/14 mandle.york.ac.u Wed Sep 10 15:36 - 15:38 (00:01) rm591 pts/14 mandle.york.ac.u Wed Sep 10 14:48 - 15:34 (00:45) rm591 pts/14 mandle.york.ac.u Wed Sep 10 14:32 - 14:35 (00:02) yx664 pts/13 :1001.0 Wed Sep 10 13:41 - 15:42 (02:00) yx664 pts/10 :1001.0 Wed Sep 10 13:17 - 15:44 (02:26) yx664 pts/9 :1001.0 Wed Sep 10 12:51 - 15:42 (02:50) yx664 pts/6 :1001.0 Wed Sep 10 12:21 - 15:42 (03:20) yx664 pts/9 :1001.0 Wed Sep 10 11:57 - 12:23 (00:26) --More-- -bash-4.1$ last | sort | uniq -w 9 | cut -c1-9 abs4 at568 dl792 ff555 fjg504 jdr500 jg757 kb1024 klcm500 ma725 msr514 pbc500 pbk1 rfle500 rm591 root sjb508 sl561 sy757 tao500 tm588 wtmp begi yw679 yx664 -bash-4.1$

the last command displays all users and the dates and times they have logged in. We then sort this,and  pass it through uniq, which removes duplicate lines by comparing only the first 9 characters. We then remove the remainder of the line after the username with the cut command.

Other ways of doing this
 -bash-4.1$ last | cut -c1-9 | sort | uniq abs4 at568 dl792 ff555 fjg504 jdr500 jg757 kb1024 klcm500 ma725 msr514 pbc500 pbk1 rfle500 rm591 root sjb508 sl561 sy757 tao500 tm588 wtmp begi yw679 yx664 -bash-4.1$