GNU sed (stream editor) is a clone of UNIX sed command line program from GNU operating system. GNU sed is almost always included in every GNU/Linux distribution (e.g. Ubuntu). While the most of computer users edit a text file only with a real time text editor like Notepad, UNIX family users can edit a text file by a stream editor like sed. This article gives you the examples of stream editing for text file(s) with GNU sed. Because of it is GNU sed it is universal, so if you don't use Ubuntu (but at least you use any GNU/Linux distro), you can practice all of these command lines correctly. If you use another family of sed (e.g. UNIX or BSD one), probably you will need to adapt some commands because they have some differences.
Checking sed Version
Type this on your Terminal:
sed --version
Example Output:
If you are using GNU/Linux, then your sed will produce this output indicating the origin of your sed (GNU operating system) and another important information:
sed (GNU sed) 4.2.2 ...
Create A Text File
To practice with sed, you need at least a text file. Create some sentences (do it with another text editor) inside a file named text.txt and place it in the /tmp directory. Enter yourself into /tmp then. We will work with sed there.
master@master:~$ cat /tmp/text.txt
I have UNIX.
I have GNU.
I have BSD.
I have GNU/Linux.
master@master:~$
Note: this text file used in this article contains 4 text lines and 3 blank lines without beginning and end blank lines. We will play these characteristics with sed.
1. Read A Text File
Example Command:
1. sed r text.txt
Example Output:
master@master:~$ sed r /tmp/text.txt
I have UNIX.
I have GNU.
I have BSD.
I have GNU/Linux.
master@master:~$
Explanation:
GNU sed has options, and GNU sed also has commands. Commands are some special arguments preceded with no minus (`-` or `--`) sign. In this first example, we use sed command to read: r. So the command synopsys is sed r <file_name>. This produces the same output like the command cat <file_name>. This command will not edit anything.
2. Find & Replace
Example Commands:
1. sed ‘s/UNIX/MINIX/g’ text.txt
2. sed ‘s/GNU/XNU/g’ text.txt
3. sed ‘s/GNU\/Linux/Ubuntu/g’ text.txt
4. sed -i ‘s/have/am/g’ text.txt
5. sed ‘s/have/am/g’ text.txt > new_text.txt
Example Output:
master@master:/tmp$ sed 's/GNU\/Linux/Ubuntu/g' text.txt
I have UNIX.
I have GNU.
I have BSD.
I have Ubuntu.
master@master:/tmp$
Explanation:
(1,2) This 's' command is probably the most used by sed users. We use s command to substitute string. In this example commands number 1-3, towards the text.txt file, we try to replace the original string UNIX with new string MINIX and GNU with XNU. You will notice that the s command template is s/…/…/g, with the first column /…/ is for the original string (or regex) and second column /…/ is for the replacement string. They are technically same with any GUI text editor Find & Replace system.
One most important thing of this sed and sed 's' command is about sed does not edit the original file directly. So you will see your original file (text.txt) does not change at all.
(3) Note that the example command number 3 uses a `\` (backslash) to escape a ‘/’ (slash) in the string `GNU/Linux`. We need to do this because sed understands a slash as a part of command, not a string, so you need to “escape it” (make it as string) with a backslash then sed will understand it as a part of string. As a result, this command changes the string GNU/Linux into Ubuntu.
If you don’t escape the slash with a backslash, then sed will be confused and it will produce error message such as “sed: -e expression #1, char 13: unknown option to `s'”.
(4) Example command number 4 shows how to edit directly the original file (just the same as using any GUI text editor) by using `-i` or `--in-place` option.
(5) Example command number 5 shows an alternative command, instead using `-i` option we use redirection (`>`) to create a new text file.
3. Find & Replace (‘g’ Flag)
Command Examples:
1. echo “UNIX UNIX UNIX” | sed ‘s/UNIX/GNU/g’
2. echo “UNIX UNIX UNIX” | sed ‘s/UNIX/GNU/’
Output Examples:
master@master:~$ echo "UNIX UNIX UNIX" | sed 's/UNIX/GNU/g'
GNU GNU GNU
master@master:~$ echo "UNIX UNIX UNIX" | sed 's/UNIX/GNU/'
GNU UNIX UNIX
master@master:~$
Explanation:
sed command ‘s’ (substitute) has a generic pattern ‘s/[REGEX][REPLACEMENT]/[FLAG]’. In this example, we see ‘g’ flag usage. Here, we edit a text stream (from GNU echo command) saying “UNIX UNIX UNIX”. This text contains a single line with three words (or, three columns) with the same word ‘UNIX’ inside them. We compare the two results, one with and one without ‘g’ flag. Notice the differences:
- With ‘g’ flag, every word matched (or, every column matched) in one line edited. So “UNIX UNIX UNIX” will be “GNU GNU GNU”.
- Without ‘g’ flag, only the first word matched (or, the first column matched) in one line edited. So “UNIX UNIX UNIX” will be “GNU UNIX UNIX”.
4. Find & Replace (UPPERCASE)
Command Examples:
1. sed ‘s/have/\U&/g’ text.txt
2. echo "linux xnu hurd mach" | sed 's/linux/\U&/g'
Output Examples:
(1)
master@master:/tmp$ sed 's/have/\U&/g' text.txt
I HAVE UNIX.
I HAVE GNU.
I HAVE BSD.
I HAVE GNU/Linux.
master@master:/tmp$
(2)
master@master:/tmp$ echo "linux xnu hurd mach" | sed 's/linux/\U&/g'
LINUX xnu hurd mach
master@master:/tmp$
Explanation:
(1) The key to change lowercase into uppercase here is `\U&`. The `\U` is a special sequence to change the [REPLACEMENT] into uppercase, and the literal `&` character refers to a whole pattern space. So a special sequence `\U&` means change every matched word into uppercase.
(2) Example number 2 shows sed changing into uppercase for a text stream from GNU echo command string. It is basically the same with the example number 1.
5. Find & Replace (lowercase)
Command Examples:
1. sed ‘s/UNIX/\L&/g’ text.txt
2. sed ‘s/GNU/\L&/g’ text.txt
3. echo “UNIX MINIX” | sed ‘s/MINIX/\L&/g’
Output Examples:
(1, 2)
master@master:/tmp$ sed 's/UNIX/\L&/g' text.txt
I have unix.
I have GNU.
I have BSD.
I have GNU/Linux.
master@master:/tmp$ sed 's/GNU/\L&/g' text.txt
I have UNIX.
I have gnu.
I have BSD.
I have gnu/Linux.
master@master:/tmp$
(3)
master@master:/tmp$ echo "MINIX" | sed 's/MINIX/\L&/g'
minix
master@master:/tmp$
Explanation:
(1) The key to change uppercase into lowercase here is `\L&`. This is the opposite special sequence to the previous `\U&`, in which `\L&` will turn a whole matched word into lowercase. So the input string “MINIX” will be “minix”.
6. Delete All Lines
Command Examples:
1. sed ‘d’ text.txt
2. sed -i ‘d’ text.txt
3. sed ‘d’ text.txt > new_text.txt
Output Example:
master@master:/tmp$ sed 'd' text.txt
master@master:/tmp$
Explanation:
Another sed useful command beside ‘r’ and ‘s’ is ‘d’ (delete). By invoking ‘d’, sed will read file content line by line, deleting them one line by one line. As the result, sed show nothing to the standard output.
7. Delete Particular Line
Command Examples:
1. sed ‘1d’ text.txt
2. sed ‘1,3d’ text.txt
3. sed ‘4,6d’ text.txt
Output Examples:
(1)
master@master:/tmp$ sed '1d' text.txt
I have GNU.
I have BSD.
I have GNU/Linux.
master@master:/tmp$
(2)
master@master:/tmp$ sed '1,3d' text.txt
I have BSD.
I have GNU/Linux.
master@master:/tmp$
(3)
master@master:/tmp$ sed '4,6d' text.txt
I have UNIX.
I have GNU.
I have GNU/Linux.
master@master:/tmp$
Explanation:
(1) sed accepts particular numbering for some commands, called address range. We type a number in front of a command, and sed interprets it as the address should be done with that command. In this command number 1, we invoke ‘1d’, it means delete only the line number 1 from the text file. As the result, the first line of text.txt is deleted. The line saying I have UNIX is deleted.
(2, 3) Beside accepts a particular line address, sed also accepts more than one line address range. This achieved by typing two numbers separated with a comma, such as we find in command number 2 and 3. Command ‘1,3d’ means delete (d) lines number one (1) until number three (3). As the result, sed will delete those three lines and showing only lines number 4 until the end.
8. Delete All Blank Lines
Command Example:
1. sed ‘/^$/d’ text.txt
Output Example:
master@master:/tmp$ sed '/^$/d' text.txt
I have UNIX.
I have GNU.
I have BSD.
I have GNU/Linux.
master@master:/tmp$
Explanation:
This command involves a regular expression (regex) to match every blank line, which is ^$. The syntax format is ‘/[regex]/d’ and the `[regex]` example is `/^$/d`. This will find a blank line, delete it, then repeat that process for any remaining lines available. As a result, we see the text output now has no blank lines.
9. Reverse Delete Blank Lines
Command Example:
1. sed ‘/^$/!d’ text.txt
Output Example:
master@master:/tmp$ sed '/^$/!d' text.txt
master@master:/tmp$
Explanation:
Reverse delete blank lines means delete all non-blank lines. This command does the same as deleting all blank lines (/^$/ = match every blank line) but reverse the sed delete command (`d`) by adding exclamation mark (`!`). As a result, this command delete all text lines, showing only the 3 blank lines.
10. Delete Line Containing Particular String
Command Example:
1. sed ‘/UNIX/d’ text.txt
Output Example:
master@master:/tmp$ sed '/UNIX/d' text.txt
I have GNU.
I have BSD.
I have GNU/Linux.
master@master:/tmp$
Explanation:
This command shows how to match a string and delete the line containing that string. Example command ‘/UNIX/d’ means delete (`d`) any line containing the string `UNIX`. As the result, the first line of text is deleted.