Previously, we provided the first and second articles about GNU sed command line examples.
Now it is the third article covering combination between sed and bash to do
daily life jobs. We begin to focus in daily life jobs in this
episode. This episode is a continuation of our previous articles. So
if you miss something here, we suggest you to read the two
previous episodes.
Text Examples
text6.txt:
gnu gnu gnu gnu unix
bsd bsd bsd bsd unix
unix unix unix unix
minix minix minix
unix
As previous articles go, save this file in your /tmp and perform all sed commands there.
21. Run Multiple Commands
Command Example:
sed -e 's/gnu/\U&/g'
-e 's/bsd/\U&/g' text6.txt
Output Example:
master@master:/tmp$
sed -e 's/gnu/\U&/g' -e 's/bsd/\U&/g' text6.txt
GNU GNU GNU GNU unix
BSD BSD BSD BSD unix
unix unix unix unix
minix minix minix
unix
master@master:/tmp$
Explanation:
We see the only
different here compared to our previous examples, is, the `-e`
option. This option handles one command or script of sed, so putting
two of this options means doing two sed commands simultaneously. You
may put more `-e` option to do more commands. See, in this example
there are two `s///g` command sequences. See the result, there are
two lines result (uppercased strings) from the two commands.
22. Run Multiple Commands (Simpler)
Command Example:
sed 's/gnu/\U&/g;
s/bsd/\U&/g' text6.txt
Output Example:
master@master:/tmp$
sed 's/gnu/\U&/g; s/bsd/\U&/g' text6.txt
GNU GNU GNU GNU unix
BSD BSD BSD BSD unix
unix unix unix unix
minix minix minix
unix
master@master:/tmp$
Explanation:
This example is
simpler compared the number 21 example. It has the same result, but
without the `-e` option. Instead, this commands make use of (`;`)
character to separate two commands inside a pair of single quotes (`’
’`).
23. Combine sed with bash Looping Commands
Command Examples:
for i in {6..8}; do
sed '/bsd/d' "text$i.txt"; echo ""; done;
Output Examples:
(1)
master@master:/tmp$
for i in {6..8}; do sed '/bsd/d' "text$i.txt"; echo "";
done;
gnu gnu gnu gnu unix
unix unix unix unix
minix minix minix
unix
vms vms vms
unix dunix ultrix
irix tru64
hp-ux sunos solaris
xnu hurd linux
(2)
master@master:/tmp$
for i in {6..8}; do sed '/bsd/!d' "text$i.txt"; echo "";
done;
bsd bsd bsd bsd unix
unix bsd unix unix
gnu gnu gnu bsd
minix minix minix
minix bsd
bsd darwin
master@master:/tmp$
Explanation:
This is the first
time within all episodes we use combination of GNU sed and GNU bash.
Here, we make use of bash `for` command for looping. For the first
timer, those two examples above perhaps makes no sense. But if you
look one command in a programming manner you will understand it
easier. For example, the first command above can be written like
this:
for i in {6..8};
do
sed '/bsd/d'
"text$i.txt";
echo "";
done;
This code is GNU
bash scripting language. To read it properly, we can analyze it one
by one:
-
The first line sets a variable named `i` to contain numbers from 6 until 8 (hence, there would be 3 loops). Every loop will have `i` to contain one number, so the first loop i = 6, second loop i = 7, and third loop i = 8.
-
The second line is a starting point of loop, to wrap the actual command line to do the actual job.
-
The third line is the actual code. This is your sed command to do deleting any line containing “bsd” string. This is the actual job.
-
The fourth line is also the command line to do the job, like the sed line, but it is just secondary. The most important command is still the third line. This fourth line does the job to give newlines so the final output will be easier to read.
-
The fifth line is an ending point of loop, pairing with the second line (so there are do and done).
-
Conclusion: this code means do the sed command line, but do loop it 3 times, for the number 6 until number 8, towards the file text6.txt then text7.txt then text8.txt. The `i` variable does the job for substituting number in every file name.
-
These five lines can be written in a single line like the first command example here.
24. Delete Contents of Multiple Files’ (bash Looping)
Command Example:
for i in {6..8}; do
sed -i 'd' text$i.txt; done
Output Examples:
master@master:/tmp$
for i in {6..8}; do sed -i 'd' text$i.txt; done
master@master:/tmp$
for i in {6..8}; do cat text$i.txt; done
master@master:/tmp$
Explanation:
This example is just
the same with the example number 23. It is also a bash looping code
in single line. This code means do the sed command line, but do loop
it three times, for the number 6 until 8, towards the file text6.txt,
text7.txt, text8.txt. The sed command is just ‘d’, it means
delete the whole content of file. So this single line command do
erase all the contents of text6.txt, text7.txt, text8.txt.
WARNING: this sed `-i` option and 'd' command result in deleting contents of the files. Be careful, make sure you don't delete important contents.
25. Rename Multiple Files (bash Looping)
Command Examples:
-
for i in *.txt; do mv "$i" "`echo $i | sed "s/text/flext/g"`"; done
-
for i in *.txt; do mv -v "$i" "`echo $i | sed "s/^[0-9]*[0-9]//g"`"; done
Output Examples:
(1)
master@master:/tmp$
ls
text6.txt
text7.txt
text8.txt
master@master:/tmp$
for i in *.txt; do mv "$i" "`echo $i | sed
"s/text/flext/g"`"; done
master@master:/tmp$
ls
flext6.txt
flext7.txt
flext8.txt
master@master:/tmp$
(2)
master@master:/tmp$
touch 1lion.txt 2tiger.txt 3giraffe.txt 4monkey.txt 5eagle.txt
6deer.txt
master@master:/tmp$
ls
1lion.txt
2tiger.txt
3giraffe.txt
4monkey.txt
5eagle.txt
6deer.txt
master@master:/tmp$
for i in *.txt; do mv -v "$i" "`echo $i | sed
"s/^[0-9]*[0-9]//g"`"; done
'1lion.txt' ->
'lion.txt'
'2tiger.txt' ->
'tiger.txt'
'3giraffe.txt' ->
'giraffe.txt'
'4monkey.txt' ->
'monkey.txt'
'5eagle.txt' ->
'eagle.txt'
'6deer.txt' ->
'deer.txt'
mv: 'flext.txt' and
'flext.txt' are the same file
master@master:/tmp$
Explanation:
First example do the
rename loops towards some .txt files with number in every beginning
of their file names.
-
The loop is basically mv command, but the argument of this mv is handled by echo and sed.
-
The echo command print the $i. This $i is a notation of the `i` variable content. Command echo print this in every loop.
-
The sed command receive the echo output via the `|` (pipe). So sed processes it, find and replace the “text” string into “flext” string.
-
This loop repeated until the filename text6.txt become flext6.txt, so on until text8.txt become flext8.txt.
-
You should pay attention to the usage or quotation marks (“ ”) and the apostrophe marks (` `). Place them properly just like the example.
26. Delete Multiple Files (bash Looping)
Command Examples:
-
for i in *.txt; do rm -v "`echo $i | sed "s/^[0-9]*[0-9]//g"`"; done
-
for i in *.txt; do rm -v "`echo $i | sed -n "/^[0-9]*[0-9]/p"`"; done
Output Examples:
(1)
master@master:/tmp$
ls -w 1 *.txt
1lion.txt
2tiger.txt
3giraffe.txt
4monkey.txt
5eagle.txt
6deer.txt
deer.txt
eagle.txt
flext.txt
giraffe.txt
lion.txt
monkey.txt
tiger.txt
master@master:/tmp$
for i in *.txt; do rm -v "`echo $i | sed "s/^[0-9]*[0-9]//g"`";
done
removed 'lion.txt'
removed 'tiger.txt'
removed
'giraffe.txt'
removed 'monkey.txt'
removed 'eagle.txt'
removed 'deer.txt'
master@master:/tmp$
ls
1lion.txt
2tiger.txt
3giraffe.txt
4monkey.txt
5eagle.txt
6deer.txt
(2)
master@master:/tmp$
ls -w 1 *.txt
1lion.txt
2tiger.txt
3giraffe.txt
4monkey.txt
5eagle.txt
6deer.txt
elephant2.txt
giraffe4.txt
lion1.txt
monkey3.txt
tiger5.txt
master@master:/tmp$
for i in *.txt; do rm -v "`echo $i | sed -n "/^[0-9]*[0-9]/p"`";
done
removed '1lion.txt'
removed '2tiger.txt'
removed
'3giraffe.txt'
removed
'4monkey.txt'
removed '5eagle.txt'
removed '6deer.txt'
master@master:/tmp$
ls -w 1 *.txt
elephant2.txt
giraffe4.txt
lion1.txt
monkey3.txt
tiger5.txt
master@master:/tmp$
Explanation:
This example number
26 basically the same with number 25 except mv being rm. Here, we try
to delete beginning-numbered file names and non-beginning numbered
file names. We use substitution at the first example, and deletion at
the second example.
WARNING: this rm command is already dangerous, combined with bash looping commands it would be extremely dangerous. Be careful, make sure you don't make any regex or syntax mistake. Please avoid using sudo if you don't know what will happen.
27. Change Slash Delimiters to Another Characters
Command Examples:
-
sed 's/unix/CHANGED/g' text6.txt
-
sed 's@unix@CHANGED@g' text6.txt
-
sed 's!unix!CHANGED!g' text6.txt
-
sed 's?unix?CHANGED?g' text6.txt
Output Examples:
master@master:/tmp$
sed 's/unix/CHANGED/g' text6.txt
gnu gnu gnu gnu
CHANGED
bsd bsd bsd bsd
CHANGED
CHANGED CHANGED
CHANGED CHANGED
minix minix minix
CHANGED
master@master:/tmp$
sed 's@unix@CHANGED@g' text6.txt
gnu gnu gnu gnu
CHANGED
bsd bsd bsd bsd
CHANGED
CHANGED CHANGED
CHANGED CHANGED
minix minix minix
CHANGED
master@master:/tmp$
sed 's!unix!CHANGED!g' text6.txt
gnu gnu gnu gnu
CHANGED
bsd bsd bsd bsd
CHANGED
CHANGED CHANGED
CHANGED CHANGED
minix minix minix
CHANGED
master@master:/tmp$
sed 's?unix?CHANGED?g' text6.txt
gnu gnu gnu gnu
CHANGED
bsd bsd bsd bsd
CHANGED
CHANGED CHANGED
CHANGED CHANGED
minix minix minix
CHANGED
master@master:/tmp$
Explanation:
Using slash for sed
command regex delimiters is sometimes confusing. Especially, if your
regex contains many slashes too. The solution is you must use
delimiters other than slash. As long as they are used consistently,
you may use characters like at, exclamation mark, question mark, or
underscore to replace the slash. The examples shown in this number 27
have the same commands and the same results. The only difference is
the delimiter (`@`, `!`, and `?`).
28. Adjust Line Spacing (Single)
Command Examples:
Output Examples:
master@master:/tmp$
cat text6.txt
gnu gnu gnu gnu unix
bsd bsd bsd bsd unix
unix unix unix unix
minix minix minix
unix
master@master:/tmp$
sed ‘G’ text6.txt
gnu gnu gnu gnu unix
bsd bsd bsd bsd unix
unix unix unix unix
minix minix minix
unix
Explanation:
This single ‘G’
command of sed causes addition one blank line for every line.
29. Adjust Line Spacing (Double)
Command Example:
Output Example:
master@master:/tmp$
sed 'G;G' text6.txt
gnu gnu gnu gnu unix
bsd bsd bsd bsd unix
unix unix unix unix
minix minix minix
unix
master@master:/tmp$
Explanation:
This double `G`
command (separated with a (`;`) character) gives every line addition
of two blank lines.
30. Find & Replace (Lines Range)
Command Examples:
Output Examples:
master@master:/tmp$
sed '1,3 s/unix/CHANGED/g' text6.txt
gnu gnu gnu gnu
CHANGED
bsd bsd bsd bsd
CHANGED
CHANGED CHANGED
CHANGED CHANGED
minix minix minix
unix
master@master:/tmp$
Explanation:
You may determine
the range of lines for sed command, not only “the coordinate”.
See the two numbers separated with comma (`,`) above. That command
means do substitution command only for line number 1 until line
number 3.