ShellScripting

Shell Scripting

Define Variables

1
2
3
$ varname=varvalue
$ echo $varname
varvalue

Notice that white space is reserved in bash for separate arguments

1
2
$ varname = varvalue
command not found: varname

This is interpreted as call program varname with parameter = and varvalue

quote marks

  • For literal strings, '' and "" are the same.
  • For the rest of the thing, "" will evaluate its content while '' remain literal string
1
2
3
4
$ echo 'Value is $varname'
Value is $varname
$ echo "Value is $varname"
Value is varvalue

Some reserved “key words”

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
# $0 $1 $2 ... $9 are argv[0], argv[1], ...
$ echo $0
bash
$ cat myscript
#!/bin/bash
echo $0
echo $1
echo $2
$ ./myscript hello world
./myscript
hello
world
# $? reserved for return value
$ echo hello
hello
$ echo $?
0
$ eecho
command not found: eecho
$ echo $?
127
# $_ reserved for the last argument of the previous cmd
$ echo hello\ world
hello world
$ echo $_
hello world
# !! reserved for representing last cmd, handy for scenario that cmd lack for permission
$ echo hello
hello
$ !!
hello
$ cat test.txt
cat: test.txt: Permission denied
$ sudo !!
OK
# $$ reserved for current pid
$ ps
PID TTY TIME CMD
543 ttys020 0:00.36 -zsh
$ echo $$
543
# $# reserved for the number of arguments (like argc)
$ cat myscript
#!/bin/bash

echo $#
echo $$
$ ./myscript 1 2 3
3
2075
# $@ reserved for all arguments, usually used in script for loop
$ cat myscript
#! /bin/bash
echo $@
$ ./myscript 1 2 3
1 2 3

Boolean operation and short-circuited

1
2
3
4
5
6
7
8
9
$ true
$ echo $?
0
$ false
$ echo $?
1
$ true || echo hi
$ false || echo hi
hi

Semicolon

Semicolons are used to concatenate cmds and they will be excuted no matter what happens

1
2
3
$ fwofsdhgohwegp ; echo hi
command not found: fwofsdhgohwegp
hi

Parenthese

1
2
3
4
5
6
7
8
9
10
11
# get the result of a cmd
$ res=$(ls)
$ echo -e "We have files as following\n$res"
We have files as following
file1
file2
file3
....
# get the result of a cmd and store it in a temp file
$ <(pwd)
permission denied: /dev/fd/11

/dev/null

Special device on Unix which will discard all things recieved

1
2
3
4
5
6
7
8
9
10
11
12
13
# redirect stdout
$ echo "fwfhowef" > /dev/null

# redirect stderr
$ echo "fefowihfo" 2>/dev/null
fefowihfo
# beware of white space
$ echo "sdfowef" 2 > /dev/null
$
$ ejowejf
command not found: ejowejf
$ ejowejf 2>/dev/null
$

Comparison and test

1
$ man test

Globbing

1
2
3
4
5
# * stand for any characters
$ ls *.md
# ? stand for exact one character
$ ls /bin/????
/bin/bash /bin/date .....

Group

1
2
3
4
5
6
7
8
# {} can expand into group
$ echo hello{,world,sam}
hello helloworld hellosam
$ echo image{.jpg,.png}
image.jpg image.png
# They are like python nested for loop, or to be more precise, the Cartesian product
$ echo folder{1,2}/file{,4,5}
folder1/file folder1/file4 folder1/file5 folder2/file folder2/file4 folder2/file5

Range

1
2
3
4
5
# ..
$ echo {a..z}
a b c d e f g h i j k l m n o p q r s t u v w x y z
$ echo {1..10}
1 2 3 4 5 6 7 8 9 10

Check your shell script

Debug shell script is hard

shellcheck will help you a lot

Shebang

You can tell bash how to run the script by add a shebang line at the beginning of the script

1
#!/bin/bash
1
#!/bin/python

However, the path you specify might be incorrect, so in order to make it more portable

1
#!/usr/bin/env python

Using env to find the interpreter will be a better way

Community-driven man

man can give you tuns of information which is hard to read, there is a program call tldr serve as alternative

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ apt install tldr
$ tldr grep

grep

Find patterns in files using regular expressions.
More information: <https://www.gnu.org/software/grep/manual/grep.html>.

- Search for a pattern within a file:
grep "search_pattern" path/to/file

- Search for an exact string (disables regular expressions):
grep --fixed-strings "exact_string" path/to/file

....

Search things

1
2
3
$ find --help
$ locate --help
$ updatedb
  • find go through the file system to search for the file
  • locate search a cached database for you target
  • updatedb can update the database for locate
Perform further operation on results of find

-exec utility [argument …] ;

  • The expression must be terminated by a semicolon (;). If you invoke find from a shell you may need to quote the semicolon if the shell would otherwise treat it as a control operator.
  • If the string {} appears anywhere in the utility name or the arguments it is replaced by the pathname of the current file.
  • Utility will be executed from the directory from which find was executed.
  • Utility and arguments are NOT subject to the further expansion of shell patterns and constructs.

this argument is very useful, see more at man find

1
2
3
4
$ find . -name '*.md' -exec echo "[{}]" \;
[./file.md]
[./f.md]
[./some.md]

Redo previous cmds

History

1
2
$ history
$ history 1 | grep ls

Ctrl-R

  • Enter your target word
  • keep pressing ctrl-R

Arithmetic

The arithmetic expansion can be performed using the double parentheses ((...)) and $((...)) or with the let built-in command.

1
2
3
4
5
6
7
8
9
10
$ var=0
$ let var+=1
$ echo $var
1
$ let "var += 1"
$ echo $var
2
$ let var++
$ echo $var
3
1
2
3
4
5
6
$ ((var += 10))
$ echo $var
13
$ ((var--))
$ echo $var
12

Construct arguments from stdin

using the xargs tools