Linux unixShellsprograming学习笔记.doc

Linux&unix Shells programing学习笔记

一、 introduction to unix/linux shells
1、 what is unix ? what is Linux a Little History

2、

3、 Definition and function of a shell
The kernel、The shell、and User

4、 Unix shell
Boume shell 、c shell、 kom shell
5、 Linux shell
Bash
6、 Cat /etc/shells(列出系统中所包含的shell)
7、 Echo $SHELL(列出目前正在使用的shell)
8、 Cat /etc/passwd(查看用户所对应的shell)
9、 What is posix?
Portable operating system interface of unix
10、 History of the shell

11、 Uses of the shell
A、 when running interactivery, one of the major functions of a shell is interpret commands entered at the command-line prompt.(交互式下解释命令行输入的命令)
B、 Another important function of the shell is customize the user’s environment, normally done in shell initialization files.（定制用户的环境）
C、 The shell can also be used as an interpreted programming language.（作为解释性的变成语言）
12、 Responsibilities of the shell
The shell is ultimately responsible for making sure that any commands typed at the prompt get executed properly. Included in those responsibilities are:
A、 reading input and parsing the command line（读取输入并解析命令行）
B、 evaluation special characters, such as wildcards and the history character（替换特殊字符）
C、 setting up pipes, redirection, and background processing（设置管道、重定向和后台处理）
D、 handing signals（处理信号）
E、 setting up programs for execution（程序执行的相关设计）
13、 System Startup and the login shell
A、 parsing the command line
when you type a command at the prompt, the shell read a line of input and parses the command line, breaking the line into words, called tokens. Tokens are separated by spaces and tabs and the command line is terminated by a newline. The shell then checks to see whether the first word is a built-in command or an executable.
l History substitution is performed (if applicable).(执行命令替换)
l Command line is broken up into tokens, or words.（命令行分解为tokens）
l History is update (if applicable).(命令更新)
l Quotes are processed.（引入的处理）
l Alias substitution and functions are defined (if applicable)（别名替换，和函数的定义）
l Redirection , background, and pipes are set up.（设置重定向、后台管理和管道）
l Variable substitution( $user, $name, etc.) is performed.（执行变量的替换）
l Command substitution (echo “Today is ‘data’ ”) is performed.（执行命令替换）
l Filename substitution, called globbing (cat abc.??, rm *.c, etc.) is performed.（执行globbing的文件替换）
l Command is executed.（执行命令）
B、 Type of commands
l Aliases
l Keywords
l Functions
l Built-in commands
l Executable programs

C、 Processes and the shell
1、 what processes are running?
l the ps command
l the pstree/ptree command
2、 Whet are system calls?
3、 Creating processes
D、 Environment and inheritance
1、 ownership and permissions
2、 the file creation mask
3、 changing permissions and ownership
l chmod command
l chown command
4、 the working directory
5、 variables
E、 redirection and pipes
1、 file descriptors （0：标准输入 1：标准输出 2：标准错误输出）
2、 redirection
3、 pipes
F、 the shell and signals
A. the signal can be ignored
B. the processes can be stopped
C. the processes can be continued
D. the signal can be caught by a function defined in the program.
G、 Executing commands from scripts
Go into your favorite editor and type in a set of unix/Linux commands, one per line. Indicate what shell you want by placing the pathname of the shell after the #! On the first line.
Save your file and turn on the execute permissions so that you can run it.
Execute your program just as you would any other unix/Linux command.

二、 Shell programming Quick Start
1、 Taking a peek at shell scripts
A. The c shell and TC shell emulate the c language syntax whereas Bourne shell is based on an older programming language called Algol.
B. The bash and korn shells tend to be combination of both the Bourne and C shells, although these shells originated from the Bourne shell
2、 Sample scripts: Comparing the Major shells
A. Before Getting Started
You must have a good handle on UNIX/Linux commands. If you do not know the basic commands, you cannot do much with shell programming.
B. VI
3、 The c and TC shell syntax and constructs
A、 The shbang line
Example: #!/bin/csh or #!/bin/tcsh
B、 Comments
Example: #This is a comment.
C、 Wildcards
Example:
rm * ; ls ??;cat file[1-3];!!
Echo “How are you?”
Echo Oh boy\!
D、 Displaying output
Example
Echo “Hello to you\!”
E、 Local variables
Set variables_name = value
Set name=”Tom Jones”
F、 Global variables
Setenv VARIABLE_NAME value
Setenv PRINTER shakespeare
G、 Extracting values from variables
Echo $variable_name
Echo $name
Echo $PRINTER
H、 Reading user input
Echo “what is your name?”
Set name = $<
I、 Arguments
Scriptname arg1 arg2 arg3 …
Echo $1 $2 $3
Echo $*
J、 Arrays
Set word_list = (word1 word2 word3)
Echo $word_list[2]
Echo $word_list[1]
Echo $word_list or $word_list[*]
K、 Command substitutioin
Set variable_name = `command`
Echo $variable_name
L、 Arithmetic
@n =5+5
Echo $n
M、 Operators

N、 Conditional statements
O、 Loops
P、 File testing

4、 The bourne shell syntax and constructs
A、 the shbang line
B、 comments
C、 wildcards(通配符)
D、 displaying output
E、 local variables
variable_name = value
F、 global variables
VARIABLE_NAME=value
Export VARIABLE_NAME
G、 Extracting values from variables
H、 Read user input
Echo “what is your name?”
Read name
Read name1 name2 …
I、 Arguments (positional parameters)
$scriptname arg1 arg2 arg3 …
Echo $1 $2 $3
J、 Arrays (positional parameters)
Set word1 word2 word3
Echo $1 $2 $3
K、 Command substitution
Variable_name = `command`
Echo $variable_name
L、 Arithmetic
N = `expr 5+5`
Echo $n
M、 Opterators

5、 The korn shell Constructs
A、 the shbang line
#!/bin/ksh
B、 comments
#this program will test some files
C、 wildcards
rm *;ls ??;cat file[1-3];
echo “How are you?”
D、 displaying output
echo “who are you?”
print “how are you?”
E、 local variables
variable_name = value
typeset variable_name = value
name = “John Doe”
F、 global variables
export VARIABLE_NAME = value
export PATH=/bin:/usr/bin;
G、 extracting values from variables
echo $variable_name
echo $name
H、 read user input
print –n “what is your name?”
read name
I、 arguments
$scriptname arg1 arg2 arg3 …
Echo $1 $2 $3 …
J、 arrays
set apples pears peaches
echo $1 $2 $3
K、 arithmetic
typeset –i
variable_name
L、 command substitution
variable_nae = `command’
variable_name = $(command)
echo $variable_name
M、 operators

N、 conditional statements
O、 loops
P、 file testing

6、 the bash shell constructs

三、 regular expressions and pattern matching
1、 definition and example
what is a regular expression? A regular expression is just a pattern of characters used to match the same characters in a search.

Vim 编辑器：
:1,$s/kernel/admin/g (替换文中的kernel为admin，做一个全局替换)
撤销操作：按键盘上的u；
2、 regular expression metacharacters
vim 编辑器：
:1,$s/\<[Kk]ernel\>/admin/g (替换稳重的K/kernel为admin，做一个全局替换)

3、 combining regular expression matacharacters

四、 The grep family
1、 the meaning of grep
l the name grep can be traced back to the editor. If you invoked that editor and wanted to search for a string, you would type at the ex prompt:
n /pattern/p
l The first line containing the string pattern would be printed as “p” by the print command. If you wanted all the lines that contained pattern to be printed, you would type:
n g/pattern/p
l when g precedes pattern, it means “all lines in the file,” or “perform g global substitution.” Because the search pattern is called a regular expression, we can substitute RE for pattern and the command reads
n g/RE/P
2、 how grep works
l the grep command searches for a pattern of characters in a file or multiple files. If the pattern contains whitespace, it must be quoted. The pattern is either a quoted string or a single word, and all other words following it are treated as filenames. Grep sends its ouput to the screen and does not change or affect the input file in anyway.
l Format
Grep word filename filename
l Metacharacters
3、 Metacharacters
A metacharacter is a character that represents something other than itself. ^ and $ are examples of metacharacters.

4、 Grep and exit status
The grep command is very useful in shell scripts, because it always returns and exit status to indicate whether it was able to locate the pattern or the file you were looking for. If the pattern is found, grep returns an exit status of 0, indicating success; if grep cannot find the pattern, it returns 1 as its exit status; and if the file cannot be found, grep returns an exit status of 2
5、 Grep with pipes
Instead of taking its input from a file, grep often gets its input from a pipe
Etc. Ls –l | grep ‘^d’

6、 Fgrep
The fgrep command behaves like grep, but does not recognize any regular expression metacharacters as being special. All characters represent only themselves. A caret is simply a caret, a dollar sign is a dollar sign, and so forth.
7、 Linux and GNU grep

8、 Grep –E or egrep

五、 Sed the streamlined editor
1、 what is sed?
Sed is a streamlined, noninteractive editor. It allows you to perform the same kind of editing tasks used in the vi and ex editors.
2、 Version of sed
Linux uses the GNU version of sed, copyrighted by the free software foundation. This version is almost identical to the sed provided by standard UNIX distributions.
Sed –version

3、 How does sed work?

4、 Regular expressions
Sed, like grep, searches for patterns in files using regular expressions. Regular expressions are patterns of characters enclosed in forward slashes for searches and substitutions.

5、 Addressing
n Format
Sed ‘command’ filename(s)
n Example
Sed ‘1,3d’ myfile
Sed –n ‘/[Jj]ohn/p’ datafile
6、 Command and options

7、 How to modify a file with sed
Sed is a nondestructive editor. It will display the edits you make on your screen, but it will not change the file you are editing. To really reflect the edits in the file, you must redirect the output to another file, and then rename the orginal file
Example:
Sed ‘1,3d’ filex > temp
Mv temp filex
8、 Error messages and exit status
When sed encounters a syntax error, it sends a pretty straightforward error message to standard error, but if it cannot figure out what you did wrong, sed gets “garbled” which we could guess means confused. If the syntax is error-free, the exit status that sed returns to the shell is a zero for success and nonzero integer for failure.
9、 Metacharacters
Like grep, sed supports a number of special metacharacters to control pattern seaching.

10、 Sed example
n Printing: the p command
Sed ‘/north/p’ datafile
Sed –n ‘/north/p’ datafile
n Deleting: the d command
Sed ‘3d’ datafile
Sed –n ‘3,$d’ datafile
Sed ‘$d’ datafile
Sed ‘/north/d’ datafile
n Substitution: the s command
Sed ‘s/west/north/g’ datafile
Sed –n ‘s/^west/north/p’ datafile
Sed ‘s/[0-9][0-9]$/&.5/’ datafile
Sed –n ‘s/hemenway/jones/gp’ datafile
Sed –n ‘s/\(Mar\)got/\1ianne/p’ datafile
Sed ‘s#3#88#g’ datafile
n Range fo selected lines: the comma
Sed –n ‘/west/,/east/p’ datafile
Sed –n ‘5,/^northeast/p’ datafile
Sed ‘west/,/east/s/$/**/VACA**/’ datafile
n Multiple edits: the e command
Sed –e ‘1,3d’ –e ‘s/hemenway/jones/’ datafile
n Reading from files: the r command
Sed ‘/suan/r newfile’ datafile
n Writing to files: the w command
Sed –n ‘/north/w newfile’ datafile
n Appending: the a command
Sed ‘^north /a\-àBOOKBOOKEß--‘ datafile
n Inserting: the I command
Sed ‘/eastern/i\-àBOOBOOKEß-‘ datafile
n Changing: the c command
Sed ‘/eastern/c\ -à BOOBOOKEß-‘ datafile
n Next: the n command
Sed ‘/eastern/{n;s/AM/Archie/;}’ datafile
n Transform: the y command
Sed ‘1,3y/asdfgajasdflawehfihaiwehfihasd/ADSFASKDHFLIASFI/’ datafile
n Quit: the q command
Sed ‘5q’ datafile
Sed ‘/Lewis/{s/Lewis/Joseph/;q;}’ datafile
n Holding and getting: the h and g commands
Sed –e ‘/northeast/h’ –e ‘$G’ datafile
Sed –e ‘/WE/{h;d;}’ –e ‘/CT/{G;}’ datafile
Sed –e ‘/northeast/h’ –e ‘$g’ datafile
Sed –e ‘/WE/{h;d;}’ –e ‘/CT/{g;}’ datafile
n Holding and exchanging: the h and x command
Sed –e ‘/Patricia/h’ –e ‘/Margot/x’ datafile

11、 Sed scripting
n Sed script examples

n
n