We discussed selected Linux commands. These commands will assist you to become familiar with some "first aid kit" or "survival" commands that will assist you in developing scripts in further chapters. The next section will discuss how to become familiar with the terminal and other tools. Finally, the last section contains information relating to creating scripts. The next chapter, i.e., Ch. 1. Intro to Scripting discusses the details on how to implement a script. The differences between a program and a script and how important are to develop a script these days in the context of software engineering, administration tools, and secure scripting (i.e., in particular for forensics purposes).
Setting up your Linux Environment
There are multiple ways to set up your Linux environment and many distributions of Linux as well. You can click here to see some of your options. If you are in my class, please visit the link provided here.
The terminal and location.
Connect to the terminal and explore the file system. The file system is the structure of the operating system, in this case, Linux, that contains a hierarchy of directory and files that can be access and use by the computer user and another group of users.
The very first command that you should learn is the pwd, which means "print working directory". the pwd will display the current location that the current user is located in the file directory. it displays something like this:
cservin$ pwd
/Users/cservin
We would like to list the current files that are located in the directory, therefore we use the command 1s.
Christians-MacBook-Pro-2:~ cservin$ ls
Applications Movies
Applications (Parallels) Music
Desktop Parallels
Documents Pictures
Downloads Public
Dropbox VirtualBox VMs
Google Drive anaconda3
Google Drive alias get-pip.py
Google Drive alias 2 jython2.7.0
Google Drive alias 3 logo.png
Library
Christians-MacBook-Pro-2:~ cservin$
listing the files from a file directory is useful to recognize what are the elements that you are working with. However, occasionally, we need more details than just the names. The command 1s -1 will retrieve more detail information about the files in the current directory.
christians-MacBook-Pro-2:~ cservin$ ls -l
total 11952
drwx------ 8 cservin staff 256 Nov 30 2017 Applications
drwx------@ 4 cservin staff 128 Jan 17 13:17 Applications (Parallels)
drwx------+ 29 cservin staff 928 Jan 26 20:51 Desktop
drwx------+ 21 cservin staff 672 Jan 26 11:51 Documents
drwx------+ 18 cservin staff 576 Jan 26 11:47 Downloads
drwx------@ 11 cservin staff 352 Aug 30 00:24 Dropbox
drwx------@ 34 cservin staff 1088 Jan 24 13:55 Google Drive
drwx------@ 82 cservin staff 2624 Aug 31 12:02 Library
drwx------+ 5 cservin staff 160 Sep 3 2018 Movies
drwx------+ 7 cservin staff 224 Sep 3 2018 Music
drwx------ 3 cservin staff 96 Nov 30 2017 Parallels
drwx------+ 10 cservin staff 320 Sep 3 2018 Pictures
drwxr-xr-x+ 5 cservin staff 160 May 13 2015 Public
drwxr-xr-x 7 cservin staff 224 Jun 1 2015 VirtualBox Vm
drwxr-xr-x 22 cservin staff 704 Nov 24 2018 anaconda3
-rw-r--r-- 1 cservin staff 1661676 Nov 24 2018 get-pip.py
drwxr-xr-x 2 cservin staff 64 Dec 13 2015 jython2.7.0
-rw-r--r--@ 1 cservin staff 18068 Aug 18 2018 logo.png
Directories and Path Navigation
Linux provides a variety of useful commands that can be used for file manipulation and/or editing. Here is a list of the most popular commands that you might need to interact between your operating
system and your file management.
cat: To create a file from scratch and provide information, you can use this command
cat > newFileName
Once the file is created you can edit it right away. If you want to exit press Ctl + c.
If you want simply create a file, simply type:
> newFileName
cal: will display the calendar of specified month or year. For example:
cal feb 2016
Outputs:
February 2016
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29
cd: is used to change directory (cd) from one to another.
cd nameOfDirectoryToMove
In case you need to return to the previous directory you can use
cd ..
clear: will clean up the screen of the terminal.
clear
comm: command that will find common lines in two files
comm -12 file1.txt file2.txt
Where:
-1: suppresses col 1 (i.e. the unique lines in file1.txt)
-2: suppresses col 2 (i.e. the unique lines in file2.txt)
date: prints the current date
diff: by providing two files, the command diff will find the difference between both files and will print the differences in the screen.
echo: will be used to print in the screen information or it will be used to execute commands in the script.
head: will provide the FIRST 5 continued lines of a filename
head filename
history: this command will display the history of all the commands that you have used in the terminal. It is a good command that will help you keep track of the actions you performed or to audit a programmer activities
ls: will list the files and directories located in the current directory
ls
In case you want to list the files with details use -l after the command ls
ls –l
man: will provide you a manual regarding any program. If you are interested to learn about any programming tool use man in front of the tool you are interested to find.
man ls
mkdir: means make a directory, will create a directory
mkdir nameOfDirectory
more: will open the document only to READ only.
more filename
mv: means move, and will move a specified file(s) as a source to a destination. E.g.,
mv src dest
where src is the source file and dest is the destination directory.
nano: Use the nano to edit a document quickly and to modify an existing file
nano filename
pwd: stands for a print working directory, and is used to output the path of the current working directory.
rm: will removes (deletes) a specified file
sort: this command will sort the content of a file in ascending order.
tail: will provide the LAST 5 continued lines of a filename
tail filename
uniq: command that reports the lines that are duplicated in a file. Several options are provided for the command.
Finds lines with “hello” in file1.txt
$uniq “hello” file1.txt
Counts the number of occurrences the target “hello” appears in file1.txt
$uniq –c “hello” file1.txt
wc: means word count, will retrieve the many lines, words, and characters are in a specified file
These scripts would require preliminary actions such as create a file, change its attributes, and edit the file with specific commands.
Create a file
There are different ways to create a file through the terminal. Let us simply create a file as follows
> myscript
By listing the attributes specifically for the file we just create as follows ls –l myscript01 we obtain the following description:
-rw-rw-rw- 1 cservin cservin 0 Jun 4 16:24 myscript
The redirection operator > is used to “redirect” output from a source program into a destination. The redirection reads from left to right. To redirect standard output to a file, the “>” character is used like this:
source > destination
if the destination is a file, the output from the source will be store it into the destination file. If the destination file, does not exists, then it will be created with the standard output from the source.
If you want the new results to be appended to the file instead, use “>>” like this:
source >> destination
To create a file in the command line you can use the
cat > filename.txt
This command will allow you to create a file in the current directory. In addition, you can add information while creating it. To get out of the editing mode, press Ctrl + c.
To create, open or edit any file, Linux provides you a wide diversity of editors. The most basic editors are the nano, pico, or gedit.
Note: There are additional editors and tools that you can use such as vi or emacs. However, these require more detail instructions. They are considered to be advance editors and consoles.
In Linux, everything is either a file or a directory. Both have file permissions to either read, write, and execute for: user, group, or for everybody else.
In order to read, write, and/or execute our script, we must provide the proper rights to the targeted users by using the command chmod.
We can extract the ownership of a file by using the ls -l. For example:
ls -l
-rw-r--r-- 1 cservin staff 16 Jan 27 21:59 myscript
The file called myscript has the following ownership attributes -rw-r--r--
The first – provides the information of the file. In case the file is a directory, then a ‘d’ appears instead of the -.
The next three ---, represent the permissions of the user who wrote this file. In this case, we have: rw-, meaning that the user can read and write into the file myscript.
The followed three ---, corresponds the permissions to the group. In this case we have: r--, meaning that members of the group can only read myscript.
Finally, the last ---, corresponds to the permissions for the “everybody else”. In our case: r--, means that everybody else can read the file only.
Depending on your policies that you want to establish on the files you can change the permissions accordingly. On Linux there are several ways to set up the permissions on a file based on the user (u), group (g), and other(o), and using the command chmod. The chmod command allows to provide read, write, and execute (r, w, x) options to users. The two traditional ways are by providing the mode or by providing the octal numeric value corresponding to each group according to its permissions.
Notice that we only need three bits to modify the read, write, and execute permission of a file for the three potential users. Use the command chmod to change the permission of a file
Symbolic Octal
rwx 7
rw- 6
r-x 5
r-- 4
-wx 3
-w- 2
--x 1
--- 0
For example, if you would like to add executable permission to everybody, i.e., all users, members of a group, and owner, for the file myscript, you can run the following command:
chmod +x myscript
Execute the First Script
A script is a sequence of combined commands together in a form of a ``program''. Remember that in Linux, everything is either a directory or a file. The script will execute these sequence of commands at once. Therefore, we need to make the program in executable mode.
The sequence of steps can be done by using system commands (like the ones we presented on Page 2 and 3. Also, later on, we will build our scripts by using the structures of programming (e.g., conditional statements, repetitive structures, functions) in order to make effective and meaningful scripts.
Let us illustrate the design and execution of a script with two examples
Example 1:
Create a script called example1 that will list all the files in the current directory.
We can create a file from the terminal as follows:
cat > example1
ls -l
[Ctl + c]
At this point you created a file, which contents only has the text ls –l. In order to run it and execute it as a script, we need to add the attributes of executable as follows:
chmod +x example1
Now is ready to execute. You can run it as follows:
./example1
Now, the file example1 is an executable file. The commands inside the script will simply list the files that are in the current directory
Example 2:
Write a script called script01 that perform the following 3 operations
print the directory contents
print the calendar of Feb 2016
create a file named output containing the output of Question 1 and Question 2
First, let us remember the commands that will allow us to perform our ultimate task:
to print the directory contents, we use: ls –l
to print the calendar for February, we use: cal feb 2016
to create a file, we know that we can use cat. However, for this particular task we need to append the results from Task (1) and Task (2) into the file called output.txt
Suppose we have the following files in the current working directory:
ls -l
-rw-rw-r-- 1 ec2-user ec2-user 0 Jan 24 00:53 file1.txt
-rw-rw-r-- 1 ec2-user ec2-user 0 Jan 24 00:53 file2.txt
-rw-r--r-- 1 ec2-user ec2-user 570 Jan 23 17:04 README.md
-rw-rw-r-- 1 ec2-user ec2-user 64 Jan 24 00:52 script01
Notice that script01 was just created and now we will add the commands
Let us edit script01
Let us assign the output of listing the files in the directory into the file output.txt
ls -l >> output.txt
Next, let us redirect the output of the calendar into the file output.txt
cal 2 2016 >> output.txt
Now your script looks as follows:
more script01
ls -l >> output.txt
cal 2 2016 >> output.txt
Next step is to change the executing properties for script01
chmod +x script01
ls -l
-rw-rw-r-- 1 ec2-user ec2-user 0 Jan 24 00:53 file1.txt
-rw-rw-r-- 1 ec2-user ec2-user 0 Jan 24 00:53 file2.txt
-rw-r--r-- 1 ec2-user ec2-user 570 Jan 23 17:04 README.md
-rwxrwxr-x 1 ec2-user ec2-user 45 Jan 24 00:57 script01
Execute script01 by using the following:
./script01
**Different flavors of Linux/shells requires that calendar provide a numeric value or the actual text to represent the month. For example some can work with cal feb 2016 some will require cal 2 2016
That will generate output.txt. By using more output.txt, we will see:
more output.txt
-rw-rw-r-- 1 ec2-user ec2-user 0 Jan 24 00:53 file1.txt
-rw-rw-r-- 1 ec2-user ec2-user 0 Jan 24 00:53 file2.txt
-rw-rw-r-- 1 ec2-user ec2-user 0 Jan 24 00:57 output.txt
-rw-r--r-- 1 ec2-user ec2-user 570 Jan 23 17:04 README.md
-rwxrwxr-x 1 ec2-user ec2-user 45 Jan 24 00:57 script01
February 2016
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29
Check Point
Please see Lab01.pdf that will be used to verify your skills obtained in this activity