Linux/Bash Tutorial

1 - Linux and Other Operating Systems

2 - Accessing Linux

Windows SSH-key setup

Mac/Linux SSH-key Setup

3 - Bash basics

Navigation - pwd / ls / cd

File Reading - cat / tail / head

File Manipulation - cp / mv / rm / pipes

history

File Editors - vim/nano/emacs

Getting help - man / --help

1 - Linux and Other Operating Systems

At a high level, an operating system (OS) is a piece of software that serves as the base of a system and provides useful services and utilities to other programs. Many of you are likely familiar with either Windows or MacOS. Both of these OSs provide various functionalities and programs that make your life easier: a graphical desktop, a file system, internet connection capabilities, etc... Traditionally, Linux-based OSs have been popular amongst programmers because they provide a variety of tools that make programming much easier.

Notice that I wrote "Linux-based." Linux is not quite an operating system, but is more like a foundation that operating systems can be built on. Operating systems built on Linux are called Linux distributions. There are many hundreds, if not thousands of Linux distributions out there (some examples being: Debian, Red Hat, Ubuntu, Arch Linux, Linux-Mint, ...) Each distribution is customized with different features and often meant for a different purpose. For instance, a Linux distribution for a battery-powered seismic sensor must be lightweight, and power-efficient, but a Linux distribution for your personal desktop needs to be able to interact with your printer and send emails. With such a wide array of use-cases, there exist many Linux distributions to meet these needs.

One last property that makes Linux unique is that it is open source and freely available. Developers around the world are constantly contributing to the latest updates for Linux as well as searching for potential security vulnerabilities in past versions. Many distributions as a result are also free-to-use, though some, such as Red Hat require licenses.

2 - Accessing Linux

So now that we've introduced Linux, you may be wondering "why?" For this course we're going to be looking into a lot of the internals of what goes on in an operating system by having you interact with it. To standardize everyone's operating system experience, we will be using the Ubuntu OS (a Linux distribution) in containers provided by OIT. These are remote computers that you will be connecting to and running code on in order to do your labs and projects. (Note: for advanced users, we also provide the necessary tools to set up a local docker container, but this provides no additional functionality - link)

Here we will show how to set up and connect to your OIT Ubuntu containers.

This link goes to the container management website. Under "Reservations Available" will be a container for CS210. Click the "reserve CS210" button. Next, the webpage should load back and there will be a link for CS210 under "My Reservations" - click it. It should look something like this:

Notice the "ssh term@..." command. 'ssh' is a tool used to interact with remote computers.

If you click the "Login" button, it will start a shell terminal inside your browser. You can do all of your work for this class this way, but it would be a lot of extra work for you. Instead, we're going to be setting up VSCode so you can program on your local machine and it runs your code inside the container. To do this we have to set up password-less login. This will also make it way easier for you when you may want to log into the container and don't want to go find the randomized password.

Instead of using a password, a password-less login uses encryption keys stored on your computer to communicate securely with the remote computer.

Windows SSH-key setup

First, we generate our keys on our local machine. Open up a fresh shell/powershell session and type:

ssh-keygen

This command will ask you several questions (file locations, passwords, etc...), but we just want the default settings so press the enter key 3 times until you're back to the shell prompt again.

Now it's time to copy these keys to the container so that it will recognize us without using a password. Enter the following lines below in your terminal. For <YOUR PORT>, check your container page on OIT and copy the port ID there into this command. You will be asked for your container password, so use the one provided by OIT on the container page. For <YOUR CONTAINER ADDRESS>, use the URL listed after 'term@' provided on the OIT container page.

cd .ssh

type id_rsa.pub | ssh term@<YOUR CONTAINER ADDRESS> -p <YOUR PORT> "mkdir -p ~/.ssh && cat >> .ssh/authorized_keys"

Now the container should recognize you without a password. Check that this works by ssh'ing into your container and see if it doesn't ask your for your password. You can do this by using the command listed on the OIT container site.

Note: In previous semesters, OIT sometimes resets the containers for one reason or another, which results in some nasty warnings that prevent you from working. We wrote up a solution guide here.

Mac/Linux SSH-key Setup

First, we generate the keys for our local machine by opening a terminal and entering this command:

ssh-keygen

This command will ask you several questions (file locations, passwords, etc...), but we just want the default settings so press the enter key 3 times until you're back to the shell prompt again.

ssh-copy-id -p <YOUR PORT> term@<YOUR CONTAINER ADDRESS>

Note: In previous semesters, OIT sometimes resets the containers for one reason or another, which results in some nasty warnings that prevent you from working. We wrote up a solution guide here.

3 - Bash basics

Now that you have copied your keys to the container, try the ssh command listed in the container manager. You should end up in a screen like this

Here we can enter a variety of commands. These commands have tons of helpful options that I've listed a few of. Don't worry about understanding and memorizing these commands just now. It's helpful to know what you can do. Whenever you have something you need to do, you'll remember that there exists a command to help you do that, you'll find it, and you begin to remember the commands you like to use and are useful for you.

Here's an introduction to some of the basics.

Navigation - pwd / ls / cd

'pwd' stands for print working directory. Whenever you open up a terminal, you're located at the home directory in your file system. You can navigate around your file system with the 'cd' command. 'pwd' will print the path to where you currently are.

_________________________________________________________________________________________

'ls' stands for list subdirectories. Whenever you're navigating to a different directory, it's often helpful to know what the different files and folders in your current directory are. 'ls' will print out all of these things so you can modify/use these files and folders.

Bonus options:

-l - (lowercase L) this option stands for "long" prints out the files and subdirectories with additional information like file permissions, file size, owner, creation date, etc...

-h - this option stands for "human readable". The file sizes in the previous option (-l) are in bytes and can get really big, forcing you to do a lot of uncomfortable math. This option converts all the numbers into kilobytes/megabytes/whatever is closest.

-a - this stands for "all". Files starting with a period (.) are usually ignored by most programs, like your personal file browser and 'ls'. For this reason, many system folders start with a period, to not clutter up your directories. One for instance is the .ssh directory that we used in the previous section.

___________________________________________________________________________________________

'cd <location>' stands for change directory. This command lets you navigate around your file system!

Bonus:

Sometimes you switch into directories and you need to find your way back out.

'cd ~' takes you back to your home directory. You can even combine it into paths like 'cd ~/Movies' means change directory to my movies folder inside my home directory.

'cd ..' takes you up one level in your directory structure. You can combine them with paths as well like 'cd ../Movies/../Desktop/../Documents/../..'

'cd -' Sometimes you're mainly working in two directories and switching between them a lot. If they have long path names it can get really tiring to type them in. 'cd -' changes directory to the last subdirectory that you were in. Using 'cd -' twice will take you back to the directory you're currently in.

File Reading - cat / tail / head

'cat <file name>' prints out the contents of a file to the screen. There are several similar commands:

'tail <file> -n <lines>' can print out just the end of the file if you give it how many lines to print out

'head <file> -n <lines>' prints out just the n first lines of the file

File Manipulation - cp / mv / rm / pipes

File movement.

'cp <file source> <file destination>' is used to copy files. The ' -r' option can be used whenever you want to copy directories.

_______________________________________________________________________________________

'mv <file source> <file destination>' is used to move files. This will remove it from the source and place it in the directory. This is often used to rename files as well. For instance, if I'm currently in a directory with a file called 'a.txt', I can rename it by doing 'mv a.txt something_else.txt'. If I just want to move the file to a directory and keep the name, 'mv a.txt my_directory/' is also valid. 'mv' will overwrite files if you name them the same thing.

_______________________________________________________________________________________

'rm <file> [<more files>]' stands for remove. You can provide one or more files to remove and it will delete them. A word of caution: there is no recycling bin like when you delete something in your file explorer, 'rm' deletes thing permanently and immediately so tread carefully.

Bonus:

'-r' By default, 'rm' will not delete folders. You can tell it that you want to delete folders with this option.

'-f ' If a folder or file that you list does not exist, it will fail and give an error. '-f ' will silence these warnings. This is useful when writing scripts because you don't want the entire script to halt execution because a file didn't exist to delete.

_______________________________________________________________________________________

Finally, piping output. Often times, we will have commands that work in tandem with each other. For instance, if we have a program that reads in from user input and we have a file with contents that we'd like to feed into the program, it would be really slow to type it all in manually. instead, we can do this:

cat my_file.txt | ./my_program

Above the ./my_program syntax is the syntax for running an executable program, which you will become more familiar with during the first few course lectures.

You may also run into a situation where a program of yours produces a LOT of output and you'd like to save it into a file so that you can read through it and/or save it for later. You can do that like so:

./my_program > my_output_file

history

'history' shows all of the bash commands you've previously written. You may want to remember a command that you previously used and you can see the last couple hundred of them here.

If you want to use one of these previously written commands, you have a couple options:

Write out the command manually
Use the up/down arrow keys. Pressing the up arrow key will bring up the previously written command. Pressing it multiple times will go further back in the history.
Advanced: Reverse-r-search. My personal favorite! If you remember even a part of the command you wrote previously (doesn't have to be the beginning), you can press 'ctrl+r' and type the fraction of the command and it will bring up the most recent command that contains that substring. Pressing 'ctrl+r' again will go further back in the history searching for your string.

File Editors - vim/nano/emacs

Editing files! Sometimes you will need to edit files inside the terminal. nano, vim, and emacs are all popular options and if you thought tabs vs. spaces was the last programmer culture war you'd encounter, you were wrong: there are many fervent supporters of vim and emacs in particular. They are all feature-rich, powerful editors and vim/emacs offer many plugins to extend these functionalities. It's not extra useful to know how to use multiple of these editors as they're usually available on all systems you will interact with. If you want to know how to use these editors, I suggest looking at tutorials online as they are all quite different to use.

Getting help - man / --help

Lastly, if you're not sure how to use a command-line tool or any of their options, the tool will usually provide an interface for reading its manual through the 'man' command or the '--help' option. Whenever you're unsure about how to use something, try these commands:

<command> --help
man <command>

For instance, lets say we were confused about the usage of the `cp` command, then I would type `man cp` into my shell and it would show me the format and usage of `cp` and all possible options.

Page updated

Report abuse