Linux review
Post date: Sep 24, 2016 10:56:10 PM
Run background service
# Ignore the hangup signal and keep running. Output will be put in nohup.out
nohup python bgservice.py &
File viewing, create
more # primitive view with only forward
less # advance view with back and forth navigation
touch # create a new empty file
# change owner/mode
sudo chown sbuser foo.py
chmod +x foo.py
File times
3 types of times associated with each file: created, modified, last-accessed
Text editing
tr ' ' '\n' | sort | uniq -c | awk '{print $2"@"$1}' # see [7]
tr '{}' '()' < inputfile > outputfile # tr for translation
awk '/search_pattern/ { action_to_take_on_matches; another_action; }' file_to_parse
Execute program/load program into memory as process
#e: array of environment var
#p: use the PATH environment var
#l: list of arguments
#v: vector(array) of arguments
execve
Unix system calls
openat # open at, similar to open but can choose a different dir rather than the working dir
getdents # get directory entries
Trace a program
strace
Running process information
top
hi # hard interrupt
si # soft interrupt
PR # priority
NI # nice = PR-20
VIRT # virtual memdstatdstat
RES # physical resource/mem
SHR # shared mem
nFLT # number of page faults
nDRT # number of dirty pages (mem pages changed in mem but not yet in disk)
About zombie and orphan process, see [5]
kill
By default, send SIGTERM to a process
with -9, send SIGKILL
Check open ports
netstat -latun
-l # listening
-a # all
-t # tcp
-u # udp
-n # numeric instead of dns resolving
Check routing table
route
Network manager
nmcli dev list iface eth0
Firewall
sudo ufw enable
sudo ufw allow 22 # SSH
sudo ufw disable
Common ports:
HTTP # TCP 80
HTTPS # TCP 443
FTP # TCP 21
DNS # UDP 53 (can be TCP)
SSH # 22
SMTP # 25
DNS
Hierarchical servers, starting with the last domain, e.g. google.com.us: .us domain first, then .com
Trace networking route
traceroute hostname
TCP/IP congestion control
slow start (start from 1, exponentially increase to a threshold)
AIMD (kicked in above threshold)
additive increase
If 3-dup ACK (same sequence #), then half rate (maybe congestion???)
In TCP, only ACK with the maximum contiguous number, e.g. received packet 1,3,4, ACK 1. When received 2, ACK 4.
If timeout, then reset to 1 (confirmed congestion)
TCP/IP connection
# open
SYNC ->
<- SYNC ACK
ACK ->
# close
FIN ->
terminate connection<- ACK
wait 20 s
<- FIN
wait 1 min
terminate connection
IP address
192.168.1.0/24: There are 2^8 - 2 addresses, because 192.168.1.0 is the network identifier, and 192.268.1.255 is the broadcast address
Set static/dynamic IP
sudo vim /etc/network/interfaces
dynamic:
auto eth0
iface eth0 inet dhcp
static:
auto eth0
iface eth0 inet static
address 10.253.0.50
netmask 255.255.255.0
network 10.253.0.0
gateway 10.253.0.1
dns-nameservers 8.8.8.8
Find text in files
grep -r -e "scipy" --include \*.py
Replace string in multiple files
sed -i 's/foo/bar/g' * # see [20]
Find certain file
find / -name "*.py"
find . -name '*.py' | xargs wc -l # From [8], count the number of lines of code
IO status
iostat /dev/sda
%user CPU utilization at user level
%system CPU utilization at system level
%iowait CPU idle while waiting for IO
%idle CPU idle without waiting for IO
List open files of a process
lsof -p [pid] # see [6]
IO redirection: See [1]
0 # stdin
1 # stdout
2 # stderr
M>N
# "M" is a file descriptor, which defaults to 1, if not explicitly set.
# "N" is a filename.
# File descriptor "M" is redirect to file "N."
M>&N
# "M" is a file descriptor, which defaults to 1, if not set.
# "N" is another file descriptor.
Kernel message buffer
dmesh
Mount devices
sudo mount /dev/sdb1 /media/usb
sudo umount /media/usb
sudo mount -t vfat /dev/sdb1 /media/external # see [3]
sudo mount -t ntfs-3g /dev/sdb1 /media/external
List disks and its partitions
sudo lsblk # see [2]
df -h # human readable. displays all of the mounted partitions.
sudo fdisk -l # displays all of the partitions that exist on your disk.
Apparently there are also some partitions which exist, but are not mounted. See [4]
Check disk/file system
fsck: file system check and repair
du -h: check disk usage
List PCI, USB
lspci
lsusb
Different lib folders
# see [9]
/lib # contain "essential" libraries that may be needed even in single-user (admin-only) mode and without /usr mounted
/usr/lib # are the libraries for the normal user-programs, that mostly can be found under /usr.
/usr/local/lib # are the libraries for locally installed programs and packages ie. things you've compiled and installed from source-packages yourself.
View gcc system header include (and its quirks)
cpp -v # See [10]
View symbols in .so files
# See [11]
nm -gC lib.so
# For ELF formats
objdump -TC lib.so
readelf -Ws lib.so
CMake by examples
# See [12,13]. CMake find_package(OpenCV) is defined in either
/usr/share/OpenCV/OpenCVConfig.cmake
/usr/local/share/OpenCV/OpenCVConfig.cmake
# Standard packages are listed by
cmake --help-module-list
# CMake Variables [14]
# CMake and RPATH [15]
Profile process
time prog # see [16]
Compressed multiple folders
tar -cf out.tar /dir1 /dir2 # see [17]
tar -czf out.tar.gz /dir1 /dir2
tar -tf out.tar.gz # view contents without extraction, see [18]
zip -r out.zip /dir1 /dir2
Linux-Windows dual boot
see [19]
Diff files
diff --strip-trailing-cr -ru <srcDir/> <dstDir/> # ignore line endings
SSH
ssh <username>@<ip/hostname>
scp -r <username>@<ip/hostname>:<remote/src/dir> <local/dst/dir>
[1] http://www.tldp.org/LDP/abs/html/io-redirection.html
[2] http://askubuntu.com/questions/527790/list-all-partition-labels
[3] https://help.ubuntu.com/community/Mount/USB
[4] http://stackoverflow.com/questions/16307484/difference-between-df-h-and-fdisk-command
[5] https://en.wikipedia.org/wiki/Zombie_process
[7] http://stackoverflow.com/questions/10552803/how-to-create-a-frequency-list-of-every-word-in-a-file
[9] https://askubuntu.com/questions/281660/what-is-the-difference-between-different-lib-folder-in-ubuntu
[10] http://commandlinefanatic.com/cgi-bin/showarticle.cgi?article=art026
[11] https://stackoverflow.com/questions/34732/how-do-i-list-the-symbols-in-a-so-file
[12] https://mirkokiefer.com/cmake-by-example-f95eb47d45b1
[14] https://cmake.org/Wiki/CMake_Useful_Variables/Logging_Useful_Variables
[15] https://cmake.org/Wiki/CMake_RPATH_handling
[16] https://stackoverflow.com/questions/556405/what-do-real-user-and-sys-mean-in-the-output-of-time1
[17] https://superuser.com/questions/748565/create-tar-with-multiple-directories-and-file-locations
[19] http://www.everydaylinuxuser.com/2016/06/how-to-remove-ubuntu-from-dual-boot.html