Sources: linux academy, cyberciti, linuxjournal
mkdir x #Makes a directory named 'x' in current folder.
mkdir -p /tmp/y/x #Makes a directory named '/tmp/y/x', '/tmp/y' and '/tmp' if they don't exist.
rm -r x #Remove Directory/file named 'x' & its contents.
rm -rf x #Remove Directory/file & its contents without confirmations.
ls #Lists all files/directories in current directory.
ls -ltr #List files sorted in reverse order of time of modification.
7za a outputFile.7z file1tocompress file2tocompress... # Compress/Append file using 7za
tar -zcf final.tar.gz final.tsv #Compress file using tar
xz -d file.txz #Extract .txz files.
tar -xf file.tar #Extract .tar files.
7za x file.7z #Extract .7z file.
7za x file.7z specificFileToExtract #extract specific file from 7z file.
unzip file.zip #Extract .zip file
gunzip file.gz #Extract .gz file
gunzip --keep file.gz #Extract .gz without loosing input file
gunzip -c --keep file.gz | awk -F'\t' '{print $1,$2}' #Pass gunzip result to default outputstream (Console).
7za d file.7z fileToRemove #Remove specific file from 7z file
7za l file.7z #Lists content of .7z file
find / -type f -mtime +30 -print | xargs -I {} mv {} /storage/archive/dbdumps #Find and move
less file.txt #opens file in read-forward only mode. hit q to quit
more file.txt #opens file in read only mode, and allows forward/backward scroll. hit q to quit
head -10 file.txt #reads top 10 records from file.
tail -10 file.txt #reads last 10 records from file.
tail -f file.txt #reads file and follows it for any appended row in file.
cat file.txt #displays entire content of file.
cat > file.txt #Cat can also be used to write to file. It creates new or write to existing.. hit Ctrl+d to exit writing. use double output-redirection for append.
To download content from any URL using either wget or curl:
wget "Source URL" -O downloadPathFile.txt
curl -O "Source URL"
What if download fails in mid? -c
wget -c "Source URL"
Sending payload using wget or curl. #User agent can be obtained by hitting various site. eg- http://www.whatsmyua.com/ Sources: stackoverflow
wget -O --post-data='{"x":1, "y":"hello"}' --header=Content-Type:application/json --user-agent='<USER AGENT>' "Destination URL"
curl --header "Content-Type: application/json" --request POST --data '{"x":1, "y":"hello"}' "Destination URL"
Connect to another server:
ssh <USER>@<IP> [-p<PORT_REQUIRED_IF_OPENSSH_ON_REMOTE_IS_USING_OTHER_THAN_22>] [-i<FILE.PEM IF_PEM_FILE_PROVIDED_INSTEAD_OF_PASSWORD>]
Execute Command on remote server: same as above and supply command at last:
ssh <USER>@<IP> "ls -l" #Lists files in user's home directory available on remote server.
grep -oh "word" toFindInFile.txt | wc -l # -oh will print each occurance in new line. wc -l counts lines finally giving occurrence of word in file.
grep -c "word" toFindInFile.txt #Count lines that contains a word.
grep -n "word" toFindInFile.txt #Print Line numbers, where word is found.
grep -A2 -B2 "word" toFindInFile.txt OR grep -2 "word" toFindInFile.txt #Print before/after lines where word is found.
grep --color=[always|never|auto] word toFindInFile.txt #Highlight matches.
grep -v "word" toFindInFile.txt #Print lines that Does NOT contain word.
grep -f fileWithLinesToSearch.txt toFindInFile.txt #Find lines from one file into another.
grep --lines-buffered "abc"
grep "esac" * -R #Find pattern/text in all file/folder/subfolder recursively.
Variable declaration: Its simple, think a name and value, place a equal sign in between, that's all. But there are definitely few rules. Spaces are not allowed any where. Name must not start with number or special character . What if we have space in our value? enclose them within quotes. following are valid declarations:
a=5
b="xyz"
c="x yz"4
d1='x yz'5
e=`date` #commands within back ticks are executed and result is stored in variable 'e'
f=$(date) #Alternate to back ticks.
Values in declared variables can be accessed by variable names prefixed with $ sign. e.g- echo $a Or, echo ${a} #this will print value stored in variable a.
Arrays declaration:
arr=(value1 value12 value123)
${arr[@]} # All of the items in the array
${!arr[@]} # All of the indexes in the array
${#arr[@]} # Number of items in the array
${#arr[0]} # Length of zero'th item
Loop Constructs:
For
for i in {0..10}; do echo $i; done # prints 0 to 10, available since bash 4.0
for i in {0..10..2}; do echo $i; done # prints 0 to 10 at step 2
for each in ${array[@]}; do echo $each; done ##prints each entry in array
*Input Field Separator(IFS) can be used to token/split any char sequence. Its recommended to take backup of IFS before altering it and restore to default(\n) after usage.
ifs_bkp=$IFS #Backup of IFS
a=ABC#DEF#GHI #defines '#' separated Text
IFS=# #Specifies '#' as IFS
for each in $a; do echo $each ; done ##prints each tokens
IFS=$ifs_bkp #Restore IFS to default.
While
while [ TRUE ]; do echo "hi"; sleep 2; done #infinite loop at interval of 2 seconds.
while read line; do echo $line; done < toFindInFile.txt #for each line from file.
i=0; while [ -le 5 ]; do echo ; i=`expr + 1 `; done
Conditional Constructs:
if
* Double [[]] allows to write expressions[e.g- [[ "This is a cat." == *a* ]] ]
*[[ "209809" =~ ^[0-9]+$ ]] && echo "True" || echo "False"
*String condition check ' =~ ' can be used within [[]] only
if [ "abc" == "" ]; then { echo "true"; } fi
if [ "abc" == "" ]; then { echo "true"; } else { echo "false"; } fi
if [ $a -eq 1 ]; then { echo "First true"; } else if [ $a -eq 2 ];
then { echo "Second True"; } else { echo "None True"; }
fi
fi
[ "" == "" ] && {echo "True"} || {echo "False"} #Is a shortend way of writing conditions. && represents 'THEN' block and || 'ELSE' block
Switch case. note - closing circular braces after a value, and ;; as break statement while * for default case.
case $animal in
tiger ) echo "$animal is a wild animal." ;;
cow ) echo "$animal is a domestic animal." ;;
*) echo "$animal is unknown to me" ;;
esac
Logical Operators:
*AND operator is '-a' and OR is '-o' if multiple condition. e.g.
[ "209809" == "^[0-9]+$" -a 2 -gt 1 ] && echo "True" || echo "False"
[ "209809" == "^[0-9]+$" -o 2 -gt 1 ] && echo "True" || echo "False"
Comparison Operators:
== #Used with text, for equality comparision
!= #Used with text, for non-equality comparision
-gt #Used with numbers, for Greater than check
-lt #Used with numbers, for Lesser than check
-le #Used with numbers, for Lesser than Or Equal check
-ge #Used with numbers, for Greater than Or Equal check
-eq #Used with numbers, for Equal check
-ne #Used with numbers, for Not Equal check
Other conditional switches:
*All switches can be prefixed with "! " for negative check.
*File conditions are : a,b,c,d,e,f,g,G,h,H,k,L,N,O,p,r,s,S,t,u,w,x
*String conditions : >,<,-n,-z,=~
*Miscellaneous conditions: -o
* && can also be used to put to commands together.eg- mkdir test && cd test
[ -f /tmp/a ] #returns true if file '/tmp/a' exists else false.
[ -d /tmp/a ] #returns true if '/tmp/a' is a directory else false.
[ file1 -nt file2 ] # newer than
[ file1 -ot file2 ] # older than
[ -z ] # checks is var is empty, returns true if empty else false.
Runtime value Input: Reading user inputs using read. -p - to prompt a message.
read -p "Enter your Name: " name ; #accepts input from user and store in 'name' variable.
Ouput :
echo "Hello World" # prints hello world on screen.
echo -e "rohit\tverma\n33" # prints rohit[TAB]verma and 25 on next line. -e=express/expand
echo -n "hello world" #prints hello world and remains on same line.
echo ${varName,,} #double comas prits value in varName is printed
echo ${varName^^} #double carat prints value in varName is printed in upper case
Arguments:
$0 #Command name itself
$1 #First argument after command, $N for rest arguments
$# #number of arguments supplied
$! #Process ID of last process
$@ #All arguments supplied
Function Declaration:
x(){
echo $#;
echo $0;
}
Function call:
x Rohit Verma
Print a sequence of numbers.
seq 0 1 10 #prints 0 to 10 at step 1
echo -e " 1rohit\n2ram\n 3rahul\n4robin\namit\nmanoj" | sort # sorts given sequence
seq 0 5 | sort -r # sorts sequence 0 to 5 in reverse order.
echo -e "abc\ndef\nab\nabc\nghs" | sort | uniq #unique sorted list.
echo Hello World | rev #reverse text.
cut usage.
*Default delimeter is \t TAB. Single Character delimeter can be specified using '-d' option
echo -e "Name=Rohit\tAge=31\nName=Rahul\tAge=33" | cut -f1 | cut -d'=' -f2
echo "This is a cat, and it mews." | cut -d' ' -f4,7 --output-delimiter=' sounds '
tr usage: Changing case of Text using tr command.
echo Hello | tr [:lower:] [:upper:] #Converts all lower case chars to upper case
echo Hello | tr [:upper:] [:lower:] #Converts all upper case chars to lower case
echo Hello | tr -s l #squeezes multiple occurrences of char to 1
echo "hello world" | tr -s " " #Squeeze spaces
Using Tee:
echo "Hello World" | tee /dev/tty | sed s/e/E/g
compgen -c # find find available commands and keywords.
cat /etc/shells #List all available shells.
Maths bc expr ;)
echo "5*3 + 3 - 19/5" | bc #return 17 dropping decimals.
echo "5*3 + 3 - 19/5" | bc -l #return decimal values up to 20.
echo "scale=3; 5*3 + 3 - 19/5;" | bc -l #return decimal values up to 3.
expr " 5 \* 3 + 3 - 19 \/ 5 "
Simple example (test.sh)
date > /tmp/dates
while [ .TRUE. ]
do
date >> /tmp/dates
sleep 2
done
# How to export variable in parent shell from within any script.. e.g.- I want to set JAVA_HOME and put same on PATH first for running jdk8 where jdk 7 is on default path.
1. Create script file in /usr/bin named J8
export JAVA_HOME=/opt/jdk1.8.0_161
export PATH=$JAVA_HOME/bin:$PATH
2. Run script as follows: chmod 777 J8 if required...
. J8
#Grep and kill all process with name flume
kill -9 ` ps -ef |grep flume | tr -s " " | cut -d ' ' -f2 | tr '\n' ' '`