[TBD]Useful tips for bash script

Introduction

For Linux programmers, it should be obvious to know that bash script is used at many places. For example, it is used during boot-up and performs initialization (like daemon creation). However, many times we wish to know ways for our own requirement. For example, to debug, we may wish to know about simple ways to print commands executed by bash script. This document tries to give list of such tips.

Help guide for operations

Processing multi line output of a bash command

Process substitution for reading output and processing on this.

Example to get diff of two files

diff <(sort list1) <(sort list2)

This command will sort list1 and list2 and then it will calculate diff of both files

Useful link: http://mywiki.wooledge.org/ProcessSubstitution

Easy way To display commands executed as part of bash script

set -x will give you what you want.

Here is an example shell script to demonstrate:

#!/bin/bashset -x #echo on ls $PWD

This expands all variables and prints the full commands before output of the command.

output:

+ ls /home/user/

file1.txt file2.txt

Bash FAQs

Difference between single square bracket and double square bracket

[ is a bash Builtin

[[ ]] are bash Keywords

Keywords: Keywords are quite like builtins, but the main difference is that special parsing rules apply to them. For example, [ is a bash builtin, while [[ is a bash keyword. They are both used for testing stuff, but since [[ is a keyword rather than a builtin, it benefits from a few special parsing rules which make it a lot easier:

$ [ a < b ]

-bash: b: No such file or directory

$ [[ a < b ]]

The first example returns an error because bash tries to redirect the file b to the command [ a ]. The second example actually does what you expect it to. The character < no longer has its special meaning of File Redirection operator.

Useful link: https://serverfault.com/questions/52034/what-is-the-difference-between-double-and-single-square-brackets-in-bash

Handling Ctrl+C while executing bash script

If user presses this key combination, then SIGINT signal is sent to script. By default, it will terminate the script abruptly. However, we can decide if we need to continue or exit.

Following example gives control to user to decide if he really wants to terminate

Shell program to handle SIGINT

root@9049c7418b2a:/# cat test.sh

#!/bin/shfunction stop(){while true; do read -rep $'\nDo you wish to stop playing?(y/n)' yn case $yn in [Yy]* ) echo "Thanks for playing !!!"; exit 1;; [Nn]* ) break;; * ) echo "Please answer (y/n)";; esacdone} trap 'stop' SIGINT echo "going to sleep"for i in {1..100}do echo "$i" sleep 3 done echo "end of sleep"

Below test output helps to understand behavior of above program

Test output

root@test:/# bash test.sh

going to sleep

1

^C

Do you wish to stop playing?(y/n)n

2

3

4

^C

Do you wish to stop playing?(y/n)n

5

6

^C

Do you wish to stop playing?(y/n)y

Thanks for playing !!!

root@test:/#

Method to compare substrings (prefix of string)

In below example, it matches with any string starting with 123

var=123

[[ 1234 = $var* ]] && ...

Ref: https://stackoverflow.com/questions/4132510/how-to-test-that-a-variable-starts-with-a-string-in-bash

How to pass variables to bash shell program

Variable=value can be passed to bash program as prefix of the command.

Example to pass BUILD_OS info to the bash program

[root@ubuntu /personal/testcode/makefile_learning]# cat test.sh

echo $GREETINGS

echo build os is $BUILD_OS

======================================

[root@ubuntu /personal/testcode/makefile_learning]# BUILD_OS=apple GREETINGS=hello ./test.sh

hello

build os is apple

---------------------

Note that BUILD_OS and GREETINGS are user defined variables. In other word, we can use any string

How to compare two strings

== and != operator is useful. In below example, x is a variable and it compares to string valid.

if [ "$x" == "valid" ]; then echo "x has the value 'valid'"fi

if [ "$x" == "valid" ] || [ "$x" == "tested" ]; then echo "x has the value 'valid'"fi

if [ "$x" == "valid" ] && [ "$x" == "genuine" ]; then echo "x has the value 'valid'"fi

Useful link:- http://stackoverflow.com/questions/2237080/how-to-compare-strings-in-bash

http://stackoverflow.com/questions/3826425/how-to-represent-multiple-conditions-in-a-shell-if-statement

How to concatenate two strings

Just write two strings next to each other

a='hello' b='world' c=$a" "$b echo $c > hello world

Useful link:- http://stackoverflow.com/questions/4181703/how-to-concatenate-string-variables-in-bash

How to compare two directories ignoring extra files in right side directory

Compare two directories

for f in `find $1 -not -path '*/\.*' -type f | sed 's:^cpxgw_src/::'`

do

# Suppress details of differences

#diff -rq $f $2/${f##*/}

diff -rq $1/$f $2/$f

done

===

>> Above code prints missing file in right side folder and also different file

nonroot@ubuntu:~/$ bash compare.sh src1 src2/

Files src1/test/Makefile and src2/test/Makefile differ

diff: src2//test.c: No such file or directory

Useful link: https://unix.stackexchange.com/questions/49496/recursively-compare-two-directories-with-diff-r-without-output-on-broken-links

How to check an environment variable is set or not

-z option to check if environment variable is set

if [ -z ${MYVAR+x} ]; then

echo notset

else

echo set to ${MYVAR}

fi

useful link:http://stackoverflow.com/questions/3601515/how-to-check-if-a-variable-is-set-in-bash

How to unset an environment variable

use unset command

How to sort words in a sentence using bash command

sort and xargs command will help in this regard

[root@ubuntu /personal]# echo "zebra ant spider spider ant zebra ant" | xargs -n1 | sort -u | xargs ant spider zebra

useful link:- http://stackoverflow.com/questions/8802734/sorting-and-removing-duplicate-words-in-a-line

How to check a string contains sub-string

Below example shows that name substring is part of value string

if [[ "$string" == *"$substring"* ]];then

echo $substring is part of $string

fi

Example to check if value string contains name substring

value=`echo "zebra ant spider spider ant zebra ant" | xargs -n1 | sort -u | xargs`

value2=`echo "zebra ant zebra ant anty" | xargs -n1 | sort -u | xargs`

echo $value

echo $value2

for name in $value2

do

if [[ "$value" == *"$name"* ]];then

echo $name

fi

done

Useful link:- http://stackoverflow.com/questions/4277665/how-do-i-compare-two-string-variables-in-an-if-statement-in-bash

Method to set default value of a variable in a single line

Refer https://stackoverflow.com/questions/2013547/assigning-default-values-to-shell-variables-with-a-single-command-in-bash

Example for setting default value in single line

root@e9076f0e6e52:/ws1/rs_121_48_triton/usr.src/netscaler# export LOGLEVEL='abcd'

root@e9076f0e6e52:/ws1/rs_121_48_triton/usr.src/netscaler# bash test.sh

abcd

root@e9076f0e6e52:/ws1/rs_121_48_triton/usr.src/netscaler# cat test.sh

function test()

{

LOGLEVEL=${LOGLEVEL:-"INFO"}

}

test

echo $LOGLEVEL

root@e9076f0e6e52:/ws1/rs_121_48_triton/usr.src/netscaler# bash test.sh

abcd

root@e9076f0e6e52:/ws1/rs_121_48_triton/usr.src/netscaler# unset LOGLEVEL

How to notify via mail once IP address of a server comes up

root@e9076f0e6e52:/ws1/rs_121_48_triton/usr.src/netscaler# bash test.sh

INFO

Refer example script below

Useful link: https://stackoverflow.com/questions/2013547/assigning-default-values-to-shell-variables-with-a-single-command-in-bash

Monitor IP 10.105.96.200

[root@ubuntu /personal/testcode]# cat pingandMail.sh

#!/bin/bash

HOSTS="10.105.96.200"

COUNT=4

pingtest(){

for myHost in "$@"

do

ping -c "$COUNT" "$myHost" && return 0

done

return 0

}

if pingtest $HOSTS

then

# 100% worked

echo "Server worked at $(date)" | mail -s "Server up" deepak@citdel.com

echo "All hosts ($HOSTS) are down (ping worked) at $(date)"

fi

Useful link:https://stackoverflow.com/questions/8918159/script-for-email-alert-and-ping

How to convert CIDR format to netmask in bash

Below script code will help

cdr2mask () { # Number of args to shift, 255..255, first non-255 byte, zeroes set -- $(( 5 - ($1 / 8) )) 255 255 255 255 $(( (255 << (8 - ($1 % 8))) & 255 )) 0 0 0 [ $1 -gt 1 ] && shift $1 || shift echo ${1-0}.${2-0}.${3-0}.${4-0} }

$cdr2mask 16 --> 255.255.0.0

Useful link:https://serverfault.com/questions/54981/linux-command-line-tool-to-work-with-netmasks-cidr-notation

Method to take diff of two command output

Example is 'diff <(ls old) <(ls new)'

Diff of two command output

# cat test.sh

all_macs=$(ifconfig -a | grep HW | awk '{print $5}')

echo All MACs are $all_macs

all_macs_count=`echo $all_macs | xargs -n1 | wc -l`

echo $all_macs_count

all_unique_macs=`echo $all_macs | xargs -n1 | sort -u`

echo All unique MACs are $all_unique_macs

all_unique_macs_count=`echo $all_unique_macs | xargs -n1 | wc -l`

echo All unique MACs are $all_unique_macs_count

if [ $all_macs_count -gt $all_unique_macs_count ]; then

echo found duplicate mac address

diff -du <(echo "${all_macs%x}") <(echo "${all_unique_macs%x}")

dup_macs='found'

fi

echo $dup_macs

# bash test.sh

All MACs are 02:42:3c:19:6a:23 ba:7b:50:b3:2d:30 f2:73:fb:0a:94:37 8a:ef:91:97:39:c3 52:54:00:3f:9b:62 52:54:00:3f:9b:62

6

Useful link: https://askubuntu.com/questions/229447/how-do-i-diff-the-output-of-two-commandsHow to check if a bash variable is empty

All unique MACs are 02:42:3c:19:6a:23 52:54:00:3f:9b:62 8a:ef:91:97:39:c3 ba:7b:50:b3:2d:30 f2:73:fb:0a:94:37

All unique MACs are 5

found duplicate mac address

--- /dev/fd/63 2018-09-10 01:57:07.760817349 +0530

Below code will return true if a $VAR is unset or set to the empty string ("").

+++ /dev/fd/62 2018-09-10 01:57:07.760817349 +0530

@@ -1,6 +1,5 @@

if [ -z "$VAR" ];

02:42:3c:19:6a:23

Useful link: https://serverfault.com/questions/7503/how-to-determine-if-a-bash-variable-is-empty

+52:54:00:3f:9b:62

+8a:ef:91:97:39:c3

How to suppress or hide command output in bash script when command is running in background

ba:7b:50:b3:2d:30

f2:73:fb:0a:94:37

Use {} as shown in below example

-8a:ef:91:97:39:c3

-52:54:00:3f:9b:62

-52:54:00:3f:9b:62

{} for suppressing output of monit

{

monit -I &

} &> /dev/null

No output will be shown for above command

found

Useful link:https://stackoverflow.com/questions/18062778/how-to-hide-command-output-in-bash

Method to set variable only if not set already

use ? (question mark) with variable

Method to change column size

echo $COLUMNS and changing it worked for me

Ref:https://askubuntu.com/questions/219547/how-do-i-get-more-than-80-columns-in-command-line-mode

Reference

http://stackoverflow.com/questions/2853803/in-a-shell-script-echo-shell-commands-as-they-are-executed http://unix.stackexchange.com/questions/240602/how-can-i-handle-sigint-trap-with-a-user-prompt-in-shell-script