One of the things I like about Linux is shell scripting which for me is scripting with Bash (Bourne again shell).
Linux systems have much in common with Unix systems including a design philsophy of having discrete tools that do one job and do it well. Bash provides me with a powerful set of tools and also allows me to use a host of other command line applications when there is something I either want to do my own way or I can not find an application that suits the task.
Below you will find some shell scripts written by me. I learned how to script by reading and playing with scripts and perhaps these might be useful to someone else either for dissection or modification and use.
Some of the scripts listed below were used in a Bash Scripting tutorial I gave at the Melbourne Linux Users Group although the versions linked below may be more recent. You can find the complete tutorial
here. If you want to learn a little more about the Unix Philosopy you can find an introduction
here, and if you are familar with BASH and scripting, you might be interested in reading Steve Bourne's original shell tutorial,
An Introduction to the Unix Shell written at Bell Labs in1978.
countbytes: a script for logging incomming TCP traffic for a given period of time.
mtnfs: a script for mounting NFS fileshares at boot but only if the NFS server is up.
autoshtdn: shuts down a network server if their are no workstations up on the network.
chkserver: uses
kdialog to create a popup window to notify a user after login if the server is down.
save-dvb: an interactive command line script for scheduling and recording DVB-T free to air broadcasts.
get-weather: a very long one liner that max's out on stream editing to deliver weather to my console, a kdialog pop up, or a virtual webpage on a server.
create-dvd: a script for encoding, shrinking, cropping and burning dvd's from mpeg.ps videos.
This page is being revised. The next next four scripts have not been worked into this page yet.
- audcap: (audio capture) a quick and dirty script that uses arecord and kdialog to capture audio directly off your sound card. I link it to an icon and use it to timeshift audio streams off the net.
- mouseup: a script that assumes if the mouse hasn't been moved in a while and nobody is watching videos then its a good idea to shut down the box.
I don't use these scripts anymore but I've included them here in case someone else can make use of them.
- gx-logon: a network dialup connection script with GUI wrapper provided by gx-message. With gx-login it is possible for a user to remotely initiate a dialup connection on a gateway box
- qburn: for quick and simple copying/burning of
ISO's and data disks.
Countbytes:
Iptraf is handy to get a realtime ncurses display of network traffic listed by interface but it also has a logging function. This script takes a log created by iptraf run in a given time period and then parses out the inward TCP traffic, totals it and appends it to a logfile. I had always wondered how I could log downloads on individual workstations but never really found anything simple. When someone on the MLUG list asked if anyone knew of something he could use to log downloads on his networked workstations I decided to have a go at scripting something up using iptraf.
This script uses sleep and will take a time period for logging as minutes or hours as part of the command. If there is no incomming TCP traffic it will say so in the log entry. Otherwise the download total will be given as true kilobytes, i.e. total bytes by 1,000. Bash only supports math with integers so a decimal of kilobytes is achieved by using the modulus of total bytes by 1,000.
You can find
countbytes here and below is a snippit from the logfile
iptraf_summary created by
countbytes.
Begin: Monday-February-22:14; End: Monday-February-22:15; Total download on eth0 is: 1155.98 Kb.
Begin: Tuesday-February-05:19; End: Tuesday-February-05:19; There was no incoming TCP traffic during this period.
Begin: Tuesday-February-05:20; End: Tuesday-February-05:40; Total download on eth0 is: 8064.720 Kb.
Mtnfs:
We have a home network with static IP's that includes a server for printer/scanner shares, file shares and timeshifting DVB-T broadcasts. If this server is not turned on it takes forever for the workstations to boot if the NFS shares are listed in their /etc/fstab but can't be found.
The solution is to wait for the system to boot and then ping the server to see if it is up. If it is up the NFS shares can be mounted using the
mount command.
This is a basic script that fulfills a need. While there is nothing fancy about it, mtnfs shows how handy a little scripting can be . Since we use Slackware here, we can run mtnfs at the very end of a boot by listing the command in /etc/rc.d/rc.local. Mounting NFS shares like this might cause a shutdown to hang. Read the comments in the script to see how that can be avoided.
Autoshtdn:
The server for our home network is only required if there are work stations up on the lan that need to access it. In order to save some electricity and avoid heating up the room in which it resides unnecessarily, we use the script autoshtdn to shut it down when it is not needed.
Run as a cron job,
autoshtdn checks to see if:
- there are any workstations up,
- /var/spool/atjobs is empty, since our save-dvb script schedules timeshifts as at jobs,
- dvbstream is running which indicates that there is a show being saved to disk, and
- create-dvd, our encoding script is running.
Autoshtdn uses " 2 " to indicate up or running and " 1 " to indicate down or not running. As you can see from the log snippit below the first and third entries have one workstation up so the server is not shut down. However, in the second entry, the number one was returned for all 5 tests so the server was shutdown.
Mon Feb 23 21:47:11 EST 2009 ran autoshutdn script Mon Feb 23 21:47:11 EST 2009
rick=2, leila=1, atjobs=1, create-dvd=1, dvbstream=1, no shutdown now.
Mon Feb 23 22:47:14 EST 2009 ran autoshutdn script Mon Feb 23 22:47:14 EST 2009
rick=1, leila=1, atjobs=1, create-dvd=1, dvbstream=1, shutting down now.
Tue Feb 24 05:47:05 EST 2009 ran autoshutdn script Tue Feb 24 05:47:05 EST 2009
rick=2, leila=1, atjobs=1, create-dvd=1, dvbstream=1, no shutdown now.
You can find
autoshtdn here.
Chkserver:
A user in another room has no idea whether or not the server is up and more often then not if they are the first one out of bed or home in the evening they just turn on their machine and later will wonder why they cannot access shared directories or print.
Chkserver is run from ~/.kde/Autostart and checks to see if the server is up with ping.
If there are no return packets chkserver assumes that the server is down and uses kdialog to display an error message on the user's desktop.
Click the thumbnail above to see a screenshot of the chkserver popup window and click here to see the script
Save-dvb:
We use this script to schedule and record, i.e. timeshift, free to air DVB-T broadcasts. It requires dvbstream and a channels.conf for our locality (Melbourne, Australia). Everything else is bash and commands standard on just about any Linux distro.
I originally wrote this script so that we could timeshift using a DVB-T card on our server that usually sits in the corner without too much to do. We could ssh into the server and run save-dvb from our workstations. Shows are recorded to disk and we can access them via NFS fileshares in real time as they are recording or view them some other time. Later on when prices came down I purchased a second DVB-T card for my workstation where this script is also used for timeshifting. Scheduling is faster CLI than opening up and point/clicking my way through a GUI interface.
Save-dvb will parse a standard channels.conf file and display only those stations which have not been commented out. Uncommented stations are offered for selection after which the user is prompted for running time, a name for the recording, etc. The four thumbnails below will give you screenshots in sequence left to right of save-dvb in use.
This script parses the necessary variables from a channels.conf then combined with other variables based on user input, it writes a temporary script called .sched-show which is run either "now" or as an at job. The temporary script is overwritten the next time save-dvb is run.
The
.sched.show script for the above screenshots is
here. You can find
save-dvb here and my channels.conf (Melbourne, Mt. Dandenong)
here. If you are interested in how to use command line tools for DVB you may find
this page I put together a while back of interest although it is getting a bit long in the tooth.
Get-weather:
Get-weather is a script we all use around the house rather than hope and wait for a weather forecast on the radio. This script downloads our current weather forecast from the Australian B.O.M and the removes unnecessary text, does some reformatting and adds a few lines to make the resulting output to a terminal easier to read. The information comes from a text file that feeds the publicly accessable
http://www.bom.gov.au/products/IDV17102.shtml. I want something a little more concise and either available in a terminal or one click away with an icon.
I had previously made up a script called
get-forecast that I could run in a terminal or be used to provide content for a kdialog popup several years ago. However, several months ago B.O.M screwed everything up with some website redesign. Luckily for me the text based web page I was using,
www.bom.gov.au/cgi-bin/wrap_fwo.pl?IDV17102.txt, still existed even if it was no longer the default page for the Central District forecast. The layout and content had changed and I had to figure out a new way to parse the content as well do a little more to make the output presentable.
Get-weather turned out to be a bit simpler than my previous get-forcast being a one liner made up of piped sed commands that either prints to stdout or is redirected to a temporary text file which is then displayed in a kdialog popup. Get-forecast had been used in a bash script to generate a forecast page on my web site. The get-weather one liner has now been worked into my Mornington-weather.cgi to feed the latest forecast into the web page.
The thumbnails below are links to screenshots of local weather displayed in a terminal, in a popup and on a web page left to right. There are links to the scripts directly below each thumbnail. If you want to see what the forecast is right now go here:
http://www.turtlespond.net/Mornington-weather.cgi
If you have had a look at this or several of the other scripts above you will have noticed that I often use stream editors and pipes to work text.files and the output of other commands into something useful. There is usually more than one way to edit things on the fly and I don't claim to be the most elegant but if you are not familiar with stream editing you might find the explanantion of
how get-forecast works helpful if limited introduction.
Create-dvd:
Create-dvd was written solely for the purpose of shrinking, cropping, encoding as an mpeg-ps (.vob) the free to air digital TV programs that we have recorded and then burning them to a dvd. Any editing out of commercials or cropping the video before or after the desired program is done with
Gopdit, my prefered editor prior to processing with
create-dvd.
This script is used both on my workstation or on our server. The server is my preference as I can pop a blank dvd in the drive, ssh in, start the script, turn my machine off and then walk away and go to bed, go to work, whatever. When the script is done and the dvd is burned, the server will just shut itself doen unless there is a recording scheduled.
Create-dvd offer four different options:
- Burn a dvd without encoding, if the video file is not properly encoded the nothing will be burned.
- Encode to dvd quality with with a 16/9 aspect ratio then burn to a dvd.
- Crop, then encode to dvd quality with a 4/3 asopect ration then burn to a dvd, this is used on older movies.
- Encode as above with a 16/9 aspect ratio, shrink to a requant factor provided by user input then burn to a dvd.
After several years I have found these are the only options I require. There has never been an old movie with a 4/3 aspect ratio that required shrinking, and I've never had the need or desire to crop black borders out of video with a 16/9 aspect ratio.
This script requires empty space. I have written it so that when finished it will have left a directory containing the original video, the encoded video and the iso. temporary working directory will require additional room while shrinking/encoding. I usually leave the line that deletes this directory at the close of encoding commented but thats not neccesary.
Theres not much in the way of screenshots for this script other than the selection menu and the user being asked for a requant value:
Fun with stream editors:
You might have noticed that I often use stream editors and other CLI aps that sort, sift and edit text in my scripts. While intimidating at first and I lay no claim to mastering regular expressions, I find using these applications fun since using them involves problem solving, i.e. figuring out how I can get what I want using which application(s).
Case in point is my script save-dvb above where I use a variety of small aps in various combinations to select digital TV stations and then create the variables required to lock onto a dvb-t signal.
Below are several simple scripts I have written to do something trivial but useful (to me) which may be of interest to someone wanting to dissect them to learn little more about stream editing.
deps
I often build my own slack-packs and having a script to tell me what required packages I need to install before compiling was a covenience worth the time it took to write this script.
deps compares the installed packages listed in /var/log/packages with the packages listed in a slack-required file and prints each required packaged and the corresponding installed package line by line to stdout. If a required package is installed the required package and the installed package will be printed to stdout. If the required package is not installed, only the required package is printed to stdout.
I could have written this script to just print the required uninstalled package but I wanted to see both the required and the installed listed side by side in two columns albeit long file names throw the columns out of whack. Maybe I'll sit down on a rainy day a figure out how to fix that.
rick@rick:~$ deps
Reqd: alsa-lib >= 1.0.17a-i486-1 Installed: alsa-lib-1.0.17a-i486-1
Reqd: atk >= 1.23.5-i486-1 Installed: atk-1.23.5-i486-1
Reqd: cairo >= 1.6.4-i486-1 Installed: cairo-1.6.4-i486-1
Reqd: expat >= 2.0.1-i486-1 Installed: expat-2.0.1-i486-1
Reqd: fontconfig >= 2.6.0-i486-2 Installed: fontconfig-2.6.0-i486-2
Reqd: freetype >= 2.3.7-i486-1 Installed: freetype-2.3.7-i486-1
Reqd: gtk+2 >= 2.12.12-i486-1 Installed: gtk+2-2.12.12-i486-1
Reqd: libICE >= 1.0.4-i486-1 Installed: libICE-1.0.4-i486-1
Reqd: libSM >= 1.0.3-i486-1 Installed: libSM-1.0.3-i486-1
Reqd: libX11 >= 1.1.5-i486-1 Installed: libX11-1.1.5-i486-1
Reqd: libXau >= 1.0.4-i486-1 Installed: libXau-1.0.4-i486-1
snipped!
#!/bin/bash
# /usr/local/bin/deps 20090614rm
# parses ./slack-required (default) and compares required packages to
# installed packages listed in /var/log/packages. Alternatively enter
# a /path/filename after command to parse a file in a directory other
# than the pwd. This script isn't perfect or foolproof but it saves
# me time checking dependencies before building a package
# If a file is not entered after the command the script will read the
# slack-required file in the pwd
input_file=$1
test -n "$input_file"
if [ $? -eq 1 ]; then
input_file=slack-required
fi
# delete some lines that I know are givens
sed '/gcc*/d' $input_file | sed '/glibc*/d' > required.txt
# Assign a variable for the number of lines in required text so the script
# will stop looping a the last line
line_no="`sed -n '$=' required.txt`"
# Assign a variable to start counting at line 1
count=1
while [ "$count" -le "$line_no" ]
do
# This creates a test variable of the required package name less its version no.
test=`sed -n "$count"p required.txt | awk -F ">=" '{print $1}'`
# This creates a test variable of the required package name including its version no.
required=`sed -n "$count"p required.txt`
# See if the package has been installed
installed=`ls /var/log/packages | grep $test`
# Print the slack-required line plus installed package together, sed is needed to
# Remove any other installed packages printed to stdout in the above grep command
echo -e "Reqd: $required\t\tInstalled: $installed" | sed -n '/Reqd:/p'
count=$((count + 1))
done
# End of script
selektde:
This a script which allows me to switch between using a .kde for kde3 and or one for kde4 in the same /home/user directory. It it creates a variable named version by getting the first didgit of the installed KDE version number whic is then used to switch a case statement selecting the use of either .kde3 or .kde4.
#!/bin/bash
# /usr/local/bin/selektde 200905012rm
version=`ls /var/log/packages | grep kde | awk -F "-" '{print $2}' | awk -F "." '{print $1}' | sed 1q`
if [ $HOME/.kde ]; then
rm $HOME/.kde
fi
case $version in
3) ln -sf $HOME/.kde3 $HOME/.kde
;;
4) ln -sf $HOME/.kde4 $HOME/.kde
;;
esac
# end of script
get-quote:
This is actually a snippet from a signature script although I provide it below as a standalone script if you care play with it.
I wanted to set up my signature script so that it would use a different quote from fortune each time it ran. I only wanted to use quotes that were 10 lines or less and I wanted the quote displayed with the person the quote was attributed to.
I used sed to do a line count and to check to see if the last line had the chracters " --" in it which would indicate an attribution. If these two conditions were met it was a quote and would print to stdout. Putting everything in a loop kept it running untill a suitable quote was found.
#!/bin/bash
while [ TRUE ]
do
fortune > .quote-test
number=`sed -n '$=' .quote-test`
if [ "$number" -le 10 ]; then
test0=0
else
test0=1
fi
sed '$!d' .quote-test | grep -e '--' > /dev/null 2>&1
if [ "$?" = 0 ]; then
test1=0
else
test1=1
fi
if [ "$test0" -eq 0 -a "$test1" -eq 0 ]; then
cat .quote-test
break
fi
done
#end of script