In the quest to keep track of farm activity I have developed a little How-to on setting up a web based farm monitoring system.
First some definitions.
Web server – the place to collect, collate and build the html page. This could be any linux/unix platform but will probably be your LTSP server.
Monitored Client – The platform to make the collect from
SSH – secure shell protocol to make the call from the server to the client.
RSH - shell protocol to make the call from the server to the client. (non-secure)
Apache – the web server software
First an operational web server needs to be installed. The base Fedora distros contain a version if Apache. The re are plenty of basic How-to floating about on getting a Apache up and running and so I won’t delve into that at all. Suffice it to say you will need to be able to see the default Apache index.html page before proceeding.
In order to collect the data a communications method must be employed between the web server and the client. I suggest using SSH. This requires the exchange of keys between the platforms.
http://kimmo.suominen.com/docs/ssh/ provides and excellent reference to get SSH operational for a specific user and does more justice to the subject than I could here. Pay attention to the sections on Key generation, exchange and remote command execution.
Make sure a simple shell command can be run from the server to the client.
Run from the server ssh <client_hostname> ls / This needs to return a listing of the remote host root directory without prompting for password.
The client scripts
A simple method of delivery and parse is employed to generate the data on the client machine Here is an example of the script that is run on the client, It returns a single line of data to the calling program on the server. It is saved in the user directory of the D2OL client box, in my case root. It may look complicated at first glance but is really a very simple program.
#!/bin/bash
D2OL="/root/SengentD2OL/D2OL" #Home dir for the D2OL client
CAN="" # Initialize some variables
TARG=""
LINE=`uptime` # Populate the LINE variable with the output of uptime command
# this sections determines the different variations in the way uptime reports time.
if [ `echo $LINE | grep day | wc -l` -eq "1" ]
then
if [ `echo $LINE | grep min | wc -l` -eq "1" ]
then
DAY=`echo $LINE | cut -f3 -d" "`
HOUR="0"
MIN=`echo $LINE | cut -f5 -d" " `
LOAD=`echo $LINE | cut -f11 -d" " |cut -f1 -d","`
else
DAY=`echo $LINE | cut -f3 -d" "`
HOUR=`echo $LINE | cut -f5 -d" " | cut -f1 -d ":"`
MIN=`echo $LINE | cut -f5 -d" " | cut -f2 -d ":" | cut -f1 -d","`
LOAD=`echo $LINE | cut -f10 -d" " |cut -f1 -d","`
fi
elif [ `echo $LINE | grep min | wc -l` -eq "1" ]
then
DAY="0"
HOUR="0"
MIN=`echo $LINE | cut -f3 -d" " `
LOAD=`echo $LINE | cut -f9 -d" " |cut -f1 -d","`
else
DAY="0"
HOUR=`echo $LINE | cut -f3 -d" " | cut -f1 -d ":"`
MIN=`echo $LINE | cut -f3 -d" " | cut -f2 -d ":" | cut -f1 -d","`
LOAD=`echo $LINE | cut -f8 -d" " |cut -f1 -d","`
fi
# This section gets the pertinent data from the d20l.out.txt file
ID=`head -3 $D2OL/d2ol.out.txt | grep "Node ID:" | cut -f4 -d" "`
tail -12 $D2OL/d2ol.out.txt > /tmp/tail.out
LINE=`cat /tmp/tail.out | grep "Analyzing Candidate" `
CAN=`echo $LINE | cut -f3 -d" "`
TARG=`echo $LINE | cut -f2 -d":"`
LINE=`cat /tmp/tail.out | grep "Tasks Assigned:"`
ASS=`echo $LINE | cut -f3 -d" "`
LINE=`cat /tmp/tail.out | grep "Tasks Downloaded:" `
DL=`echo $LINE | cut -f3 -d" "`
LINE=`cat /tmp/tail.out | grep "Results to Upload:" `
RES=`echo $LINE | cut -f4 -d" "`
LINE=`cat /tmp/tail.out | grep "Completed Candidates:" `
COMP=`echo $LINE | cut -f3 -d" "`
# this printf statement returns the data in a form that can be parsed at the server. Note
# the quotes around $TARG, this is required cause it contains spaces that printf would
# otherwise interpret.
printf "%s %s %s %s %s %s %s %s %s %s ;%s
" $DAY $HOUR $MIN $LOAD $ID $ASS $DL $RES $COMP $CAN "$TARG"
exit 0
The output looks like this:
6 20 04 0.98 114747 242 243 7 12834 IBS_STOCK3S-52575 ; Smallpox Target I
The next step is to get a web page format that you like and design the elements in the page that you prefer. In my case I just used a word processor to design a page and marked a place to a set of table entries. You will cut this file at the table marking into 2 different files. It will all make sense soon.
Now create a directory to run the server scripts from. Then place the following script in that directory. I used the server cgi-bin directory and called the script status.sh
#!/bin/bash
printf "<CENTER><TABLE BORDER="7" CELLPADDING="10"\ CLASS=\"boldtable\"><TH><FONT SIZE=10>%s</TH></TABLE></CENTER>
"\ "`date`" >./tmp.html
printf "<CENTER><TABLE BORDER="7" CELLPADDING="10" CLASS=\"boldtable\">
">>./tmp.html
printf "<TH COLSPAN=\"12\"><FONT SIZE=8>Statistics</TH>
" >> ./tmp.html
printf\ "<TR><TH>Host</TH><TH>Up<br>Days</TH><TH>Up<br>Hours</TH><TH>Up<br>Min</TH><TH>Load<br>Average</TH>
" >> ./tmp.html
printf\ "<TH>Node<br>ID</TH><TH>Tasks<br>Assigned</TH><TH>Tasks<br>Download</TH><TH>Results<br>Upload</TH><TH>Completed</TH><TH>Candidate</TH>\
<TH>Target</TH></TR>
" >> ./tmp.html
# the file boxes.ssh contains the names of the clients to access
for i in `cat ./boxes.ssh`
do
ping -c 1 $i 2>&1 > /dev/null # ping if up get the data
if [ $? -eq "0" ]
then
LINE=`ssh $i ./get_d2ol.sh` # the call to the client box
# chop up the input into separate variables
DAY=`echo $LINE | cut -f1 -d" "`
HOUR=`echo $LINE | cut -f2 -d" "`
MIN=`echo $LINE | cut -f3 -d" " `
LOAD=`echo $LINE | cut -f4 -d" "`
ID=`echo $LINE | cut -f5 -d" "`
ASS=`echo $LINE | cut -f6 -d" "`
DL=`echo $LINE | cut -f7 -d" "`
RES=`echo $LINE | cut -f8 -d" "`
COMP=`echo $LINE | cut -f9 -d" "`
CAN=`echo $LINE | cut -f10 -d" "`
TARG=`echo $LINE | cut -f2 -d";"`
INT=`echo $LOAD | cut -f1 -d"."` # integer part of load avg. from uptime
DEC=`echo $LOAD | cut -f2 -d"."` # decimal part of load avg. from uptime
if [ $CAN == ";" ] # in if the output is empty the client is switching to a new WU
then
TARG="Switching"
CAN="Switching"
fi
if [ $INT -eq "0" -a $DEC -lt "50" ] # if the load avg. is to low print in red
then
printf "<TR BGCOLOR=\"#FF0000\"><TD><FONTCOLOR=\"#FF0000\">%s</FONT></TD><TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD>
" $i $DAY $HOUR $MIN $LOAD $ID>>./tmp.html
else # print in normal
printf "<TR><TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD>
"$i $DAY $HOUR $MIN $LOAD\ $ID>>./tmp.html
fi
printf "<TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD></TR>
" $ASS $DL $RES $COMP $CAN "$TARG" >>./tmp.html
else # if the ping failed then print the down message in RED
LINE="DOWN"
DAY="DOWN"
HOUR="DOWN"
MIN="DOWN"
LOAD="DOWN"
ID="DOWN"
ASS="DOWN"
DL="DOWN"
RES="DOWN"
COMP="DOWN"
CAN="DOWN"
TARG="DOWN"
printf "<TR BGCOLOR=\"#FF0000\"><TD><FONT COLOR=\"#FF0000\">%s</FONT></TD><TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>\
%s</TD><TD>%s</TD>
" $i $DAY $HOUR $MIN $LOAD $ID>>./tmp.html
printf "<TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD><TD>%s</TD></TR>
" $ASS $DL $RES $COMP $CAN "$TARG" >>./tmp.html
fi
done
printf "</TABLE></CENTER>
" >> ./tmp.html
printf "</BODY> </HTML>
" >> ./tmp.html
# done making the Bottom half of the html page
cp ./top.txt ../html/index.html # copy the top half of the page
cat ./tmp.html >> ../html/index.html # cat the dynamic portion on the ass end.
exit 0
My example of top.txt that came from my design page.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="CONTENT-TYPE" CONTENT="text/html; charset=utf-8">
<TITLE>The Major Strikes Again</TITLE>
<META NAME="GENERATOR" CONTENT="OpenOffice.org 1.1.2 (Linux)">
<META NAME="CREATED" CONTENT="20050101;8035300">
<META NAME="CHANGED" CONTENT="20050101;11075700">
<STYLE>
<!--
@page { size: 8.5in 11in; margin-left: 1.25in; margin-right: 1.25in; margin-top: 1in; margin-bottom: 1in }
P { margin-bottom: 0.08in }
-->
</STYLE>
</HEAD>
<BODY BGCOLOR="grey" LANG="en-US" DIR="LTR">
<P ALIGN=CENTER STYLE="margin-top: 0.17in; margin-bottom: 0.2in; page-break-after: avoid">
<A HREF="http://www.wunderground.com/US/AZ/Phoenix.html"><IMG SRC="http://banners.wunderground.com/banner/bigwx_both_cond/language/www/US/AZ/Phoenix.gif" NAME="Graphic2" ALT="Click for Phoenix Arizona Forecast" ALIGN=BOTTOM WIDTH=468 HEIGHT=60 BORDER=0></A><BR><FONT SIZE=7><FONT FACE="Albany, sans-serif"><FONT COLOR="#0047ff">
<MARQUEE SCROLLAMOUNT=5 SCROLLDELAY=10>Farming for TPR on D2OL</MARQUEE></FONT></FONT></FONT></P>
<P ALIGN=CENTER><SPAN ID="Frame1" DIR="LTR" STYLE=" width: 8.73in; height: 6.55in; border: none; padding: 0in; background: #ffffff">
<P STYLE="margin-top: 0.08in"><IMG SRC="farm.jpg" NAME="Graphic1" ALIGN=CENTER WIDTH=100% BORDER=0><BR ><BR><BR>
</P>
</SPAN>