CLI Tools: screen

One of my favourite CLI tools is screen. Basically it lets you detach from your shell session and it keeps running. You can reattach to it later from anywhere, start your shell from a local TTY then reattach via SSH. You can even share your session and have others join in so you can collaborate on shell.

The sample code for today is how to make an init script to run your program as a detached daemon using screen. The cool thing about this is it lets you connect to it at any time to see what it is doing. This is not recommend for normal use but it is a great way to debug that new program before you commit to making it a proper daemon.

#!/bin/bash

cd /etc/init.d

start() {
 echo "Starting Service: "
 APP_CMD="/usr/local/bin/MyProgram.pl"
 screen -dmS pmsgd $APP_CMD
 echo
}

stop() {
 echo "Cannot stop screen programs this way. Open the screen sessions."
 echo
}

restart() {
 stop
 start
}

# See how we were called.
case "$1" in
  start)
 start
 ;;
  stop)
 stop
 ;;
  restart)
 restart
 ;;
  *)
 printf "Usage: %s {start|stop|restart}\n" "$0"
 exit 1
esac

exit 0

Find zen for your users with Zenity

"That is a great program you made for us but we don't like that command line thing. Can you make it a GUI?" To which you reply, "Why yes I can. Just give me a few minutes."

Shell scripts are great. You have all the power from thousands of Linux commands that you can connect together to make the most impressive scripts since the ENIAC. No matter how great your scripts are there will be users that just don't like it because of the command line.

You have three options:  create a web server interface,  re-write it as a binary with graphical objects or use pre-existing dialog tools to add GUI features to your program. Zenity can do that for you.

There are a few dialog tools like zenity, kdialog, xdialog and may more. They are not very hard to use and can provide a lot of helpful iteration with things like information boxes, input boxes, selections and radio buttons.

zenity --entry --text "Please enter your name"

Daemonize your code

If you write a shell script or program and need to fork it to the background as a daemon there could be some issues. The normal standard daemon library may not be able to totally disconnect from any open file descriptors. This simple C++ program can do it all for you and it is so easy to use, just put this program in front of your normal command and it will turn any normal program in to a system level daemon.

#include <sys/types.h>
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <errno.h>
#include <unistd.h>
#include <syslog.h>
#include <string.h>

int main(int argc, char *argv[]) {
 int i;
 // int reterr;
 pid_t pid, sid;
 
 //Fork the Parent Process
 pid = fork();
 
 if (pid < 0) { exit(EXIT_FAILURE); }
 
 //We got a good pid, Close the Parent Process
 if (pid > 0) { exit(EXIT_SUCCESS); }
 
 //Change File Mask
 umask(0);
 
 //Create a new Signature Id for our child
 sid = setsid();
 if (sid < 0) { exit(EXIT_FAILURE); }
 
 //Change Directory
 //If we cant find the directory we exit with failure.
 if ((chdir("/")) < 0) { exit(EXIT_FAILURE); }
 
 //Close Standard File Descriptors
 close(STDIN_FILENO);
 close(STDOUT_FILENO);
 close(STDERR_FILENO);
 
 //----------------
 //Main Process
 //----------------
 for(i=0; i < argc - 1; i++) {
  argv[i]=argv[i+1];
 }
 argv[argc-1] = '\0';
 execv(argv[0], argv);
 //reterr = execv(argv[0], argv);
 //printf("execv failed with '%s'\n", strerror(errno));

 //Close the log
 closelog ();
}

Objective thinking in JavaScript

It seems all too common that small projects grow to become big projects and then problems happen. Something that is very frustrating is when your JavaScript code starts to act goofy and you cannot figure out why. A lot of times it is because you are over lapping the name space and a value is getting altered in some other section of code that you did not expect. This can kill a project, it can become so hard to debug that you eventually scrap the code and move on.

Before you write a single line of JavaScript, think about how you can objectify it. In JavaScript you can wrap your code in object like statements so that name space does not get clobbered by other code.

Notice how the functions are set as variables and calls to them are invoked as if it was a real object language.

function myobject() {
   var MyVar = 0;

   this.PageSetup = function() {
      var _this = this;
   }

   this.SendCommand = function(Url) {
      var GetHttp = new XMLHttpRequest();
      GetHttp.open("GET", Url, true);
      GetHttp.send(null);
   }
}; // end of object

var myobject1 = new myobject();

myobject1.SendCommand("mydomain.url/doStuff");

Sqlite, the DB engine that could

sqlite3 is a great little DB engine for projects that need a small DB. It uses common SQL commands to store data in a local file. The file created by using the sqlite3 commands is the only database. Simple and affective it can be added to any script, or program with very little code.


BASH

echo "SELECT MAX(id)+1 FROM mytable;" | sqlite3 mydata.db 

PERL

use DBI;
$dbh = DBI->connect("dbi:SQLite:dbname=mydata.db", "", "", { sqlite_use_immediate_transaction => 1, RaiseError =>1 }) or die $DBI::errstr;
my $NextId = $dbh->selectall_arrayref("SELECT MAX(id)+1 FROM mytable;")->[0][0];

Perl data collation

When you need to gather data from many different sources and combine it in to one table of data there are a few ways to do it. The pr command can take data from many files and convert them to one table. Awk is a good choice also. Here is how I did it using Perl.

Here is the sample input data.

server1
09:53:01 94
09:53:02 96
09:53:03 95
server2
09:53:01 94
09:53:02 96
09:53:03 95
server3
09:53:01 94
09:53:02 96
09:53:03 95
server4
09:53:01 94
09:53:02 96
09:53:03 95
server5
09:53:01 94
09:53:02 96
09:53:03 95
server6
09:53:01 94
09:53:02 96
09:53:03 95

Here is the code. Save it as a Perl file and pipe your data into it.

#!/usr/bin/perl

while (<>) {
 @data = split(/\t/);
 if($data[0] =~ m/^\d/) {  
  $Times{$data[0]} = 1;
  chomp($data[1]);
  $Result{$Server}{$data[0]} = $data[1];
 } else {
  chomp($data[0]);
  $Servers{$data[0]} = 1;
  $Server = $data[0];
 }
}
printf "Time     \t";
foreach $S (keys %Servers) {
 printf "$S\t";
}
print "\n";
foreach $T (keys %Times) {
 print "$T\t";
 foreach $S (keys %Servers) {
  printf "$Result{$S}{$T}\t";
 }
 print "\n";
}

Here is your output.

Time      server3 server1 server2 server5 server4 server6 
09:53:02 96 96 96 96 96 96 
09:53:03 95 95 95 95 95 95 
09:53:01 94 94 94 94 94 94 

Better shell history for power users.

One of the very first things I set up on any new OS install is my history. I like to keep a lot of it and I like to know when I ran the command. This is very easy to do with these two environment variables set in your profile. You can set them locally or globally.

[royce@royce-desktop ~]$ cat /etc/profile.d/history.sh 
export HISTSIZE=100000
export HISTTIMEFORMAT="[%F %T] "
[royce@royce-desktop ~]$ history |tail -n 2
74519  [2015-01-22 10:52:40] cat /etc/profile.d/history.sh 
74520  [2015-01-22 10:52:41] history |tail -n 2

The Power of AWK: svn log to RPM change history.

For many years I used awk and gawk to just do simple column parsing because it is so good at stripping out the white space. Recently I have begun to use awk to do more complex task. One of these is reading the subversion log and converting it to RPM change log history.

I am using this awk as part of my build system to automatically generate rpm.spec files as one step of my build system for my many projects.
The goal is to turn this

------------------------------------------------------------------------
r4 | royce | 2015-01-21 13:03:49 -0700 (Wed, 21 Jan 2015) | 1 line

New SVN project
------------------------------------------------------------------------
r3 | royce | 2015-01-21 10:13:03 -0700 (Wed, 21 Jan 2015) | 1 line


------------------------------------------------------------------------
r2 | royce | 2015-01-21 10:12:25 -0700 (Wed, 21 Jan 2015) | 1 line


------------------------------------------------------------------------
r1 | royce | 2015-01-21 10:02:31 -0700 (Wed, 21 Jan 2015) | 1 line

First build
------------------------------------------------------------------------

In to this

* Wed Jan 21 2015 Revision r4
- First build
- New SVN project

Below is the awk code.
Save it as the file SvnLog2SpecChangeLog.awk.
Then from your project directory run it as svn log | awk -f SvnLog2SpecChangeLog.awk

BEGIN {
   DI=1;
   LASTDATE="";
}

{
   # print "working on "$0
   if($1 == "------------------------------------------------------------------------") {
      nextrow=0
      # print "split row"
   } else {
      # print "text row"
      if(nextrow > 0) {
         if ($1 == "") {
            # print "Blank row"
            nextrow++;
         } else {
            DOCS[DI++]=$0;
         }
      }
      if(nextrow==0) {
         # print "Revison row also contains date"
         NEWREV=$1;
         NEWDATE=$5;
         nextrow++;
         if(LASTDATE != "") {
            # print "LASTDATE="LASTDATE;
            if((LASTDATE!=NEWDATE) && (DI > 1)) {
               DATECMD="date +\"%a %b %d %Y\" -d"LASTDATE
               DATECMD | getline DATEOUT;               
               printf("* %s Revision %s\n", DATEOUT, LASTREV);
               for (DK in DOCS) {
                  printf("- %s\n", DOCS[DK]);
               }
               printf "\n";
               delete DOCS;
               DI=1
               LASTDATE=NEWDATE;
               LASTREV=NEWREV;
            }
         } else {
            LASTDATE=NEWDATE;
            LASTREV=NEWREV;
         }
      }
   }
}

END {
   DATECMD="date +\"%a %b %d %Y\" -d"LASTDATE
   DATECMD | getline DATEOUT;               
   printf("* %s Revision %s\n", DATEOUT, LASTREV);
   for (DK in DOCS) {
      printf("- %s\n", DOCS[DK]);
   }
   printf "\n";
}