3.141592653589793238462643383279

Happy Pi day everyone. March 14th is Pi day because 3.14.

I created a BASH script to test the calculation in this video. It calculates Pi without pre-known values but by adding and subtracting fractions of one over odd numbers. 1/1 - 1/3 + 1/5 - 1/7 + 1/9 - 1/11 + 1/13 ... It gets more accurate the longer you let it run but only for a short time as fractional math in BASH is limited.

#!/bin/bash

D=1
P=0
i=1
while true; do
 O="-"
 (( i++ % 2 )) && O="+"
 P=$(echo "scale=66; ${P} ${O} 1/${D}" | bc)
 printf "%0.20f               \r" $(echo "scale=21; ${P} * 4" | bc)
 ((D+=2))
done

Tunneling UDP

We use Graylog at work, a very nice tool that collects logs from all our systems and helps us analyze them. We ran into a problem with a special cluster of servers that are isolated and could not get to our Graylog server. We only needed to connect them to Graylog for a short time so this is not a long term solution.

These systems have never been able to connect to Graylog so they were configured to send all logs to 127.0.0.1, and that is great because we can setup a listening tunnel on localhost to forward the connection and we can do all this without making changes to the target systems.

The Graylog server has SSH access to the target systems so we use that to create a reverse tunnel. SSH works over TCP but Graylog uses UDP so we use NC (netcat) to convert the UDP to TCP for tunneling and then convert it back again on the server side.

The first command forks a local listener to send TDP packets to our Graylog on UDP.
The second command opens a reverse tunnel and starts listening on UDP to send packets back via the tunnel over TCP.

nc -l 127.0.0.1 12203 | nc -u 127.0.0.1 12201 &
ssh target2 -R 12201:localhost:12203 "nc -kluw 0 127.0.0.1 12201|nc 127.0.0.1 12201"

Some important points.

• Each connection needs its own port, Graylog is using 12201, so target1 would listen on 12202 and target2 is using 12203
• Netcat wants to disconnect when the logger on target finishes a message to that nc uses 'k' to keep the connection alive and is uses 'w 0' to have no wait time out.

Update

After a day or so running netcat likes to die so this little change will automatically relaunch nc instantly by running this on the target side.

while nc -kluw 0 127.0.0.1 12201; do true; done | while cat - | nc 127.0.0.1 12201; do true; done

Recursion

My son is learning C++ in school and was given an assignment to create a program that would convert Roman Numerals to decimal integers. Most people new to C++ would do this using a while loop. Up on completion of his version I will show him an alternate method that does not use a while loop but rather is a good example of recursion.

This example makes it much harder to debug as the calculation happens in the recursion call, and I have placed that in the return of the same function, but this is just for fun.

#include <iostream>
#include <string>

using namespace std;

int Decode(const char* CodeString, int Position, int PreviousValue) {
    // When we have reached the end of the string
    if(CodeString[Position] == '\0') return(0);

    int NextValue = 0;
    switch (CodeString[Position]) {
        case 'M': NextValue = 1000;  break;
        case 'D': NextValue = 500;  break;
        case 'C': NextValue = 100;  break;
        case 'L': NextValue = 50;   break;
        case 'X': NextValue = 10;   break;
        case 'V': NextValue = 5;    break;
        case 'I': NextValue = 1;    break;
    }
    if(PreviousValue < NextValue) NextValue -= PreviousValue * 2;
    return(NextValue + Decode(CodeString, Position+1, NextValue));
}

int main()
{
    cout << "Please enter a value as valid formated Roman numerals ";
    string UserInput = "";
    cin >> UserInput;
    int Answer = Decode(UserInput.c_str(), 0, 0);
    cout << "The answer is " << Answer << endl;
    return 0;
}

Parsing a moving target.

A problem was presented to me where the raw data was expected to have columns added and moved around at some point but a report was needed that could deal with this. I chose to use Perl Named Capture Containers to solve this problem.

#!/usr/bin/perl -w

# This is an example of using named capture containers to parse data from lines 
# when the columns of data could move around, new columns are added and even if 
# the column is removed.

# Matching a pattern with multiple parts cannot deal with columns that move or are missing.
# Each part must be matched on it's own line.
# Every time a match is made it is put into the hash for use later.

# sample3.data as input
#######################
# 01:00:00 httpd=on sshd=on crond=on ntpd=on winbind=off cups=off
# 01:01:00 [567] httpd=on sshd=on crond=on ntpd=on winbind=off cups=off
# 01:02:00 [567] crond=on ntpd=on winbind=off httpd=off cups=off sshd=on
# 01:03:00 [PID:567] ntpd=on winbind=off cups=off sshd=on named=on httpd=on crond=on 
# 01:04:00 [PID:567] named=on httpd=on crond=on

# results from output
#######################
# Time  httpd ntpd sshd 
# 01:00:00  on  on  on 
# 01:01:00  on  on  on 
# 01:02:00  off  on 
# 01:03:00  on  on  on 
# 01:04:00  on 


my %DataSet; 
# The data set is dynamically gathered so to change
# the report just add or remove column names here.
my @ReportFields = ( "httpd", "ntpd", "sshd" );

sub Pack {
 my $Time = shift;
 my $FieldName = shift;
 my $DataValue = shift;
 $DataSet{$Time}->{$FieldName} = $DataValue;
}

while (<>) {
 $Line = $_;
 $Line =~ s/\n//;
 $Line =~ m/^(?<time>\d\d:\d\d:\d\d).*/;
 my $NewTime = $+{time};

 foreach my $R ( sort @ReportFields) {
  if ($Line =~ m/.* \Q$R\E=(?<value>\w*) .*/) { Pack($NewTime, $R, $+{value}) }; 
 }
}

printf "Time\t\t";
foreach my $F ( sort @ReportFields ) {
 printf $F."\t";
}
printf "\n";

foreach my $Time (sort keys %DataSet) {
 printf $Time."\t";
 foreach my $F ( sort @ReportFields ) {
  printf " ".$DataSet{$Time}->{$F}."\t" if (defined $DataSet{$Time}->{$F});
 }
 printf "\n";
}

Stop eating my pipe!

We love to use while loops in our scripts and they are a great way to read a file one line at a time to get a job done.

For example:

cat myFile.txt | while read LINE; do
   echo $LINE
   sleep 1
done

Now comes along SSH and lets say that your file contains a list hostnames you want to get uptime from:

cat myHostsFile.txt | while read LINE; do
   echo -n "Host = $LINE "
   ssh $LINE "updtime"
done

How disappointed you are when your loops stops after the first host. This is because every child process inherits it's first three file descriptors from it's parent, so SSH takes everything from STDIN for itself.

Sometimes you may want that but this time you don't. What can you do?

The simple solution here is to disassociate SSH from STDIN, and you do this using a simple re-director.

cat myHostsFile.txt | while read LINE; do
   echo -n "Host = $LINE "
   ssh 0>/dev/zero $LINE "updtime"
done

Something to keep in mind is that STDIN supplies a data stream, it does not take it. So for this reason we attache to /dev/zero not /dev/null.

LibreOffice BASIC Easy GetCell

Improving on GetCell from an earlier post I have added a Column2Index function to allow you to pass the names of the column and row rather then the index.

If the column is a number then it will use the values as index values, otherwise it will convert both the column and the row.

These two calls are the same
GetCell("Sheet1", "D", 3) <- using the names of the columns and rows
GetCell("Sheet1", 3, 2) <- using the index of the columns and rows


For cell that contain numbers, ether fixed or formula generated you can simply read from them or write to them like so.

If GetCell("Sheet1", "D", 3).Value > 100 Then
   GetCell("Sheet1", "D", 3).Value = 0
End If

For cells that contain text you use GetCell("Sheet1", "D", 3).String the same way you would any normal string variable.

Function Column2Index(ColNameX As String)
 ReturnInt = 0
 ColNameU = Ucase(ColNameX)
 While ColNameU <> ""
  LastChar = Left(ColNameU, 1)
  ColNameU = Right(ColNameU, Len(ColNameU) - 1)
  ReturnInt = ReturnInt * 26
  ReturnInt = ReturnInt + Asc(LastChar) - 64
 Wend
 Column2Index = ReturnInt - 1
End Function

' Now accepts column and row as they are named or their index value
' These two calls are the same
' GetCell("Sheet1", "D", 3) <- using the names of the columns and rows
' GetCell("Sheet1", 3, 2) <- using the index of the columns and rows


Function GetCell(SheetName As String, Column, Row) As com.sun.star.table.XCell
 ColumnNumber = 0
 RowNumber = 0
 If ISNUMERIC(Column) Then  
  ColumnNumber = Column
  RowNumber = Row
 Else
  ColumnNumber = Column2Index(Column)
  RowNumber = Row - 1 
 End If
 AllSheets = ThisComponent.Sheets()
 FindSheet = AllSheets.GetByName(SheetName)
 TheCell = FindSheet.GetCellByPosition(ColumnNumber,RowNumber)
 GetCell = TheCell
End Function

Simple Process Locking In Perl

My co-work Rick asked for a simple process lock control in Perl that would work over a server load balanced pool with a common NFS share. The easiest way is a loop that tries to create a directory, if the command fails then it waits and tries again, if the command succeeds then it has the lock and can do it's job. When done it removes the directory and another process gets to continue.

We started to build in an extra step so that if the script dies during the "meat and potatoes" section anther process can check if the PID from the previous process is still running and if not remove it. We did not get that far, I will let you figure the rest out.

The first usleep command is just to prevent process hammering, you should set it to match your expected time for a process to run. The second usleep is to simulate process congestion and is only for testing, you should remove it from your project.

#!/usr/bin/perl -w

use Time::HiRes qw(usleep nanosleep);

my $COUNTER=0;
my $dir = "/mnt/share/templock.dir";

while (! mkdir($dir) )
{
 $COUNTER++; 
 usleep(100);

 printf ("$$ waiting: $COUNTER\n");
 if ($COUNTER > 1000)
 {
  #open(my $fh, "<", $dir/PID");
  #my $row = <$fh>;
  
  die("squak\n");
 }
}

open(my $fh, ">", "$dir/PID");
printf $fh "$$\n";
close($fh);

#meat and potatoes
usleep(10000);


unlink("$dir/PID");
rmdir($dir);