Itrating Hashes in Perl: Many ways

1) If you just want the keys and do not plan to ever read any of the values, use keys():

foreach my $key (keys %hash) { ... }

2) If you just want the values, use values():

foreach my $val (values %hash) { ... }

3) If you need the keys and the values, use each():

keys %hash; # reset the internal iterator so a prior each() doesn't affect the loop
while(my($k, $v) = each %hash) { ... }

4) If you plan to change the keys of the hash in any way except for deleting the current key during the iteration, then you must not use each(). For example, this code to create a new set of uppercase keys with doubled values works fine using keys():

%h = (a => 1, b => 2);

foreach my $k (keys %h)
{
$h{uc $k} = $h{$k} * 2;
}

Command line options with Perl: EG

#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Std;
use vars qw/ %opts /;

getopts('he:c:', \%opts );
usage() if $opts{h};
usage() if($opts{e} eq "" or $opts{c} eq "");

sub usage(){
print "This program should be used as below\n";
print "usage: $0 [-h] [e petrol] [-c elgi]\n";
print " -h : this (help) message\n";
print " -e : energy name\n";
print " -c : compressor name\n";
print " example: $0 -e petrol -c elgi\n";
exit;
}

Tips and Tricks in Perl

Create an array from a string
@months = split ' ', "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
or
@months = qw/Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec/

Create a string from an array.
@stuff = ("hello", 0..9, "world"); $string = join '-', @stuff

Generate an array with even numbers from 1 to 100
perl -le '@even = grep {$_ % 2 == 0} 1..100; print "@even"'

The spaceship operator <=>
my @numbers = (-59, 99, 87, 1900, 42, 1, -999, 30000, 0);
my @sorted = sort { $a <=> $b } @numbers;
print "@sorted\n";
# output: -999 -59 0 1 42 87 99 1900 30000

Random password generator in perl

Here is a oneliner for this

perl -le 'print map { ("a".."z")[rand 26] } 1..8'

Here the map function executes ("a".."z")[rand 26] code 8 times (because it iterates over the dummy range 1..8). In each iteration the code chooses a random letter from the alphabet. When map is done iterating, it returns the generated list of characters and print function prints it out by concatenating all the characters together.

If you also wish to include numbers in the password, add 0..9 to the list of characters to choose from and change 26 to 36 as there are 36 different characters to choose from:

perl -le 'print map { ("a".."z", 0..9)[rand 36] } 1..8'

If you need a longer password, change 1..8 to 1..20 to generate a 20 character long password

From http://www.catonmat.net/blog/perl-one-liners-explained-part-four/

Getopts::Long HowTo in Perl

Here is an excellent link

http://www.devshed.com/c/a/Perl/Using-GetoptLong-More-Command-Line-Options-in-Perl/

Callback function in Perl: EG

#!/usr/bin/perl
#use strict;
#use warnings;

my $adder = sub {
my ( $arg1, $arg2 ) = @_;
return $arg1 + $arg2;
};

my $multiplier = sub {
my ( $arg1, $arg2 ) = @_;
return $arg1 * $arg2;
};

sub doit {
my ($action, $arg1, $arg2) = @_;
return $action->( $arg1, $arg2 );
}

# This returns 30
my $val1 = doit( $adder, 10, 20 );

# This returns 200
my $val2 = doit( $multiplier, 10, 20 );

Automatically include many file from many directory

In case you write a perl program to include a require with lot of file here is a small program for that.

#!/usr/bin/perl
use strict;
use warnings;
&putAllIncludeFiles;
sub putAllIncludeFiles{
        my @thefiles=readAllFilesFromDirectories("/home/tullas/lesson/framework/lib","/home/tullas/lesson/framework/conf");
        my $f;
        foreach $f (@thefiles){
                require $f;
        }
}
sub readAllFilesFromDirectories{
   my $filename;
   my @filenames;
   foreach (@_) {
        opendir ( DIR, $_ ) || die "Error in opening dir $_\n";
        while( ($filename = readdir(DIR))){
                unless ( ($filename eq ".") || ($filename eq "..") ){
                        #print("$filename\n");
                        push @filenames,$_ . "/" . $filename;
                        $filename = "";
                }
        }
   }
   return @filenames;
   closedir(DIR);
}

Parallel execution of command in BASH

This script can be used for parallel execution of any command or shell script for faster execution of that command. Say you have a program called prog which process some jpg files. Then you do

parallel -j 3 prog *.jpg

Here 3 prog program will run simultanously
or
parallel -j 3 "prog -r -A=40" *.jpg
to pass args to prog.

Furthermore, -r allows even more sophisticated commands by replacing asterisks in the command string by the argument:
parallel -j 6 -r "convert -scale 50% * small/small_*" *.jpg
I.e. this executes convert -scale 50% file1.jpg small/small_file1.jpg for all the jpg files. This is a real-life example for scaling down images by 50% (requires imagemagick).
Finally, here’s the script. It can be easily manipulated to handle different jobs, too. Just write your command between #DEFINE COMMAND and #DEFINE COMMAND END.

Here is the parallel program

#!/bin/bash
NUM=0
QUEUE=""
MAX_NPROC=2 # default
REPLACE_CMD=0 # no replacement by default
USAGE="A simple wrapper for running processes in parallel.
Usage: `basename $0` [-h] [-r] [-j nb_jobs] command arg_list
  -h  Shows this help
 -r  Replace asterix * in the command string with argument
 -j nb_jobs  Set number of simultanious jobs [2]
 Examples:
  `basename $0` somecommand arg1 arg2 arg3
  `basename $0` -j 3 \"somecommand -r -p\" arg1 arg2 arg3
  `basename $0` -j 6 -r \"convert -scale 50% * small/small_*\" *.jpg"

function queue {
 QUEUE="$QUEUE $1"
 NUM=$(($NUM+1))
}

function regeneratequeue {
 OLDREQUEUE=$QUEUE
 QUEUE=""
 NUM=0
 for PID in $OLDREQUEUE
 do
  if [ -d /proc/$PID  ] ; then
   QUEUE="$QUEUE $PID"
   NUM=$(($NUM+1))
  fi
 done
}

function checkqueue {
 OLDCHQUEUE=$QUEUE
 for PID in $OLDCHQUEUE
 do
  if [ ! -d /proc/$PID ] ; then
   regeneratequeue # at least one PID has finished
   break
  fi
 done
}

# parse command line
if [ $# -eq 0 ]; then #  must be at least one arg
 echo "$USAGE" >&2
 exit 1
fi

while getopts j:rh OPT; do # "j:" waits for an argument "h" doesnt
    case $OPT in
 h) echo "$USAGE"
  exit 0 ;;
 j) MAX_NPROC=$OPTARG ;;
 r) REPLACE_CMD=1 ;;
 \?) # getopts issues an error message
  echo "$USAGE" >&2
  exit 1 ;;
    esac
done

# Main program
echo Using $MAX_NPROC parallel threads
shift `expr $OPTIND - 1` # shift input args, ignore processed args
COMMAND=$1
shift

for INS in $* # for the rest of the arguments
do
 # DEFINE COMMAND
 if [ $REPLACE_CMD -eq 1 ]; then
  CMD=${COMMAND//"*"/$INS}
 else
  CMD="$COMMAND $INS" #append args
 fi
 echo "Running $CMD" 

 $CMD &

 #Change:
 #$CMD &

 #To:
 #eval “$CMD &”

 #If you want to do things like:
 #par.sh ‘tr -d ” ” * > $(basename * .txt)-stripped.txt’ *.txt

 #Without the eval it’ll treat > and $(basename…) as arguments to tr.


 # DEFINE COMMAND END

 PID=$!
 queue $PID

 while [ $NUM -ge $MAX_NPROC ]; do
  checkqueue
  sleep 0.4
 done
done
wait # wait for all processes to finish before exit

Source is at 
http://pebblesinthesand.wordpress.com/2008/05/22/a-srcipt-for-running-processes-in-parallel-in-bash/

ETL4ALL look like a good bet for small companies

Here is a link to ETL4ALL company website.

http://www.ikan.be/

Converting XML with XSLT and Java

Here is the code in Java to do this. No external library is needed.


import java.io.*;

/**
* A simple demo of JAXP 1.1
*/
public class SimpleJaxp {

/**
* Accept two command line arguments: the name of
* an XML file, and the name of an XSLT stylesheet.
* The result of the transformation
* is written to stdout.
*/
public static void main(String[] args)
throws javax.xml.transform.TransformerException {
if (args.length != 2) {
System.err.println("Usage:");
System.err.println(" java " + SimpleJaxp.class.getName( )
+ " xmlFileName xsltFileName");
System.exit(1);
}

File xmlFile = new File(args[0]);
File xsltFile = new File(args[1]);

javax.xml.transform.Source xmlSource =
new javax.xml.transform.stream.StreamSource(xmlFile);
javax.xml.transform.Source xsltSource =
new javax.xml.transform.stream.StreamSource(xsltFile);
javax.xml.transform.Result result =
new javax.xml.transform.stream.StreamResult(System.out);

// create an instance of TransformerFactory
javax.xml.transform.TransformerFactory transFact =
javax.xml.transform.TransformerFactory.newInstance( );

javax.xml.transform.Transformer trans =
transFact.newTransformer(xsltSource);

trans.transform(xmlSource, result);
}
}

XML Command tool for UNIX

Here is a tool which can help automation of XML file from UNIX

http://xmlstar.sourceforge.net/