NAME

OmniPITR::Tools::CommandPiper - Class for building complex pipe-based shell commands

SYNOPSIS

General usage is:

my $run = OmniPITR::Tools::CommandPiper->new( 'ls', '-l' );
$run->add_stdout_file( '/tmp/z.txt' );
$run->add_stdout_file( '/tmp/y.txt' );
my $checksummer = $run->add_stdout_program( 'md5sum', '-' );
$checksummer->add_stdout_file( '/tmp/checksum.txt' );

system( $run->command() );

Will run:

mkfifo /tmp/CommandPiper-26195-oCZ7Sw/fifo-0
md5sum - < /tmp/CommandPiper-26195-oCZ7Sw/fifo-0 > /tmp/checksum.txt &
ls -l | tee /tmp/z.txt /tmp/y.txt > /tmp/CommandPiper-26195-oCZ7Sw/fifo-0
wait
rm /tmp/CommandPiper-26195-oCZ7Sw/fifo-0

While it might look like overkill for something that could be achieved by:

ls -l | tee /tmp/z.txt /tmp/y.txt | md5sum - > /tmp/checksum.txt

the way it works is beneficial for cases with multiple different redirections.

For example - it works great for taking single backup, compressing it with two different tools, saving it to multiple places, including delivering it via ssh tunnel to file on remote server. All while taking checksums, and also delivering them to said locations.

DESCRIPTION

It is important to note that to make the final shell command (script) work, it should be run within shell that has access to:

  • mkfifo

  • rm

  • tee

commands. These are standard on all known to me Unix-alike systems, so it shouldn't be a problem.

MODULE VARIABLES

$fifo_dir

Temporary directory used to store all the fifos. It takes virtually no disk space, so it can be created anywhere.

Thanks to File::Temp logic, the directory will be removed when the program will end.

METHODS

new()

Object constructor.

Given options are treated as program that is run to generate stdout.

set_write_mode()

Sets whether writes of data should overwrite, or append (> vs. >>)

Accepted values:

  • overwrite

  • append

Any other would switch back to default, which is overwrite.

set_tee_path()

Sets path to tee program, when using tee is required.

$program->set_tee_path( '/opt/gnu/bin/tee' );

Value of tee path will be automatically copied to all newly created stdout and stderr programs.

add_stdout_file()

Adds another file destination for stdout from current program.

add_stdout_program()

Add another program that should receive stdout from current program, as its (the new program) stdin.

add_stderr_file()

Add another program that should receive stdout from current program, as its (the new program) stdin.

add_stderr_program()

Add another program that should receive stderr from current program, as its (the new program) stdin.

new_subprogram()

Helper function to create sub-programs, inheriting settings

command()

Returns stringified command that should be ran via "system" that does all the described redirections.

Alternatively, the command can be written to text file, and run with

bash /name/of/the/file

base_program()

Helper functions which returns current program, with its arguments, properly escaped, so that it can be included in shell script/command.

get_command_with_stdin()

This is the most important part of the code.

In here, there is single line generated that runs current program adding all necessary stdout and stderr redirections.

Optional $fifo argument is treated as place where current program should read it's stdin from. If it's absent, it will read stdin from normal STDIN, but if it is there, generated command will contain:

... < $fifo

for stdin redirection.

This redirection is for fifo-reading commands.

get_all_fifos()

To generate output script we need first to generate all fifos.

Since the command itself is built from tree-like data structure, we need to parse it depth-first, and find all cases where fifo is needed, and add it to list of fifos to be created. Actual lines to generate "mkfifo .." and "... < fifo" commands are in command() method.

All stdout and stderr programs (i.e. programs that should receive given command stdout or stderr) are turned into files (fifos), so after running get_all_fifos, no command in the whole tree should have any "stdout_programs" or "stderr_programs". Instead, they will have more "std*_files", and fifos will be created.

While processing all the commands down the tree, this method also checks if given command doesn't have multiple stderr_files.

Normally you can output stdout to multiple files with:

/some/command | tee file1 file2 > file3

but there is no easy (and readable) way to do it with stderr.

So, if there are many stderr files, new fifo is created which does:

tee file1 file2 > file3

and then current command is changed to have only this single fifo as it's stderr_file.

get_fifo_name()

Each fifo needs unique name. Method for generation is simple - we're using predefined $fifo_dir, in which the fifo will be named "fifo-$id".

This is very simple, but I wanted to keep it separately so that it could be changed easily in future.