Contents

• OmniPITR - omnipitr-archive
  • USAGE
  • DESCRIPTION
    • Remote destination specification
  • COMPRESSION
  • EXAMPLES
    • Minimal setup, with copying file to local directory:
    • Minimal setup, with copying file to remote directory over rsync:
    • 2 remote, compressed destinations, 1 local, with auto rotated logfile:
  • IMPORTANT NOTICES
  • COPYRIGHT

OmniPITR - omnipitr-archive

USAGE

/some/path/omnipitr/bin/omnipitr-archive [options] "%p"

Options:

--data-dir (-D)

Where PostgreSQL datadir is located (path)

--force-data-dir (-f)

Forces usage of given (or default) data dir even if it doesn't look like PostgreSQL data directory.

--dst-local (-dl)

Where to copy the wal segment on current server (path) (you can provide many of these).

You can also specify compression per-destination. Check COMPRESSION section of the doc.

--dst-remote (-dr)

Where to copy the wal segment on remote server. Supported ways to transport files are rsync and rsync over ssh. Please see DESCRIPTION for more information (you can provide many of these)

You can also specify compression per-destination. Check COMPRESSION section of the doc.

--dst-backup (-db)

Special destination, that cannot be compressed (existence of compression prefix will cause omnipitr-archive to raise exception, has to be absolute path (starts with /), and can be non-existing.

This is special directory that shouldn't usually exist, but if it exists - it will receive copy of every wal segment rotated.

The sole purpose of this option is to make omnipitr-master-backup working. If you do not intend to make hot backups on master - you can safely not use it.

--dst-pipe (-dp)

Specifies path to program that should receive xlog on stdin.

The program will be called multiple times (for each xlog separately). Name of the xlog will be given as first, and only, argument to the program.

If the file is to be compressed (using standard compression=/path syntax) - given file name will contain compression extension.

--temp-dir (-t)

Where to create temporary files (defaults to /tmp or $TMPDIR environment variable location)

--log (-l)

Name of logfile (actually template, as it supports %% strftime(3) markers. Unfortunately due to the %x usage by PostgreSQL, We cannot use %% macros directly. Instead - any occurence of ^ character in log dir will be first changed to %, and later on passed to strftime.

Please note that on some systems (Solaris for example) default shell treats ^ as special character, which requires you to quote the log filename (if it contains ^ character). So you'd better write it as:

--log '/var/log/omnipitr-^Y-^m-^d.log'

--state-dir (-s)

Name of directory to use as state-directory to handle errors when sending wal segments to many locations.

--pid-file

Name of file to use for pidfile. If it is specified, than only one copy of omnipitr-archive (with this pidfile) can run at the same time.

Trying to run second copy of omnipitr-archive will result in an error.

--parallel-jobs (-PJ)

Number of parallel jobs that omnipitr-archive can spawn to deliver archives to remote destinations.

--verbose (-v)

Log verbosely what is happening.

--not-nice (-nn)

Do not use nice for compressions.

--nice-path (-np)

Full path to nice program - in case you can't set proper PATH environment variable.

--gzip-path (-gp)

Full path to gzip program - in case you can't set proper PATH environment variable.

--bzip2-path (-bp)

Full path to bzip2 program - in case you can't set proper PATH environment variable.

--lzma-path (-lp)

Full path to lzma program - in case you can't set proper PATH environment variable.

--lz4-path (-ll)

Full path to lz4 program - in case you can't set proper PATH environment variable.

--xz-path (-xz)

Full path to xz program - in case you can't set proper PATH environment variable.

--rsync-path (-rp)

Full path to rsync program - in case you can't set proper PATH environment variable.

--version (-V)

Prints version of omnipitr-archive, and exists.

--help (-?)

Prints this manual, and exists.

--config-file (--config / --cfg)

Loads options from config file.

Format of the file is very simple - each line is treated as argument with optional value.

Examples:

--verbose
--host 127.0.0.1
-h=127.0.0.1
--host=127.0.0.1

It is important that you don't need to quote the values - value will always be up to the end of line (trailing spaces will be removed). So if you'd want, for example, to have magic-option set to "/mnt/badly named directory", you'd need to quote it when setting from command line:

/some/omnipitr/program --magic-option="/mnt/badly named directory"

but not in config:

--magic-option=/mnt/badly named directory

Empty lines, and comment lines (starting with #) are ignored.

DESCRIPTION

Call to omnipitr-archive should be in archive_command GUC in postgresql.conf.

Which options should be given depends only on installation, but generally you will need at least:

• --data-dir

  PostgreSQL "%p" passed file path relative to DATADIR, so it is required to know it. If not provided, it is assumed to be "." ( which is perfectly in line with PostgreSQL assumptions about archive_command ).

• --log

  to make sure that information is logged someplace about archiving progress

• one of --dst-local or --dst-remote

  to specify where to send the WAL segments to

If you'll specify more than 1 destination, you will also need to specify --state-dir

Of course you can provide many --dst-local or many --dst-remote or many mix of these.

Generally omnipitr-archive will try to deliver WAL segment to all destinations, and will fail if any of them will not accept new segment.

Segments will be transferred to destinations in this order:

1. All local destinations, in order provided in command line

2. All remote destinations, in order provided in command line

In case any destination will fail, omnipitr-archive will save state (which destinations it delivered the file to) and return error to PostgreSQL - which will cause PostgrerSQL to call omnipitr-archive again for the same WAL segment after some time.

State directory will be cleared after every successfull file send, so it should stay small in size (expect 1 file of under 500 bytes).

When constructing command line to put in archive_command PostgreSQL GUC, please remember that while providing "%p" "%f" will work, omnipitr-archive requires only "%p"

Remote destination specification

omnipitr-archive delivers WAL segments to destination using rsync program. Both direct-rsync and rsync-over-ssh are supported (it's better to use direct rsync - it uses less resources due to lack of encryption.

Destination url/location should be in a format that is usable by rsync program.

For example you can use:

• rsync://user@remote_host/module/path/

• host:/path/

To allow remote delivery you need to have rsync program. In case you're using rsync over ssh, ssh program has also to be available.

In case your rsync/ssh programs are in custom directories simply set $PATH environemnt variable before starting PostgreSQL.

COMPRESSION

Every destination can have specified compression. To use it you should prefix destination path/url with compression type followed by '=' sign.

Allowed compression types:

• gzip

  Compresses with gzip program, used file extension is .gz

• bzip2

  Compresses with bzip2 program, used file extension is .bz2

• lzma

  Compresses with lzma program, used file extension is .lzma

• lz4

  Compresses with lz4 program, used file extension is .lz4

• xz

  Compresses with xz program, used file extension is .xz

All compressions are done on NICE to make the operation as unobtrusive as possible.

All programs are passed --stdout option to compress without modifying source file.

If you want to pass any extra arguments to compression program, you can either:

• make a wrapper

  Write a program/script that will be named in the same way your actual compression program is named, but adding some parameters to call

• use environment variables

  All of supported compression programs use environment variables:

  • gzip - GZIP

  • bzip2 - BZIP2

  • lzma - XZ_OPT

  For details - please consult manual to your choosen compression tool.

EXAMPLES

Minimal setup, with copying file to local directory:

archive_command='/.../omnipitr-archive -D /mnt/data/ -l /var/log/omnipitr/archive.log -dl /mnt/wal_archive/ "%p"'

Minimal setup, with copying file to remote directory over rsync:

archive_command='/.../omnipitr-archive -D /mnt/data/ -l /var/log/omnipitr/archive.log -dr rsync://slave/postgres/wal_archive/ "%p"'

2 remote, compressed destinations, 1 local, with auto rotated logfile:

archive_command='/.../omnipitr-archive -D /mnt/data/ -l /var/log/omnipitr/archive-^Y-^m-^d.log -dr gzip=rsync://slave/postgres/wal_archive/ -dr bzip2=backups@backupserver:/mnt/backups/wal_archive/ -dl /mnt/wal_archive/ -s /var/lib/postgres/.omnipitr/ "%p"'

IMPORTANT NOTICES

• You need to provide state dir in case you have > 1 destination. This is crucial, as if you're using "-dr ... -db" - your number of destinations changes from 1 to 2 (depending on whether -db directory exists) - i.e. not all runs of omnipitr-archive will catch lack of state-dir as error!

• It is generally good to provide 2 destination - one used by slave system to keep warm standby, and another to be handy in case (for example) backup files get corrupted. This second set of xlogs should be cleared with cronjob that removes too old files. As for what is too old - 2x your backup schedule. I.e. if you're doing daily hot backups - you should keep 2 days worth of backup xlogs.

COPYRIGHT

The OmniPITR project is Copyright (c) 2009-2013 OmniTI. All rights reserved.