Upgrade Tests
Postgres Upgrade Test
Postgres upgrade test is used for testing postgres version upgrade with citus installed. Before running the script, make sure that:
- You have downloaded citus.
- You have two different postgres versions.
Citus is installed to both of the postgres versions. For each postgres version:
- In citus source directory run:
bash make clean ./configure PG_CONFIG=<your path to postgres pg config> PG_CONFIG=<your path to postgres pg config> make sudo PG_CONFIG=<your path to postgres pg config> make install
Make sure you do this for both postgres versions, pg_config should be different for each postgres version.
Install
pipenv
and run incitus/src/test/regress
:
bash
pipenv install
pipenv shell
- Finally run upgrade test in
citus/src/test/regress
:
bash
pipenv run make check-pg-upgrade old-bindir=<old-bindir> new-bindir=<new-bindir>
To see full command list:
bash
pipenv run citus_tests/upgrade/pg_upgrade_test.py -help
How the postgres upgrade test works:
- Temporary folder
tmp_upgrade
is created insrc/test/regress/
, if one exists it is removed first. - Database is initialized and citus cluster is created(1 coordinator + 2 workers) with old postgres.
before_pg_upgrade_schedule
is run withpg_regress
. This schedule sets up any objects and data that will be tested for preservation after the upgrade. Itafter_pg_upgrade_schedule
is run withpg_regress
to verify that the output of those tests is the same before the upgrade as after.citus_prepare_pg_upgrade
is run in coordinators and workers.- Old database is stopped.
- A new database is initialized with new postgres under
tmp_upgrade
. - Postgres upgrade is performed.
- New database is started in both coordinators and workers.
citus_finish_pg_upgrade
is run in coordinators and workers to finalize the upgrade step.after_pg_upgrade_schedule
is run withpg_regress
to verify that the previously created tables, and data still exist. Router and realtime queries are used to verify this.
Writing new PG upgrade tests
The main important thing is that we have upgrade_{name}_before
and
upgrade_{name}_after
tests. The before
files are used to setup any objects
and data before the upgrade. The after
tests shouldn't have any side effects
since they are run twice (once before and once after the upgrade).
Furthermore, anything that is basic Citus functionality should go in the
upgrade_basic_before
/upgrade_basic_after
tests. This test file is used
during PG upgrades and Citus upgrades. Any features that don't work in old Citus
versions should thus be added to their own file, because that file will then
only be run during PG versions.
Citus Upgrade Test
Citus upgrade test is used for testing citus version upgrades from specific version to master. The purpose of this test is to ensure that a newly made change does not result in unexpected upgrade errors.
Currently the citus upgrade test assumes that:
- You have citus artifact tarballs, both for old version and master.
How the citus upgrade test work:
- The script takes
citus-pre-tar
andcitus-post-tar
which should contain citus artifacts. - It installs the given citus version from
citus-pre-tar
. - It creates a citus cluster(1 coordinator 2 workers).
- It reports the initial versions.
- It installs the checked out citus version from
citus-post-tar
. - It restarts the database and runs
ALTER EXTENSION citus UPGRADE
. - It runs
after_citus_upgrade
schedule to verify that the upgrade is successful. - It stops the cluster.
Note that when the version of citus changes, we should update MASTER_VERSION
with the new version of citus otherwise that will be outdated and it will fail.
There is a target for citus upgrade test. We run citus upgrade tests both in normal mode and in mixed mode. In mixed mode, we don't upgrade one of the workers. 'citus.enable_version_checks' : 'false'
is used to prevent citus from giving an error for mixed mode.
To see full command list:
bash
pipenv run upgrade/citus_upgrade_test.py -help
In order to run citus upgrade tests locally you can use:
bash
pipenv run make check-citus-upgrade-local citus-old-version=v8.0.0
For mixed mode:
bash
pipenv run make check-citus-upgrade-mixed-local citus-old-version=v8.0.0