Changelog

All notable changes to pgclone are documented in this file.

[3.6.0]

Added

  • GDPR/Compliance Masking Report: pgclone_masking_report(schema) generates an audit report listing all sensitive columns, their masking status, and recommendations
    • Detects sensitive columns using shared sensitivity_rules (~40 patterns across 10 categories)
    • Checks if a masked view (table_masked) exists for each table
    • mask_status: MASKED (view) or UNMASKED
    • recommendation: either “OK - masked via view” or “Apply mask strategy: X”
    • Returns SET OF (schema_name, table_name, column_name, sensitivity, mask_status, recommendation)
    • Useful for GDPR, HIPAA, SOX compliance audits — “prove no PII exists unmasked”

Changed

  • Refactored sensitivity rules into shared SensitivityRule struct and pgclone_match_sensitivity() helper — used by both pgclone_discover_sensitive and pgclone_masking_report
  • Version bumped to 3.6.0

[3.5.0]

Added

  • Clone Verification: pgclone_verify() compares row counts between source and target databases, table by table, returning a side-by-side comparison
    • Two overloads: pgclone_verify(conninfo) for all schemas, pgclone_verify(conninfo, schema) for a specific schema
    • Returns SET OF (schema_name, table_name, source_rows, target_rows, match)
    • Match indicators: ✓ (equal), ✗ (different), ✗ (missing) (table not on target)
    • Uses pg_class.reltuples for fast approximate counts without full table scans
    • Works with regular and partitioned tables
  • 5 new pgTAP tests (84 total): verify function runs, row counts match after clone, correct column count in result

Changed

  • Version bumped to 3.5.0

[3.4.0]

Added

  • Clone Roles with Permissions and Passwords: pgclone_clone_roles() clones database roles from source to local, including encrypted passwords, role attributes, and all privilege grants
    • pgclone_clone_roles(conninfo) — clone all non-system roles
    • pgclone_clone_roles(conninfo, 'role1,role2') — clone specific roles (comma-separated)
    • Clones role attributes: LOGIN, SUPERUSER, CREATEDB, CREATEROLE, REPLICATION, INHERIT, CONNECTION LIMIT, VALID UNTIL
    • Copies encrypted passwords from pg_authid (requires superuser on both source and target)
    • If target role already exists: updates password, attributes, and applies permissions additively
    • Clones role memberships (GRANT role TO role)
    • Clones schema-level privileges (USAGE, CREATE) via aclexplode()
    • Clones table-level privileges (SELECT, INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER)
    • Clones sequence privileges (USAGE, SELECT, UPDATE)
    • Clones function/procedure EXECUTE privileges
  • test_reader, test_writer, test_admin test fixture roles with graduated permissions
  • 6 new pgTAP tests (79 total): role creation, LOGIN attribute, CREATEDB attribute verification

Changed

  • Version bumped to 3.4.0

[3.3.0]

Added

  • Dynamic Data Masking via Views: Role-based masking policies that preserve original data while presenting masked views to unprivileged users
    • pgclone_create_masking_policy(schema, table, mask_json, privileged_role): creates a masked view (table_masked), revokes base table SELECT from PUBLIC, grants view access to PUBLIC, grants direct table access to the privileged role
    • pgclone_drop_masking_policy(schema, table): drops the masked view and restores base table access to PUBLIC
    • Uses the same pgclone_build_mask_expr() engine as clone-time and static masking — all 8 strategies work
    • Queries local pg_attribute for column names, applies mask expressions to matching columns
  • 7 new pgTAP tests (73 total): policy creation, view existence, masked data verification, privileged role bypass, policy drop, view cleanup

Changed

  • Version bumped to 3.3.0

[3.2.0]

Added

  • Static Data Masking on Local Tables: pgclone_mask_in_place(schema, table, mask_json) applies masking to already-cloned tables via UPDATE statements — no source connection needed
    • Uses the same mask JSON format as clone-time masking: {"email": "email", "name": "name", "ssn": "null"}
    • Reuses pgclone_build_mask_expr() for consistent masking between clone-time and post-clone paths
    • Wraps user-provided JSON into clone options format for unified parsing
    • Returns summary: OK: masked N rows in schema.table (M columns)
  • 7 new pgTAP tests (66 total): clone-then-mask workflow, verify original data removed, names/SSNs masked, row count preserved

Changed

  • Version bumped to 3.2.0

[3.1.0]

Added

  • Auto-Discovery of Sensitive Data: pgclone_discover_sensitive(conninfo, schema_name) scans the source catalog for columns matching ~40 sensitive data patterns and returns suggested mask rules as JSON
    • Pattern categories: email, name, phone, SSN/national ID, financial (salary/income), secrets/credentials (password/token/api_key), address, date of birth, credit card, IP address
    • Case-insensitive matching with LIKE-style wildcards against column names
    • Output is a JSON object grouped by table, ready to paste into the "mask" option: {"employees": {"email": "email", "salary": "random_int", "ssn": "null"}}
    • Scans regular tables and partitioned tables (relkind r and p)
  • 6 new pgTAP tests (59 total): discovery function runs, detects email/full_name/phone/salary/ssn columns

Changed

  • Version bumped to 3.1.0

[3.0.0]

Added

  • Data Masking / Anonymization: Column-level data masking applied during cloning via the "mask" JSON option
    • 8 masking strategies: email, name, phone, partial, hash, null, random_int, constant
    • email: preserves domain, masks local part (alice@example.com → a***@example.com)
    • name: replaces with XXXX
    • phone: keeps last 4 digits (+1-555-123-4567 → ***-4567)
    • partial: configurable prefix/suffix retention (Johnson → Jo***on with prefix=2, suffix=2)
    • hash: deterministic MD5 hash — preserves referential integrity across tables
    • null: replaces with NULL
    • random_int: random integer in configurable [min, max] range
    • constant: fixed replacement value
    • All strategies are NULL-safe (NULL inputs produce NULL outputs, except constant and random_int)
  • Masking is applied server-side as SQL expressions inside COPY (SELECT ...) TO STDOUT — no row-by-row processing overhead
  • Fully composable with existing columns, where, indexes, constraints, triggers options
  • MaskRule struct and pgclone_build_mask_expr() internal functions for extensible mask strategy architecture
  • 15 new pgTAP tests (53 total): email masking, name masking, null masking, hash masking, constant masking, combined masks with WHERE filter
  • test_schema.employees test fixture table with realistic sensitive data (names, emails, phones, salaries, SSNs)

Changed

  • CloneOptions struct extended with masks[] array and num_masks counter
  • pgclone_parse_options() handles "mask" JSON key with both simple string values ("email") and object values ({"type":"partial", "prefix":2, "suffix":3})
  • pgclone_copy_data() queries source catalog for column names when masks are active (without explicit column list) to apply per-column mask expressions
  • Version bumped to 3.0.0

[2.2.1]

Added

  • WHERE clause SQL injection protection: Two-layer defense against malicious input in the "where" JSON option
    • Layer 1: Keyword validation rejects DDL/DML keywords (DROP, INSERT, UPDATE, DELETE, CREATE, ALTER, TRUNCATE, etc.) and semicolons before the query is sent
    • Layer 2: Source connection runs inside BEGIN TRANSACTION READ ONLY when a WHERE clause is present — PostgreSQL itself blocks any write operations even if validation is bypassed
    • Word-boundary-aware matching avoids false positives on column names like created_at, update_count, drop_rate
  • 4 new pgTAP tests (37 total): semicolon rejection, DROP keyword rejection, INSERT keyword rejection, false-positive safety with created_at column

[2.2.0]

Changed

  • Worker Pool Architecture: Parallel cloning ("parallel": N) now uses a fixed-size worker pool instead of spawning one background worker per table
    • Exactly N workers are launched, each pulling tasks from a shared queue
    • Dynamic load balancing: faster workers automatically handle more tables
    • Resource usage reduced from O(tables) to O(N) for bgworkers and DB connections
    • No longer risk exhausting max_worker_processes on large schemas

Added

  • PgclonePoolQueue struct in shared memory for task queue management
  • pgclone_pool_worker_main() background worker entry point
  • Pool worker test in test/test_async.sh (TEST 8)
  • PGCLONE_MAX_POOL_TASKS limit (512 tables per pool operation)
  • Guard against concurrent pool operations

Removed

  • Per-table background worker launch in parallel mode (replaced by pool)
  • Per-table job slot allocation in parallel mode (pool workers share fewer slots)

[2.1.4]

Changed

  • Local loopback connections now use Unix domain sockets (from unix_socket_directories GUC) instead of TCP 127.0.0.1
  • pg_hba.conf trust entry for 127.0.0.1 is no longer required for async operations — default local all all peer is sufficient
  • Falls back to TCP 127.0.0.1 automatically if Unix sockets are unavailable

Fixed

  • Security: removed unnecessary trust authentication requirement for background worker connections

[2.1.3]

Fixed

  • Async bgworker: COPY pipeline error handling — failures now logged with PQerrorMessage, source COPY result consumed on error path to prevent connection leak
  • Async bgworker: WaitForBackgroundWorkerStartup added to all async functions — jobs no longer stuck in ‘pending’ state
  • pgclone_schema_async parallel mode: fixed hardcoded jobs[0] write that corrupted slot 0 in shared memory
  • pgclone_schema_async parallel mode: parent job now correctly transitions to COMPLETED after child workers finish

Added

  • pgclone_clear_jobs() function to free completed/cancelled job slots from shared memory
  • Async test suite (test/test_async.sh) covering pgclone_table_async, pgclone_schema_async, pgclone_progress, pgclone_jobs_view, pgclone_clear_jobs
  • CONTRIBUTING.md — development setup, code guidelines, PR process
  • SECURITY.md — vulnerability reporting, security considerations
  • Documentation restructured: docs/USAGE.md, docs/ASYNC.md, docs/TESTING.md, docs/ARCHITECTURE.md, CHANGELOG.md

[2.1.2]

Added

  • Elapsed time column in pgclone_jobs_view
  • elapsed_time field in progress detail output

[2.1.1]

Changed

  • Visual progress bar in pgclone_jobs_view replaces verbose NOTICE messages
  • Per-table/per-row NOTICE messages moved to DEBUG1 level

[2.1.0]

Added

  • pgclone_jobs_view — query async job progress as a standard PostgreSQL view
  • pgclone_progress_detail() — table-returning function for detailed progress

[2.0.1]

Added

  • pgclone_database_create() — create a new database and clone into it
  • Automatic pgclone extension installation in the target database
  • Idempotent behavior: clones into existing database if it already exists

[2.0.0]

Added

  • True multi-worker parallel cloning with "parallel": N option
  • Each table gets its own background worker
  • Parent worker monitors child workers via shared memory

Changed

  • Shared memory layout expanded for parallel job tracking

[1.2.0]

Added

  • Materialized view cloning during schema clone (with "matviews": false opt-out)
  • Exclusion constraint support (cloned alongside PK, UNIQUE, CHECK, FK)
  • Materialized view indexes and data are preserved

[1.1.0]

Added

  • Selective column cloning with "columns": [...] JSON option
  • Data filtering with "where": "..." JSON option
  • Automatic filtering of constraints/indexes referencing excluded columns

[1.0.0]

Added

  • Async clone operations via background workers (pgclone_table_async, pgclone_schema_async)
  • Job progress tracking (pgclone_progress, pgclone_jobs)
  • Job cancellation (pgclone_cancel)
  • Job resume from checkpoint (pgclone_resume)
  • Job cleanup (pgclone_clear_jobs)
  • Conflict resolution strategies: error, skip, replace, rename

[0.3.0]

Added

  • Background worker infrastructure
  • Shared memory allocation for job state
  • _PG_init with shmem hooks

[0.2.0]

Added

  • Index cloning (including expression and partial indexes)
  • Constraint cloning (PRIMARY KEY, UNIQUE, CHECK, FOREIGN KEY)
  • Trigger cloning with trigger functions
  • JSON options format for controlling indexes/constraints/triggers
  • Boolean parameter variants (pgclone_table_ex, pgclone_schema_ex)

[0.1.0]

Added

  • Initial release
  • pgclone_table() — clone a single table with or without data
  • pgclone_schema() — clone an entire schema
  • pgclone_functions() — clone functions only
  • pgclone_database() — clone all user schemas
  • COPY protocol for fast data transfer
  • pgclone_version() — version string
  • Support for PostgreSQL 14–18
  • pgTAP test suite (33 tests)
  • Docker Compose multi-version test infrastructure
  • GitHub Actions CI pipeline