id: how-it-works title: How it works

How pg_describe works

Stopping the pipeline

PostgreSQL’s stages are ordinary C functions; nothing requires calling all of them.

List  *raw   = pg_parse_query(sql);
Query *query = parse_analyze_varparams(rawstmt, sql, &param_types, &num_params, NULL);
/* rewrite, plan, execute: never called */

pg_parse_query is grammar only — no catalog access, no name resolution. It parses SELECT * FROM nonexistent without complaint.

parse_analyze_varparams resolves names against the catalog, types every expression, and infers a type for each $n. The varparams variant takes an empty parameter array and grows one, filling each type from the context the parameter appears in; the fixedparams sibling is used when a client declares its types. exec_parse_message calls the former to serve a Parse message with an empty type list, so a client sending Parse/Describe runs this same code.

Reading the Query

Wire protocol Query field
ParameterDescription.oid[] param_types[0..num_params)
RowDescription.name, .typeOID targetList → TargetEntry->resname, exprType(tle->expr)
RowDescription.tableOID, .columnAttrNumber IsA(tle->expr, Var) → rt_fetch(var->varno, query->rtable)
catalog query for attnotnull SearchSysCache2(ATTNUM, ...)

The syscache lookup replaces a network round trip with a hash lookup over a relation analysis has already locked.

commandType Columns
CMD_SELECT targetList
CMD_INSERT / CMD_UPDATE / CMD_DELETE returningList, empty without RETURNING
CMD_UTILITY none; parked un-analysed in utilityStmt

Two filters apply. resjunk entries are skipped — the sort key ORDER BY adds for an unselected column is in the list but not the result, and RowDescription omits it too. Provenance requires a bare Var with varlevelsup == 0 and varattno > 0; expressions, correlated references to an enclosing query, whole-row references and system columns get a type but no source.

The join tree walk

static void
find_nullable(Node *jtnode, Bitmapset **nullable)
{
    if (IsA(jtnode, FromExpr))
        foreach(lc, ((FromExpr *) jtnode)->fromlist)
            find_nullable((Node *) lfirst(lc), nullable);

    else if (IsA(jtnode, JoinExpr))
    {
        JoinExpr *j = (JoinExpr *) jtnode;

        find_nullable(j->larg, nullable);
        find_nullable(j->rarg, nullable);

        if (j->jointype == JOIN_LEFT || j->jointype == JOIN_FULL)
            collect_rtindexes(j->rarg, nullable);

        if (j->jointype == JOIN_RIGHT || j->jointype == JOIN_FULL)
            collect_rtindexes(j->larg, nullable);
    }
}

collect_rtindexes gathers every base relation beneath a node. Combined with recursing into both arms before applying this node’s rule, that handles nesting:

a LEFT JOIN (b JOIN c)       b and c. If the outer join finds no match the whole
                             right subtree is null-extended at once, so the
                             inner join between them is irrelevant.
(a LEFT JOIN b) JOIN c       b only.
a LEFT JOIN (b LEFT JOIN c)  b and c.

All three are in the regression suite; an implementation inspecting only each join’s immediate arms passes the second and fails the first.

GROUP BY ROLLUP, CUBE and GROUPING SETS null-extend grouping columns in super-aggregate rows. When query->groupingSets is non-empty, no provenance-bearing column is reported non-null.

Not modelled

  • Set operations. UNION branches are subquery RTEs, so target-list Vars do not resolve to a base relation and no flags are reported.
  • Subqueries and CTEs. Same.
  • CHECK constraints, partial indexes, WHERE x IS NOT NULL. Predicates that guarantee non-nullness are not consulted.

All report nullable or unknown, so all fail safe.

Permissions

Parse analysis records privilege requirements in Query->rteperminfos and leaves enforcement to ExecutorStart, which is never reached. Skipping the check would expose every column name, type and relationship in the database to any caller.

userid = OidIsValid(perminfo->checkAsUser) ? perminfo->checkAsUser : GetUserId();

remaining = perminfo->requiredPerms &
    ~pg_class_aclmask(relid, userid, perminfo->requiredPerms, ACLMASK_ALL);

Four details:

  • checkAsUser is set when the rights to check are not the current role’s — a view runs with its owner’s privileges.
  • aclmask, not aclcheck: only a missing SELECT can be rescued by column-level grants, so the specific missing bits matter.
  • Column-level grants are checked when relation-level rights are absent.
  • selectedCols is offset by FirstLowInvalidHeapAttributeNumber (-7), since system columns have negative attnums and Bitmapset members cannot. InvalidAttrNumber after the shift means whole-row, which no column grant satisfies — hence SELECT * failing where SELECT email succeeds.

Failure goes through aclcheck_error, so message, SQLSTATE and object naming match what the query itself would have raised.

Error positions

A parse error’s cursor position is measured against the string being parsed. Unchanged, the client draws the caret against the outer statement:

ERROR:  syntax error at or near "WHERE"
LINE 1: SELECT * FROM pg_describe('SELECT FROM WHERE');
                    ^

An error context callback runs while the error is being built and can still edit its fields, so the position is re-reported as internal, against the inner query:

static void
describe_error_cb(void *arg)
{
    int pos = geterrposition();

    if (pos > 0)
    {
        errposition(0);
        internalerrposition(pos);
        internalerrquery((const char *) arg);
    }
}

ERROR:  syntax error at or near "WHERE"
LINE 1: SELECT FROM WHERE
                    ^
QUERY:  SELECT FROM WHERE

Errors abort the transaction

Analysis errors are not caught. pg-describe-gen therefore describes each query independently and collects failures rather than stopping at the first.

Catching them in the extension needs BeginInternalSubTransaction with the matching RollbackAndReleaseCurrentSubTransaction and saved CurrentMemoryContext / CurrentResourceOwner. A bare PG_TRY/PG_CATCH returns control without undoing locks, buffer pins or aborted transaction state.