Parser panics on CREATE TABLE / INSERT in the Trino dialect (grammar refs to keywords Trino removed)

[dkxmercury]

What happens

Parsing everyday statements in the Trino dialect panics instead of producing a parse tree or a clean error.

let dialect = kind_to_dialect(&DialectKind::Trino, None).unwrap();
let lexer = Lexer::from(&dialect);
let parser = Parser::from(&dialect);
let tokens = lexer.lex(&tables, "create table".to_string());
parser.parse(&tables, &tokens.0).unwrap(); // panics

• create table panics with Grammar refers to 'TRANSIENT' which was not found in the dialect (grammar.rs:133)
• insert into t values (1) panics with Grammar refers to 'OVERWRITE' which was not found in the dialect (grammar.rs:145)
• alter table t ... reaches the same panic for SEQUENCE

I hit these while fuzzing the parser across dialects, and confirmed each by lexing and parsing the string above through the public pipeline.

Root cause

trino.rs builds Trino from ansi::raw_dialect(), then clears the keyword sets and repopulates them from TRINO_UNRESERVED_KEYWORDS / TRINO_RESERVED_KEYWORDS:

trino_dialect.sets_mut("unreserved_keywords").clear();
trino_dialect.update_keywords_set_from_multiline_string("unreserved_keywords", TRINO_UNRESERVED_KEYWORDS);

The inherited ANSI grammar still contains Ref::keyword("TRANSIENT"), Ref::keyword("OVERWRITE") and Ref::keyword("SEQUENCE") (from TemporaryTransientGrammar, InsertStatementSegment, and the alter/sequence grammar). Dialect::expand only creates a library entry for a keyword that is present in a keyword set, so these references resolve to nothing and Dialect::ref takes its None arm and panics. Trino does not override those statement segments, so the ANSI grammar with these refs is used as is.

The tricky part, and why I did not just send a PR

The obvious quick fix, adding TRANSIENT, OVERWRITE, SEQUENCE to Trino's keyword sets, would be wrong. Trino does not support transient tables, INSERT OVERWRITE, or sequences, so registering those keywords would make the parser accept syntax that is not valid Trino.

I can see a few real fixes, and which one you want is a dialect design call, so I would rather ask than guess:

1. Override CreateTableStatementSegment, InsertStatementSegment and the alter/sequence grammar in Trino to drop the unsupported keyword references, matching what Trino actually accepts (and SQLFluff's Trino dialect).
2. Make Dialect::ref degrade gracefully when a grammar references a keyword that is not in the dialect's sets, so an optional keyword simply does not match rather than crashing. This would also guard any other dialect against the same class of panic, though it changes core behavior and might hide genuine dialect misconfigurations.

Happy to put up a PR for whichever direction you prefer.

[benfdking]

I compared this with sqruff's tracked SQLFluff revision. The current .sqlfluff-sha is 59f80e35 (2023-07-02), which predates SQLFluff's initial Trino dialect, so there is no Trino change to port from the pinned revision itself. Sqruff's Trino implementation was added ahead of that pin and has since missed several later upstream Trino changes.

The clean fix here appears to be catching the Trino dialect up through these SQLFluff commits:

• c6b39aae — replace TemporaryTransientGrammar with Nothing, removing the inherited TRANSIENT reference.
• d36f2f06 — add an explicit Trino StatementSegment and Trino-specific CREATE TABLE/column/transaction grammars, excluding unsupported inherited ANSI statement branches such as sequences.
• 74aaf399 — add the Trino-specific INSERT grammar without OVERWRITE, along with COMMIT, ROLLBACK, and SET SESSION support.
• 07db73d1 — add the Trino-specific ALTER TABLE grammar, avoiding unsupported inherited options such as MODIFY.

The solution here is really implementing these.