[bugfix] Toml comment roundtrip fixes - #2768
Open
jhheider wants to merge 2 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes two TOML round-trip (-i) comment-fidelity issues in yq’s TOML encoder/decoder path, improving preservation of TOML comments when parsing and re-emitting documents (notably Cargo.toml-like files).
Changes:
- Preserve head comments immediately preceding
[[array-of-tables]]headers by emitting those comments before each[[...]]header during TOML encoding. - Preserve inline (trailing) comments on array items by detecting same-line comments in the TOML decoder (via raw byte-range inspection) and re-emitting them inline during TOML encoding.
- Update TOML decode-error test expectation to match the go-toml v2.4.2 error message, and add round-trip regression tests for the two comment bugs.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| pkg/yqlib/decoder_toml.go | Detect whether an in-array comment is inline vs on a new line, and attach inline comments to the preceding element’s LineComment. |
| pkg/yqlib/encoder_toml.go | Emit head comments before [[array-of-tables]] headers; emit array-item LineComment inline and avoid inserting blank lines that detach trailing comments. |
| pkg/yqlib/toml_test.go | Add round-trip regression tests for [[array-of-tables]] head comments and array-item inline comments; update decode-error assertion for go-toml v2.4.2. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Two unrelated pre-existing issues in pkg/yqlib, split out from the comment-fidelity work: - The go-toml/v2 2.4.2 bump (mikefarah#2762) renamed the single-line unterminated-string parser error from `basic string not terminated by "` to `unterminated basic string`, but the decode-error scenario still asserted the old text, so TestTomlScenarios failed. Update it to match. - gofmt: a comment line in encoder_toml.go was indented with spaces.
Two comment-fidelity bugs in the TOML round-trip: - A head comment directly above an [[array-of-tables]] header was dropped. The decoder attaches it correctly, but the three encoder loops that emit [[...]] headers never wrote it, unlike the [table] path. Emit each element's head comment before its header. - An inline (trailing) comment on an array item detached and floated onto its own line as a head comment on the next item, with a spurious blank line. createArray treated every in-array comment as a head comment for the next element; use the parser's byte ranges to detect a same-line trailing comment and attach it as a line comment on the current item instead. The encoder now emits array-item line comments inline and only inserts a blank line before head-commented elements. Adds round-trip tests for both cases.
jhheider
force-pushed
the
toml-comment-roundtrip-fixes
branch
from
July 4, 2026 23:17
944e671 to
b781197
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is generally a simple bugfix. I'm happy to open an issue for discussion, if you like, but it came up when editing some Cargo.tomls, so I worked up a quick fix.
What
Fixes two TOML comment-fidelity bugs in the round-trip /
-ipath, found whilemigrating some
sed-on-Cargo.tomledits to yq.1. Head comment above
[[array-of-tables]]was droppedGiven:
yq -p toml -o toml '.' file.toml(or any-iedit) dropped# the binary.Head comments on ordinary
[table]headers already survived. The decoderattaches the comment correctly, but the three encoder loops that emit
[[...]]headers never wrote it (unlike the
[table]path). They do now.2. Inline comment on an array item detached
Given:
round-tripped to:
The decoder attached every in-array comment as a head comment on the next
element. A trailing comment on the same line as an item is now detected via the
parser's byte ranges and kept as a line comment on that item. The encoder emits
array-item line comments inline and only inserts a blank line before
head-commented elements, so the existing head-comment array layout is unchanged.
Note
The first commit is two unrelated, pre-existing repairs kept separate from the
comment work (happy to split into their own PR if you'd prefer):
error, but the decode-error test still asserted the old message, so
TestTomlScenarioswas already failing onmaster.gofmtfix (a comment line inencoder_toml.gowas space-indented).Tests
Round-trip tests added for both cases in
toml_test.go(top-level and nestedarrays of tables, plus the inline array-item comment).
go test ./pkg/yqlib/ -run Tomlpasses.