Skip to content

[bugfix] Toml comment roundtrip fixes - #2768

Open
jhheider wants to merge 2 commits into
mikefarah:masterfrom
jhheider:toml-comment-roundtrip-fixes
Open

[bugfix] Toml comment roundtrip fixes#2768
jhheider wants to merge 2 commits into
mikefarah:masterfrom
jhheider:toml-comment-roundtrip-fixes

Conversation

@jhheider

@jhheider jhheider commented Jul 4, 2026

Copy link
Copy Markdown

This is generally a simple bugfix. I'm happy to open an issue for discussion, if you like, but it came up when editing some Cargo.tomls, so I worked up a quick fix.

What

Fixes two TOML comment-fidelity bugs in the round-trip / -i path, found while
migrating some sed-on-Cargo.toml edits to yq.

1. Head comment above [[array-of-tables]] was dropped

Given:

[package]
name = "x"

# the binary
[[bin]]
name = "y"

yq -p toml -o toml '.' file.toml (or any -i edit) dropped # the binary.
Head comments on ordinary [table] headers already survived. The decoder
attaches the comment correctly, but the three encoder loops that emit [[...]]
headers never wrote it (unlike the [table] path). They do now.

2. Inline comment on an array item detached

Given:

default = [
  "a",  # why a
  "b",
]

round-tripped to:

default = [
  "a",

  # why a
  "b",
]

The decoder attached every in-array comment as a head comment on the next
element. A trailing comment on the same line as an item is now detected via the
parser's byte ranges and kept as a line comment on that item. The encoder emits
array-item line comments inline and only inserts a blank line before
head-commented elements, so the existing head-comment array layout is unchanged.

Note

The first commit is two unrelated, pre-existing repairs kept separate from the
comment work (happy to split into their own PR if you'd prefer):

  • The go-toml v2.4.2 bump (Bump github.com/pelletier/go-toml/v2 from 2.4.0 to 2.4.2 #2762) renamed the single-line unterminated-string
    error, but the decode-error test still asserted the old message, so
    TestTomlScenarios was already failing on master.
  • A stray gofmt fix (a comment line in encoder_toml.go was space-indented).

Tests

Round-trip tests added for both cases in toml_test.go (top-level and nested
arrays of tables, plus the inline array-item comment). go test ./pkg/yqlib/ -run Toml passes.

Copilot AI review requested due to automatic review settings July 4, 2026 23:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes two TOML round-trip (-i) comment-fidelity issues in yq’s TOML encoder/decoder path, improving preservation of TOML comments when parsing and re-emitting documents (notably Cargo.toml-like files).

Changes:

  • Preserve head comments immediately preceding [[array-of-tables]] headers by emitting those comments before each [[...]] header during TOML encoding.
  • Preserve inline (trailing) comments on array items by detecting same-line comments in the TOML decoder (via raw byte-range inspection) and re-emitting them inline during TOML encoding.
  • Update TOML decode-error test expectation to match the go-toml v2.4.2 error message, and add round-trip regression tests for the two comment bugs.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
pkg/yqlib/decoder_toml.go Detect whether an in-array comment is inline vs on a new line, and attach inline comments to the preceding element’s LineComment.
pkg/yqlib/encoder_toml.go Emit head comments before [[array-of-tables]] headers; emit array-item LineComment inline and avoid inserting blank lines that detach trailing comments.
pkg/yqlib/toml_test.go Add round-trip regression tests for [[array-of-tables]] head comments and array-item inline comments; update decode-error assertion for go-toml v2.4.2.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

jhheider added 2 commits July 4, 2026 19:15
Two unrelated pre-existing issues in pkg/yqlib, split out from the
comment-fidelity work:

- The go-toml/v2 2.4.2 bump (mikefarah#2762) renamed the single-line
  unterminated-string parser error from `basic string not terminated by "`
  to `unterminated basic string`, but the decode-error scenario still
  asserted the old text, so TestTomlScenarios failed. Update it to match.
- gofmt: a comment line in encoder_toml.go was indented with spaces.
Two comment-fidelity bugs in the TOML round-trip:

- A head comment directly above an [[array-of-tables]] header was
  dropped. The decoder attaches it correctly, but the three encoder
  loops that emit [[...]] headers never wrote it, unlike the [table]
  path. Emit each element's head comment before its header.

- An inline (trailing) comment on an array item detached and floated
  onto its own line as a head comment on the next item, with a spurious
  blank line. createArray treated every in-array comment as a head
  comment for the next element; use the parser's byte ranges to detect a
  same-line trailing comment and attach it as a line comment on the
  current item instead. The encoder now emits array-item line comments
  inline and only inserts a blank line before head-commented elements.

Adds round-trip tests for both cases.
@jhheider
jhheider force-pushed the toml-comment-roundtrip-fixes branch from 944e671 to b781197 Compare July 4, 2026 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants