Skip to content

Fixes data-latex content for Table Rows - #1528

Open
zorkow wants to merge 12 commits into
developfrom
fix/table_latex
Open

Fixes data-latex content for Table Rows#1528
zorkow wants to merge 12 commits into
developfrom
fix/table_latex

Conversation

@zorkow

@zorkow zorkow commented Jul 12, 2026

Copy link
Copy Markdown
Member

Rows in Tables did not get a meaningful data-latex attribute. E.g., for \begin{array}{c}a\end{array} we got data-latex="{c}".
This PR fixes this by explicitly collecting the LaTeX elements and then adding them to the final node, combined with &.

@zorkow
zorkow requested a review from dpvc July 12, 2026 08:07
@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.98%. Comparing base (a8088bb) to head (c618121).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1528      +/-   ##
===========================================
+ Coverage    86.93%   86.98%   +0.05%     
===========================================
  Files          388      388              
  Lines        87571    87923     +352     
  Branches      3290     5028    +1738     
===========================================
+ Hits         76129    76481     +352     
+ Misses       11442    11421      -21     
- Partials         0       21      +21     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dpvc dpvc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a big improvement, but there still seem to be some issues. Those are probably actually problems with the attributes of the child nodes in many cases. Perhaps they can be improved? (I haven't looked into the problem, but it may be due to modifying the parser.string value for substitutions.)

Comment thread testsuite/tests/input/tex/__snapshots__/Cases.test.ts.snap Outdated
Comment thread testsuite/tests/input/tex/__snapshots__/Cases.test.ts.snap
Comment thread testsuite/tests/input/tex/__snapshots__/Colortbl.test.ts.snap Outdated
Comment thread testsuite/tests/input/tex/__snapshots__/Colortbl.test.ts.snap Outdated
Comment thread testsuite/tests/input/tex/__snapshots__/Colortbl.test.ts.snap Outdated
Comment thread testsuite/tests/input/tex/__snapshots__/Newcommand.test.ts.snap Outdated
Comment thread testsuite/tests/input/tex/__snapshots__/Tag.test.ts.snap Outdated
Comment thread testsuite/tests/input/tex/__snapshots__/Amscd.test.ts.snap
@zorkow

zorkow commented Jul 14, 2026

Copy link
Copy Markdown
Member Author

Thanks for looking at these tests in more detail. I have to admit I mainly focussed on the expressions in Base and Ams. I believe for tag elements (coming from \label commands) we probably need to skip them when accumulating the row.
I am also wondering if it would make sense to have latex labels on the mtd elements.

@dpvc dpvc added this to the v4.2 milestone Jul 15, 2026
@zorkow

zorkow commented Jul 30, 2026

Copy link
Copy Markdown
Member Author

Summary (cleaned up and greatly enhanced by co-pilot)

Fixes several cases where the data-latex attribute on mtable (and related nodes)
did not reflect the actual LaTeX source, plus a couple of related source-tracking bugs uncovered along the way.

Problems

  1. Column templates garble data-latex. Templates like >{...} rewrite the
    parser's string entry-by-entry as it goes, so the array's data-latex ends up built
    from a leftover fragment instead of the real source.
  2. Nested arrays lose their source. When a table is nested inside another
    environment (e.g. a tagged equation, or another array), data-latex was set from just the \end{name} argument, producing a bogus value like {array}.
  3. Non-rendering macros clobber the preceding node's source. \label, \tag,
    \notag, and colortbl's \cellcolor/\rowcolor/\columncolor produce no Mml node
    of their own, but the generic source-tracking in TexParser.updateResult was still
    attaching their text to whatever node came before them.
  4. \ref/\eqref recorded the wrong source. The generic tracking captured the
    raw \ref{...} call instead of the resolved tag text actually used to build the
    reference node.
  5. Tagged/labeled rows and multline rows had no correct row-level data-latex.
  6. \newenvironment-defined environments could invalidate cached source positions in an enclosing array when their body is spliced into the parser string mid-parse.

Fixes

  • Raw source capture for arrays (ArrayItem.captureLatex / setTableLatex,
    FilterUtil.fixArrayLatex): at \begin{name}, grab the true
    \begin{name}...\end{name} text up front (or stash a start position if the end isn't
    in the string yet, e.g. for \newenvironment). At table-close, recover this text and
    store it as rawLatex/rawLatexName on the mtable. A new postprocessing filter
    then replaces data-latex on the table only when the existing value looks broken
    (matches the {name} or dangling-\end patterns), so already-correct values are left untouched. Falls back to reconstructing from row-level data-latex if nothing else
    is available.
  • envSplice generation counter: since expanding a \newenvironment splices its
    body into parser.string (invalidating any previously-stashed absolute positions), a
    global counter is bumped on every such splice and snapshotted alongside a stashed
    start position, so stale positions are detected and skipped rather than misused.
  • Blanking \end{name} with whitespace: When a user-defined environment
    closes, BeginEnv takes everything already consumed and appends edef to it,
    continuing to parse from the same position. Leaving that literal \end{name}
    text in place leads to incorrect data-latex. Simply deleting it messes up
    subsequent parsing. Replacing \end{name} with an equal-length run of
    whitespace and removing it later in setTableLatex fixes both problems.
  • TexParser.skipLatex(): a one-shot flag that suppresses the next
    updateResult() source attachment, used by \label, \tag(the tagging path),
    \notag, and colortbl's color macros so they no longer overwrite the preceding node's data-latex.
  • Row-level LaTeX tracking (rowLatex on ArrayItem/MultlineItem, colorLatex on ColorArrayItem): cell/row source is accumulated as rows are built, including
    folding in colortbl macro text and label/tag content, and used to set
    data-latex/data-latex-item on mtr/mlabeledtr rows (also used in Tags.ts for
    the label row).
  • \ref/\eqref fix: builds the reference node's content via an explicit
    internal TexParser call and sets its data-latex/data-latex-item directly from
    the resolved tag text, then calls skipLatex() so generic tracking doesn't overwrite
    it with the raw macro call.
  • NodeUtil.getChildrenLatex: shared helper (replacing the old inline logic in
    ArrayItem.cellLatex) that prefers a child's rawLatex property over its
    data-latex attribute when reconstructing source from content nodes.

New setProperty keys

Property Set where Purpose
rawLatex ArrayItem.captureLatex(), and copied onto the mtable/wrapping mstyle node in setTableLatex() / EndTable() Caches the raw, unmodified LaTeX source of the array/table environment (\begin{name}...\end{name}), captured before column-template substitutions or nested-environment parsing can corrupt the parser's string.
rawLatexName Alongside rawLatex everywhere it's set The environment name (array, matrix, etc.), needed to detect "broken" data-latex values and to rebuild \begin{name}...\end{name} wrappers.
rawLatexStart captureLatex(), only when the matching \end{name} can't be found up front (e.g. \newenvironment-defined environments) Parser position right after \begin{name}, so the source can be sliced out later once the environment has actually finished parsing.
rawLatexGen captureLatex(), alongside rawLatexStart Snapshot of the envSplice generation counter, used to detect whether the stashed rawLatexStart position has been invalidated before trusting it.

How they fit together

  1. On \begin{name}, captureLatex() grabs the raw source immediately, or stashes a
    start position + generation if the end isn't available yet.
  2. On table close, setTableLatex() uses rawLatex if captured, else recovers it
    from rawLatexStart/rawLatexGen (if the generation still matches), else
    reconstructs it from the rows' own data-latex.
  3. FilterUtil.fixArrayLatex (a new postprocessing filter in the TeX.ts pipeline)
    walks all mtable nodes and replaces data-latex with the cached rawLatex only
    when the existing value looks broken/missing (FilterUtil.isBrokenArrayLatex).

@zorkow

zorkow commented Jul 30, 2026

Copy link
Copy Markdown
Member Author

PS:

As discussed earlier, what is still missing:

  • Dealing with the prefix rows in the cases package
  • Also there is no solution for table expressions generated with amscd

The findEnvEnd parsing method, to find \end{name} if it exists, is very similar to a number of other parsing functions we have (in fact I started by simplifying Entry) that we already have. I wonder if there is some way of refactoring them into a single utility.

@zorkow
zorkow requested a review from dpvc August 7, 2026 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants