Add optional margin line number removal for PDF layout

[fangwenxie]

Summary

• add a remove_margin_line_numbers translation option exposed through the CLI/config path
• remove detected margin-only line number columns before paragraph tracking to avoid misplaced translated paragraphs
• keep the local reference-section skip and vertical text layout handling in this patch set

Testing

• ruff check on the touched BabelDOC files
• python compile check on the touched BabelDOC files
• verified a single FDA guidance page with --remove-margin-line-numbers; debug tracking no longer includes the margin line numbers

---

Summary by cubic

Add an optional flow to remove margin line numbers in PDFs to improve paragraph tracking and translation layout. Also skips reference sections and adds vertical text support in the local translator pipeline.

• New Features

  • CLI flag --remove-margin-line-numbers (config: remove_margin_line_numbers) to strip margin-only line number columns before paragraph tracking; auto-disables alternating line-number merge when on.
  • Skip translation of references/bibliography by layout labels or detected section header (in back-half pages), for both IL and llm_only translators.
  • Vertical text support: extract plain text from rotated paragraphs and render vertically; propagate vertical flag with updated boxes and layout.

• Bug Fixes

  • Prevent misplaced paragraphs by removing margin digits and spaces at page sides using detected columns.
  • Improve paragraph metadata by using a character with xobj_id when present (more accurate vertical and xobj_id).
  • Ensure vertical paragraphs and reference sections are handled correctly in pre-translation and typesetting.

^{Written for commit 731ed6d. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 6 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="babeldoc/format/pdf/document_il/midend/paragraph_finder.py">

<violation number="1" location="babeldoc/format/pdf/document_il/midend/paragraph_finder.py:301">
P1: Margin line number removal config flag unconditionally disables the alternating line number merge fallback, even when no margin columns were actually detected. The `merge_alternating_line_number_paragraphs` guard uses `not remove_margin_line_numbers` (the config flag) rather than whether removal succeeded. When `remove_margin_line_numbers=True` but `_find_margin_line_number_columns` returns no columns, `remove_margin_line_numbers()` returns early without modifying anything, yet the merge fallback is still skipped. Documents with interleaved line numbers that don't form clear margin columns will keep fragmented body paragraphs.</violation>
</file>

<file name="babeldoc/format/pdf/document_il/midend/typesetting.py">

<violation number="1" location="babeldoc/format/pdf/document_il/midend/typesetting.py:1527">
P1: Vertical column advance uses modal width without a maximum safety bound, which can cause inter-column overlap for mixed-width content.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: Margin line number removal config flag unconditionally disables the alternating line number merge fallback, even when no margin columns were actually detected. The merge_alternating_line_number_paragraphs guard uses not remove_margin_line_numbers (the config flag) rather than whether removal succeeded. When remove_margin_line_numbers=True but _find_margin_line_number_columns returns no columns, remove_margin_line_numbers() returns early without modifying anything, yet the merge fallback is still skipped. Documents with interleaved line numbers that don't form clear margin columns will keep fragmented body paragraphs.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At babeldoc/format/pdf/document_il/midend/paragraph_finder.py, line 301:

<comment>Margin line number removal config flag unconditionally disables the alternating line number merge fallback, even when no margin columns were actually detected. The `merge_alternating_line_number_paragraphs` guard uses `not remove_margin_line_numbers` (the config flag) rather than whether removal succeeded. When `remove_margin_line_numbers=True` but `_find_margin_line_number_columns` returns no columns, `remove_margin_line_numbers()` returns early without modifying anything, yet the merge fallback is still skipped. Documents with interleaved line numbers that don't form clear margin columns will keep fragmented body paragraphs.</comment>

<file context>
@@ -286,8 +289,19 @@ def process_page(self, page: Page):
+
         # 新增后处理：合并带行号交替的正文段落（a 正文、b 行号、c 正文 -> 合并 a 与 c，保留 b）
-        if getattr(self.translation_config, "merge_alternating_line_numbers", True):
+        if (
+            getattr(self.translation_config, "merge_alternating_line_numbers", True)
+            and not remove_margin_line_numbers
</file context>

[cubic-dev-ai]

P1: Vertical column advance uses modal width without a maximum safety bound, which can cause inter-column overlap for mixed-width content.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At babeldoc/format/pdf/document_il/midend/typesetting.py, line 1527:

<comment>Vertical column advance uses modal width without a maximum safety bound, which can cause inter-column overlap for mixed-width content.</comment>

<file context>
@@ -1455,6 +1490,60 @@ def _layout_typesetting_units(
+                if not current_line_widths:
+                    return [], False
+                current_y = box.y
+                current_x -= max(
+                    statistics.mode(current_line_widths) * line_skip,
+                    line_width,
</file context>