Skip to content

test: fix flaky tests — 56-67% pass rate on Hebrew layout combos #37

@chernistry

Description

@chernistry

Problem

The test suite has unacceptably low pass rates for certain layout combinations:

Layout Pair Pass Rate Status
English ↔ Russian ~95% OK
English ↔ Hebrew ~56% Failing
Hebrew ↔ Russian ~67% Failing

A 56% pass rate means the correction engine is wrong nearly half the time for Hebrew users. This blocks adoption in Israel entirely.

Root Cause Analysis

The likely issues are:

1. Hebrew keyboard has multiple common layouts

Hebrew has at least two standard layouts (SI-1452 and the older typewriter layout). The mapping tables may only cover one.

Check: RightLayout/Sources/Engine/LayoutMappings/ — verify Hebrew mapping completeness.

2. Right-to-left text handling

Hebrew is RTL. The correction engine may be reversing character order incorrectly when building the replacement string.

Check: RightLayout/Sources/Engine/CorrectionEngine.swift — look for string reversal or character iteration that assumes LTR.

3. Diacritics and Nikud

Some Hebrew characters have diacritical marks (nikud). If the engine strips or mishandles combining characters, corrections will be wrong.

Test Cases to Investigate

File: tests/test_cases.json

Look for Hebrew test cases that fail. Example patterns:

  • Characters with final forms (ך ,ם ,ן ,ף ,ץ) — these map to different keys
  • Punctuation that differs between Hebrew and QWERTY
  • Shift-state characters

Steps to Debug

  1. Run Hebrew-specific tests:
    swift test --filter Hebrew
  2. Identify which specific character mappings fail
  3. Cross-reference with physical Hebrew keyboard layout
  4. Fix mapping table or RTL handling logic
  5. Add regression tests for each fixed case

Relevant Files

  • tests/test_cases.json — test case definitions
  • RightLayout/Tests/ — test implementations
  • RightLayout/Sources/Engine/LayoutMappings/ — character mapping tables
  • RightLayout/Sources/Engine/CorrectionEngine.swift — core correction logic

Acceptance Criteria

  • Hebrew ↔ English pass rate reaches 90%+
  • Hebrew ↔ Russian pass rate reaches 85%+
  • Final-form Hebrew characters (ך ם ן ף ץ) map correctly
  • RTL text direction is handled properly in corrections
  • No regression in English ↔ Russian tests

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghacktoberfestHacktoberfest eligiblehelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions