Feature Request: Enumerator Response Pattern Checks (Enumerator Statistics)

[iabaako]

Summary

Add a suite of response pattern checks to the Enumerator Statistics tab. These checks detect whether individual enumerators are collecting responses that look systematically different from the rest of the team — a signal of data quality problems such as rounding, anchoring, or leading respondents.

Three checks are covered in this issue:

1. Digit Preference - are numeric responses being rounded to convenient values?
2. Range Compression - are numeric responses clustered in a suspiciously narrow range?
3. Categorical Response Patterns - are response option distributions anomalous for any enumerator?

Each check is a sub-tab within the Enumerator Statistics tab.

Background

These checks address common enumerator-driven data quality issues that are difficult to spot by looking at individual records but become visible when comparing across enumerators:

• Digit preference: Enumerators rounding answers to values ending in 0 or 5 (e.g., recording ages as 30, 35, 40 instead of 28, 33, 41) introduces systematic error. This is a well-documented problem in survey research.
• Range compression: An enumerator whose responses fall within a much narrower range than peers may be anchoring around a typical value rather than recording real responses.
• Categorical response patterns: An enumerator who records one response option far more than the team average may be using a default answer, leading respondents, or skipping the question mentally.

Detecting these early allows field managers to follow up with specific enumerators before data collection is complete.

Location in the app

Enumerator Statistics tab → three sub-tabs:

• Digit Preference
• Range Compression
• Categorical Response Patterns

Shared: Inputs (configuration panel)

All three checks share the same variable selection and settings expander pattern.

1. Variable selection

• Use a Data Editor component, similar to the Descriptive Statistics tab
• Show variable name and type (integer, continuous, categorical)
• Include string variables as categorical if they contain enums and are not free-text fields
• Exclude IDs, phone numbers, and coded values
• Users can manually change the column type
• Default: no variables selected

2. Minimum sample size settings (configurable in a tab-level settings expander)

Responses	Behaviour
< 15	Do not show results. Display: "Not enough responses to run this check."
15–29	Show results with a warning: "Low sample size — treat results with caution."
30+	Show results normally

The thresholds of 15 and 30 are configurable in the tab-level settings expander.

Shared: Enumerator list (left panel)

Applies to all three sub-tabs.

• List all enumerators with name, interview count, and a score badge
• Score badge labels and colours:
  • N/A (grey) — insufficient data
  • Low (green) — score below threshold
  • Moderate (amber) — score within 15 points below threshold
  • High (red) — score at or above threshold
• Clicking an enumerator loads their detail view on the right

Check 1: Digit Preference

Detects whether an enumerator's numeric responses end in 0 or 5 more often than expected.

Applies to: integer and continuous variables only

Inputs

Flag threshold

• Slider from 10 to 90
• Default: 50
• Label: "Flag enumerators with a digit preference score above [value]"
• Description updates with slider, e.g. "A score above 50 means responses ending in 0 or 5 are twice as common as expected by chance."

Output (detail panel)

Terminal digit distribution chart

• Bar chart: percentage of responses ending in each digit (0–9)
• Two bars per digit: enumerator (dark) and project average (light grey), side by side
• Digits 0 and 5 highlighted in red when they spike above the project average
• Y-axis: percentage; X-axis: terminal digit (0–9)

Digit preference score

• Single score from 0–100 shown alongside the chart header
• See scoring method below

Summary stat tiles

• % responses ending in 0 or 5 — enumerator vs. project average
• Number of responses included

Verdict box

• Flagged: "[Name]'s responses end in 0 or 5 significantly more often than the rest of the team. Consider reviewing their interviews."
• Not flagged: "[Name]'s digit distribution is consistent with the rest of the team."
• Low sample warning shown inside or just above the verdict box if applicable

Scoring method

1. For each enumerator, calculate the percentage of responses ending in 0 or 5.
2. Calculate the same percentage across all enumerators (project average).
3. Score = min(100, max(0, ((enum_pct - project_pct) / project_pct) * 100))
   • 0 = no excess rounding relative to the project
   • 100 = enumerator rounds at least twice as often as the project average

This is an adjusted scale based on the Whipple Index, rescaled to 0–100 for readability.

Check 2: Range Compression

Detects whether an enumerator's numeric responses are clustered in an unusually narrow range compared to the rest of the team.

Applies to: continuous variables only

Inputs

Flag threshold

• Slider controlling the SD ratio below which an enumerator is flagged
• Default: 0.5 (enumerator's standard deviation is less than half the project standard deviation)
• Label: "Flag enumerators whose response spread is below [value] of the project average"

Output (detail panel)

Distribution histogram

• Side-by-side bars per bucket: enumerator (dark) and project average (light grey)
• Buckets auto-generated based on variable range
• Bars highlighted in amber when the enumerator's distribution is compressed relative to the project

Summary stat tiles

• Mean - enumerator vs. project average
• Standard deviation - enumerator vs. project average
• SD ratio - enumerator SD divided by project SD (flagged if below threshold)

Verdict box

• Flagged (compression): "[Name]'s responses are clustered in a narrow range compared to the rest of the team — this may suggest anchoring around a typical value rather than recording real responses."
• Flagged (shift): "[Name]'s responses are systematically higher/lower than the project average. This may reflect a real sampling difference or interviewer influence."
• Not flagged: "[Name]'s response distribution is consistent with the rest of the team."

Scoring method

Two signals are calculated independently:

Compression score (0–100)

• Score = max(0, (1 - SD_ratio) * 100) where SD_ratio = enum_sd / project_sd
• A score of 0 means the enumerator's spread matches the project. A score of 100 means the enumerator has no variation at all.

Shift score (0–100)

• Score = min(100, abs((enum_mean - project_mean) / project_sd) * 33)
• Scaled so that a 3 standard deviation shift produces a score of 100.

The higher of the two scores is used as the displayed score and for flagging.

---

Check 3: Categorical Response Patterns

Detects whether an enumerator's distribution across response options deviates significantly from the team average.

Applies to: categorical and string enum variables only

Inputs

Flag threshold

• Slider from 10 to 90 controlling sensitivity
• Default: 50
• Label: "Flag enumerators with a response pattern score above [value]"

Output (detail panel)

Results are shown per question, not as a single aggregate view.

Per-question response distribution

• Horizontal bar chart per response option
• Two bars: enumerator percentage (dark) and project average (light grey)
• Options with a large deviation from the project average highlighted in red
• An ANOMALY label shown next to the question title if flagged

Verdict box (per question)

• Yes-bias detected: "Yes-responses are much higher than the team average — possible acquiescence bias or leading questions."
• Option dominance: "One response option dominates this enumerator's responses. Consider reviewing whether the question is being asked correctly."
• Not flagged: "Response distribution is consistent with the rest of the team."

Scoring method

Per question, per enumerator:

1. Calculate the absolute difference between the enumerator's percentage and the project average for each option.
2. Take the maximum absolute difference across all options as the question-level deviation.
3. Scale to 0–100: Score = min(100, max_deviation * 2)
   • A 50 percentage point deviation in any option produces a score of 100.

The enumerator-level score shown in the left panel is the average score across all flagged questions.

Shared: Edge cases

• All enumerators below minimum sample size — show a page-level message instead of an empty list
• No variables selected — show an info message prompting the user to select at least one variable; disable the results panel
• No variation in responses (all enumerators give the same answer) — note that the check was not calculated for that variable
• Single enumerator projects — all three checks require a project average; show a message that at least 2 enumerators are needed

Out of scope for this issue

• Heaping detection (clustering around non-round values)
• Export / download of results
• Per-question drill-down for digit preference and range compression