Skip to content

Expose constant-time incremental update API for production deployments #57

@sebastiannberg

Description

@sebastiannberg

pyBKT is an excellent research tool for fitting and evaluating Bayesian Knowledge Tracing models on full historical datasets. However, its current API is designed around batch processing and replays the complete observation sequence every time predictions are made. This makes it difficult to use in real-time tutoring systems or other applications where new data points arrive continuously and model updates must happen immediately.

In our deployment, for example, each new quiz answer triggers a call to update a student’s mastery estimate. Because pyBKT recomputes posteriors from scratch for every observation, the cost grows linearly with the number of past answers (O(n)). This overhead is acceptable in offline evaluation but becomes a bottleneck in production environments that expect sub-100 ms latency per interaction.

Many adaptive learning systems and sequential decision engines require incremental updates: after observing one more event, they need to compute the new latent state in constant time, not by replaying the entire history. While the BKT equations fully support this kind of update, pyBKT doesn’t currently expose a convenient API for it. As a result, developers who use the library for online inference end up reimplementing the update step outside the library, fragmenting codebases and increasing maintenance cost.

An incremental interface would allow a single, consistent parameterization to serve both offline research and live applications.

Proposed Addition

Introduce a lightweight helper that performs a one-step Bayesian update given:

  • the learner’s current mastery probability,
  • the outcome of the latest observation (correct/incorrect),
  • and the fitted parameters (p_init, p_trans, p_slip, p_guess) for the relevant skill.

Example usage:

new_mastery = model.update_single(
    prior=0.42,
    is_correct=True,
    skill="fractions"
)

Internally, this would:

  1. Apply the standard posterior update based on the observation.
  2. Perform the transition step to obtain the next prior.
  3. Return the updated mastery probability (and optionally metadata like timestamp or count).

This operation is O(1) and can be composed seamlessly with existing model training and evaluation workflows.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions