-
Notifications
You must be signed in to change notification settings - Fork 79
Description
pyBKT is an excellent research tool for fitting and evaluating Bayesian Knowledge Tracing models on full historical datasets. However, its current API is designed around batch processing and replays the complete observation sequence every time predictions are made. This makes it difficult to use in real-time tutoring systems or other applications where new data points arrive continuously and model updates must happen immediately.
In our deployment, for example, each new quiz answer triggers a call to update a student’s mastery estimate. Because pyBKT recomputes posteriors from scratch for every observation, the cost grows linearly with the number of past answers (O(n)). This overhead is acceptable in offline evaluation but becomes a bottleneck in production environments that expect sub-100 ms latency per interaction.
Many adaptive learning systems and sequential decision engines require incremental updates: after observing one more event, they need to compute the new latent state in constant time, not by replaying the entire history. While the BKT equations fully support this kind of update, pyBKT doesn’t currently expose a convenient API for it. As a result, developers who use the library for online inference end up reimplementing the update step outside the library, fragmenting codebases and increasing maintenance cost.
An incremental interface would allow a single, consistent parameterization to serve both offline research and live applications.
Proposed Addition
Introduce a lightweight helper that performs a one-step Bayesian update given:
- the learner’s current mastery probability,
- the outcome of the latest observation (
correct/incorrect), - and the fitted parameters (
p_init,p_trans,p_slip,p_guess) for the relevant skill.
Example usage:
new_mastery = model.update_single(
prior=0.42,
is_correct=True,
skill="fractions"
)Internally, this would:
- Apply the standard posterior update based on the observation.
- Perform the transition step to obtain the next prior.
- Return the updated mastery probability (and optionally metadata like timestamp or count).
This operation is O(1) and can be composed seamlessly with existing model training and evaluation workflows.