Fix print TTFT message in debug. - #162
Conversation
There was a problem hiding this comment.
Code Review
This pull request modifies genai_processors/debug.py to use a local variable message instead of mutating self._message directly, preventing the accumulation of TTFT suffixes across multiple calls. However, the reviewer noted that self._start and self._ttft are still not reset between calls, which can lead to incorrect TTFT calculations on subsequent calls, and suggested resetting them to None at the start of each call.
| content: processor.ProcessorStream, | ||
| ) -> AsyncIterable[ProcessorPartTypes]: | ||
| first_part = True | ||
| message = self._message |
There was a problem hiding this comment.
While this change successfully prevents the self._message string from accumulating TTFT suffixes across multiple calls, the actual self._start and self._ttft state variables are still not reset between calls. Because self._start remains set from the previous call, any subsequent call to the same TTFTSingleStream instance will immediately compute an incorrect TTFT based on the previous call's end time. To fully fix this, self._start and self._ttft should be reset to None at the beginning of each call (for example, in the call method of TTFTSingleStream).
There was a problem hiding this comment.
Nope, self._start is update in log_on_close which is always called before log_on_first, we're good here.
Small fix on the message for TTFT in debug.py: reset the TTFT for new calls.