Skip to content

Commit 022c65e

Browse files
Update openai_gen.py to include reasoning behind should_manually_add_dot to openai_gen
1 parent 8814efc commit 022c65e

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

src/monitors4codegen/monitor_guided_decoding/openai_gen.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,9 @@ def convert_bytesrep_to_bytes(x: str) -> bytes:
133133
dot_found = True
134134
break
135135

136+
# When "stop" sequence is sent to openai model, it will not generate text beyond the text sequence within the "stop" parameter.
137+
# However, when it stops because of the "stop" sequence, the returned text does not contain the stop sequence, and only includes
138+
# text upto the stop sequence. So, the following code determines if the stop sequence "." needs to be added manually.
136139
should_manually_add_dot = None
137140
if response.choices[0].finish_reason == 'stop':
138141
if dot_found:

0 commit comments

Comments
 (0)