Can we reliably improve the output of Back Translations by sending the drafts to an LLM for correction.

[davidbaines]

We could try a prompt like this, where language is the language of the Back Translation.

Prompt:
"You are a Bible editor. The text below is marked up with USFM and I would like some minimal corrections made.
Please fix only the following errors:

1. The names of people and places. (e.g. Ensure that the biblical names match common translations)
2. Factual errors (e.g., if wrong person is identified as author)
3. Missing or incorrect key theological terms.
   Do not make any changes to:
4. The language style.
5. Vocabulary choices unless clearly wrong.
6. The explanatory notes that are in parentheses.
7. The USFM markup and layout
   Make only necessary corrections and keep everything else unchanged."

We could also ask the LLM to mark changes and offer examples in the prompts.
"For all changes add # marks around the incorrect part and { } around the correction. Here are examples of markup:

Input:
\v 4 When the \nd Lord\nd* saw that he had gone over to look, God called to him from within the bush, “Peter! Peter!”
Output:
\v 4 When the \nd Lord\nd* saw that he had gone over to look, God called to him from within the bush, “#Peter#{Moses}! #Peter#{Moses}!”

This relies on the LLM knowing the correct name and obviously has the downside that it might correct some names that are incorrect in the translation. There's a tradeoff. The better the output of the AI drafting the less need there is for further correction. This might be useful where names or numbers are very poor in the draft.