Post Processing

In post-processing, our goal is to correct possible errors in asr output, based on the user's emails.

The first step is to detect possible errors in an asr output, using the merged language model. The criterium follows:

Every word (except the first and the last) belongs to three trigrams. If two of these three trigrams have low probability in the merged language model, the word is considered to be an error. For this purpose, the ngram_error_detector.py tool is implemented:

Usage:

$ python ngram_error_detector.py -h
usage: ngram_error_detector.py [-h] --input INPUT --lm LM --n N --threshold
                               THRESHOLD [--print_words] [--save]

Tool for detecting error words in ASR output based on ngrams of a corpus

optional arguments:
  -h, --help            show this help message and exit

required arguments:
  --input INPUT         Input file that contains the asr output (one sentence
                        per line)
  --lm LM               Input language model
  --n N                 Size of n-gram
  --threshold THRESHOLD
                        Threshold for error detection

optional arguments:
  --print_words         If set, print the errors words with different color
  --save                If set, save errors in pickle format

The next and final step is to substitute the detected words by the correct ones. Here, we can make three assumptions:

• The correct word has close POS tagging to the wrong one.
• The correct word has close semantic representation to the wrong one.
• The correct word is close to the wrong one based on the Levenshtein distance.

All these methods are implemented in the error_corrector.py tool.

Usage:

$ python error_corrector.py -h
usage: error_corrector.py [-h] --input_sent INPUT_SENT --input_errors
                          INPUT_ERRORS --method {pos,semantic,word}
                          [--pos POS] [--vec VEC] [--ngram NGRAM]
                          [--weight WEIGHT] [--save] [--output OUTPUT]

Tool for correcting error words in an email

optional arguments:
  -h, --help            show this help message and exit

required arguments:
  --input_sent INPUT_SENT
                        Input file that contains the asr output (one sentence
                        per line)
  --input_errors INPUT_ERRORS
                        Pickle file that holds the errors of input sentences
  --method {pos,semantic,word}
                        Method to be used for finding errors

optional arguments:
  --pos POS             Pos tags of emails in pickle format if pos method is
                        chosen
  --vec VEC             Vectors of emails in pickle format if semantic method
                        is chosen
  --ngram NGRAM         Ngrams of emails in pickle format if word method is
                        chosen
  --weight WEIGHT       Weight in computing the min distance
  --save                Save corrected ASR output
  --output OUTPUT       If save is true, this is the path to the output file

The results are not so good and another thought could be that the correct word is phonetically close to the wrong one. This is added as a possible extension of the current project.