nb-asr-paralleltext

Parallel text extraction and semantic alignment tools for Norwegian text.

This repository contains code and data for producing parallel text pairs using semantic similarity methods. The included scripts operate on sentence- and paragraph-level text and generate aligned pairs based on embedding similarity.

Included Files

Data

• bible_aligned_embeddings.csv
  Sentence-level parallel text extracted using embedding similarity.
  Contains original text, candidate match, similarity score, and metadata.

• bible_aligned_embeddings_75.csv
  Same as above, but filtered with a stricter similarity threshold (0.75).

• bible_parallel_paragraphs.csv
  Paragraph-level parallel text pairs generated by semantic comparison.

Scripts

• semantic_extract_1.py
  Extracts semantic sentence-level alignments using embeddings.
  Computes similarities and exports aligned pairs to CSV.

• semantic_extract_paragraph.py
  Computes paragraph-level semantic matches.
  Designed for larger blocks of text where sentence boundaries differ.

• similarity_sentences_with_number.py
  Computes similarity between numbered sentences and produces alignment data including sentence IDs.

Usage

Example sentence-level extraction:

python semantic_extract_1.py \
    --input text_source.txt \
    --reference text_target.txt \
    --output aligned.csv