overfitting in speech recognition task

[itsmekhoathekid]

❓ Questions and Help

Before asking:

1. search the issues. [yes]
2. search the docs. [yes]

What is your question?

I'm currently using VGGTransformer for a speech recognition task. My training loss gradually decreases, while my validation loss keeps increasing. (yeah, overfitting)

Code

this is my config

python /data/npl/Speech2Text/fairseq/train.py /data/npl/Speech2Text/fairseq/examples/speech_recognition/processed_vi --save-dir /data/npl/Speech2Text/fairseq/examples/speech_recognition/checkpoint_vi --max-epoch 80 --task speech_recognition --arch vggtransformer_1 --optimizer adam --lr 5e-4 --fp16 --memory-efficient-fp16 --warmup-updates 2500 --update-freq 4 --max-tokens 20000 --num-workers 2 --log-format json --log-interval 1 --criterion cross_entropy_acc --user-dir /data/npl/Speech2Text/fairseq/examples/speech_recognition/models --distributed-world-size 1 --ddp-backend=no_c10d --fp16

What have you tried?

I tried running it with a larger batch size, different optimizers, reducing the number of parameters, and increasing the dropout rate, but the issue still persists.

What's your environment?

• fairseq Version (e.g., 1.0 or main):
• PyTorch Version (e.g., 1.0)
• OS (e.g., Linux):
• How you installed fairseq (pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information: