Fine-Tuning an LLM for Log Anomaly Detection Using Sequence Probability

[joaorr13]

I’m exploring the use of LLMs for anomaly detection in logs. My idea is to fine-tune an LLM (e.g., LLaMA 3 8B) using only non-anomalous logs. After training, I’d like the model to estimate the likelihood of a log sequence and classify it as anomalous or not based on a probability threshold. First, do you think this could be an interesting approach for this task? Second, are you aware of similar work—maybe in a different domain—where an LLM was fine-tuned to detect anomalies or outliers, or to estimate the probability of a sequence? I’m very new to fine-tuning and would love pointers to examples or resources to see how it’s done.