Add an LLM based spam filter#96
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces an optional, per-distribution-list spam filter backed by Mistral AI, and replaces the previous single newsletter boolean with a more extensible flags object for distribution list behavior.
Changes:
- Add
flags(override recipient, spam filter) to distribution list API + web UI and update forwarding behavior accordingly. - Implement server-side LLM spam classification with MIME text extraction (plain + HTML) and persist classification results on inbound emails.
- Add EF Core migration for spam classification fields and add tests/resources for spam filtering and text extraction.
Reviewed changes
Copilot reviewed 27 out of 30 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| webapp/src/views/EditDistributionList.vue | Replaces newsletter with two flag checkboxes and sends flags payload on save. |
| webapp/src/views/DistributionLists.vue | Displays icons for enabled distribution list flags in the table. |
| webapp/src/services/distribution-list.ts | Changes create/modify request type to include flags object. |
| webapp/package-lock.json | Updates minimum Node engine version. |
| server/Mailist/SpamFilter/SpamFilterService.cs | New service that builds prompts, calls the LLM, and deserializes classification JSON. |
| server/Mailist/SpamFilter/SpamFilterOptions.cs | Adds configuration options for enabling and configuring the spam filter. |
| server/Mailist/SpamFilter/SpamCategory.cs | Defines classification categories used by the spam filter and email processing. |
| server/Mailist/SpamFilter/MimeTextExtractionService.cs | Extracts text from MIME bodies (plain/html) for LLM input. |
| server/Mailist/Models/Json/DistributionList.cs | Replaces Newsletter with a Flags object in API models; adds conversion helpers. |
| server/Mailist/Migrations/DatabaseContextModelSnapshot.cs | Updates snapshot for unique index change and new inbox email spam fields. |
| server/Mailist/Migrations/20260331155808_SpamFilter.Designer.cs | New migration designer snapshot for schema changes. |
| server/Mailist/Migrations/20260331155808_SpamFilter.cs | Adds spam category/justification columns; switches alias uniqueness constraint to unique index. |
| server/Mailist/Mailist.csproj | Adds AngleSharp + Mistral SDK dependencies; bumps JWT package version. |
| server/Mailist/Extensions/IServiceCollectionExtensions.cs | Binds and validates SpamFilter options section. |
| server/Mailist/EmailRelay/MimeMessageCreationService.cs | Adds rejection message generation for spam-filtered emails. |
| server/Mailist/EmailRelay/Entities/InboxEmail.cs | Adds SpamCategory and SpamJustification fields to the inbox email entity. |
| server/Mailist/EmailRelay/EmailRelayJobController.cs | Integrates spam classification into the relay pipeline and updates forwarding flag behavior. |
| server/Mailist/EmailRelay/DistributionListFlags.cs | Renames newsletter flag to OverrideRecipient and adds SpamFilter. |
| server/Mailist/DatabaseContext.cs | Updates EF mappings for distribution list alias uniqueness and spam category conversion. |
| server/Mailist/Controllers/DistributionListController.cs | Updates API to accept/return Flags object instead of Newsletter. |
| server/Mailist/appsettings.json | Adds SpamFilter configuration block (disabled by default). |
| server/Mailist.Tests/SpamFilterServiceTests.cs | Adds (optional) tests that call the real LLM when enabled via config. |
| server/Mailist.Tests/Resources/23437.header | Adds embedded test email header fixture (spam example). |
| server/Mailist.Tests/Resources/23437.body | Adds embedded test email body fixture (spam example). |
| server/Mailist.Tests/Resources/23377.header | Adds embedded test email header fixture (legit example). |
| server/Mailist.Tests/Resources/23377.body | Adds embedded test email body fixture (legit example). |
| server/Mailist.Tests/MimeTextExtractionServiceTests.cs | Adds unit test validating HTML-to-text compression. |
| server/Mailist.Tests/Mailist.Tests.csproj | Embeds .body/.header resources into the test assembly. |
| server/Mailist.Tests/Integration/DistributionListIntegrationTests.cs | Updates integration tests to use new Flags API. |
| .vscode/settings.json | Adds Tailwind CSS file association. |
Files not reviewed (2)
- server/Mailist/Migrations/20260331155808_SpamFilter.Designer.cs: Language not supported
- webapp/package-lock.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds a spam filter that can be enabled for each distribution list individually and classifies emails using Mistral AI.
Resolves #94