feat(tags): implement auto-creation of tags and ensure system tags exist - #683
feat(tags): implement auto-creation of tags and ensure system tags exist#683storm1er wants to merge 4 commits into
Conversation
…ist. - Added `ensureSystemTagsExist` function to create critical system tags if they don't exist. - Updated `UpdateDocuments` method to support auto-creation of tags based on settings. - Introduced `TagSettings` component for managing tag auto-creation settings. - Enhanced UI to prevent deletion of last tag and provide user feedback. - Updated styles for last tag indication and tooltip.
WalkthroughAdds startup bootstrap to ensure required system tags exist; introduces Settings.TagsAutoCreate with UI toggle and backend support; updates tag mapping/creation and fallbacks in UpdateDocuments; enforces at-least-one-tag in UI and backend; prevents deleting the last tag; makes settings endpoint partial-update; updates getSuggestedTags to respect TagsAutoCreate; adds dev watch. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant M as main.go
participant P as Paperless API
participant Cache as Local Tag Cache
M->>P: GET /tags (fetch all tags)
P-->>M: Tag list
M->>Cache: Update known tags
loop For each required system tag
M->>P: POST /tags (create if missing)
alt Created
P-->>M: 201 Created
M->>Cache: Add created tag
else Exists
Note over M,Cache: No action
else Error
P-->>M: Error
M-->>M: Fatal exit
end
end
Note over M: Continue startup/background tasks
sequenceDiagram
autonumber
participant UI as Web App
participant S as Settings API
participant B as Backend UpdateDocuments
participant P as Paperless API
rect rgba(230,245,255,0.5)
note over UI,S: Tag auto-create toggle
UI->>S: GET ./api/settings
S-->>UI: settings { tags_auto_create }
UI->>S: POST ./api/settings (partial: tags_auto_create)
S-->>UI: 200 OK
end
rect rgba(240,255,240,0.5)
note over UI,B: Update documents with tags
UI-->>UI: Validate: each doc has ≥1 tag
UI->>B: UpdateDocuments(finalTagNames)
B-->>B: Map names→IDs (cache)
alt Missing tag
B-->>B: If system tag or TagsAutoCreate=true → create
B->>P: POST /tags
P-->>B: Created/ID
end
alt Resulting tag list empty
B-->>B: Fallback to pdfOCRCompleteTag or return error
end
B-->>UI: Success/Errors
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Should fix #659 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
paperless.go (1)
1254-1266: Handle “tag already exists” races when auto-creating tagsIf another server instance or concurrent request creates the tag after our initial
GetAllTagssnapshot, Paperless returns a 400/409 (“tag with this name already exists”). Today we bubble that up, causing the whole update (or even startup viaensureSystemTagsExist) to fail. Please treat this race as success by refreshing the tag list and returning the existing ID before giving up.A minimal fix inside
CreateTag:- if resp.StatusCode != http.StatusCreated && resp.StatusCode != http.StatusOK { - bodyBytes, _ := io.ReadAll(resp.Body) - return 0, fmt.Errorf("error creating tag: %d, %s", resp.StatusCode, string(bodyBytes)) - } + if resp.StatusCode != http.StatusCreated && resp.StatusCode != http.StatusOK { + bodyBytes, _ := io.ReadAll(resp.Body) + if (resp.StatusCode == http.StatusBadRequest || resp.StatusCode == http.StatusConflict) && + strings.Contains(strings.ToLower(string(bodyBytes)), "already exists") { + if refreshed, err := client.GetAllTags(ctx); err == nil { + if existingID, ok := refreshed[tagName]; ok { + return existingID, nil + } + } + } + return 0, fmt.Errorf("error creating tag: %d, %s", resp.StatusCode, string(bodyBytes)) + }web-app/src/components/CustomFieldsEditor.tsx (1)
9-14: Extract SettingsData interface to a shared types file.The
SettingsDatainterface is duplicated in bothCustomFieldsEditor.tsx(lines 9-14) andTagSettings.tsx(lines 4-9). This violates the DRY principle and creates a maintenance burden—changes to the settings structure must be synchronized across multiple files.Create a shared types file (e.g.,
web-app/src/types/settings.ts) and export the interface from there:// web-app/src/types/settings.ts export interface SettingsData { custom_fields_enable: boolean; custom_fields_selected_ids: number[]; custom_fields_write_mode: 'append' | 'replace' | 'update'; tags_auto_create: boolean; }Then import it in both components:
-interface SettingsData { - custom_fields_enable: boolean; - custom_fields_selected_ids: number[]; - custom_fields_write_mode: 'append' | 'replace' | 'update'; - tags_auto_create: boolean; -} +import { SettingsData } from '../types/settings';
♻️ Duplicate comments (1)
web-app/src/components/TagSettings.tsx (1)
4-9: Extract SettingsData interface to a shared types file.This interface is duplicated between
TagSettings.tsxandCustomFieldsEditor.tsx. See the comment onCustomFieldsEditor.tsx(lines 9-14) for the detailed refactoring suggestion.
🧹 Nitpick comments (2)
web-app/src/DocumentProcessor.tsx (1)
209-228: Consider persisting the error message when preventing last tag deletion.The error message "Cannot remove all tags..." is set using the shared
errorstate, which may be cleared by other operations (e.g., line 145 inhandleUpdateDocuments). If the user attempts to delete the last tag and then immediately clicks "Update Documents," the deletion error could be overwritten before the user sees it.Consider one of these approaches:
Option 1: Use a separate error state for tag deletion
const [tagDeletionError, setTagDeletionError] = useState<string | null>(null); // In handleTagDeletion if (currentTags.length <= 1) { setTagDeletionError('Cannot remove all tags. Documents must have at least one tag.'); setTimeout(() => setTagDeletionError(null), 3000); // Auto-clear after 3s return doc; } // Display both errors in the UI {error && <div className="mb-4 p-4 bg-red-100 ...">{error}</div>} {tagDeletionError && <div className="mb-4 p-4 bg-red-100 ...">{tagDeletionError}</div>}Option 2: Use toast/notification system
If you have a toast notification library, use that for transient validation messages like tag deletion prevention, reserving the main error state for critical failures.web-app/src/components/TagSettings.tsx (1)
73-75: Loading state could be more informative.The loading text "Loading settings..." uses a generic gray color that may not be immediately visible in dark mode contexts. Consider using the same styling pattern as the error state for consistency.
if (loading) { - return <div className="text-gray-400">Loading settings...</div>; + return ( + <div className="bg-gray-800 p-4 rounded-lg"> + <p className="text-gray-300">Loading settings...</p> + </div> + ); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (10)
main.go(2 hunks)paperless.go(3 hunks)settings.go(1 hunks)types.go(1 hunks)web-app/src/DocumentProcessor.tsx(2 hunks)web-app/src/components/CustomFieldsEditor.tsx(1 hunks)web-app/src/components/Settings.tsx(2 hunks)web-app/src/components/SuggestionCard.tsx(1 hunks)web-app/src/components/TagSettings.tsx(1 hunks)web-app/src/index.css(1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.go
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Ensure all Go source files are formatted with gofmt before committing
Files:
types.gosettings.gopaperless.gomain.go
web-app/src/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Fix all frontend lint/TypeScript issues by running npm run lint in web-app before committing
Files:
web-app/src/components/Settings.tsxweb-app/src/components/SuggestionCard.tsxweb-app/src/components/CustomFieldsEditor.tsxweb-app/src/DocumentProcessor.tsxweb-app/src/components/TagSettings.tsx
🧬 Code graph analysis (2)
web-app/src/components/Settings.tsx (1)
web-app/src/components/TagSettings.tsx (1)
TagSettings(11-137)
main.go (1)
paperless.go (1)
PaperlessClient(34-39)
🔇 Additional comments (4)
web-app/src/components/Settings.tsx (1)
4-4: LGTM!The integration of the TagSettings component follows the established pattern and is correctly positioned between PromptsEditor and CustomFieldsEditor sections.
Also applies to: 13-16
web-app/src/components/SuggestionCard.tsx (1)
103-105: LGTM!The conditional styling correctly applies the
react-tags__tag--lastclass when only one tag remains, which aligns with the UI requirement to visually indicate the last non-deletable tag.web-app/src/DocumentProcessor.tsx (1)
148-159: LGTM!The pre-update validation correctly ensures all documents have at least one tag before proceeding with the update, preventing invalid states and providing clear user feedback.
web-app/src/components/TagSettings.tsx (1)
1-137: Verify lint and TypeScript complianceEnsure
web-app/src/components/TagSettings.tsxpasses the project’s frontend checks by running in your repo root:cd web-app npm run lint -- src/components/TagSettings.tsx
storm1er
changed the title
|
Some UI issue remains, I have more tests to do |
…to-creation logic
|
@icereed |
|
If any of @xJylin, @onestopjs, @martinenko7 or @seafish1972 can help by testing this PR it would be nice. I only tested locally using mistral. Steps:
enjoy =) |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
web-app/src/components/TagSettings.tsx (1)
4-9: Consider narrowing the interface to managed fields only.The
SettingsDatainterface includes all settings fields, but this component only managestags_auto_create. While the partial update (line 54) correctly prevents overwrites, including unused fields in the interface can be confusing for maintainers.Consider creating a focused interface:
interface SettingsData { - custom_fields_enable: boolean; - custom_fields_selected_ids: number[]; - custom_fields_write_mode: 'append' | 'replace'; tags_auto_create: boolean; }Alternatively, if you need the full settings for display purposes, rename it to clarify scope:
interface FullSettingsData { custom_fields_enable: boolean; custom_fields_selected_ids: number[]; custom_fields_write_mode: 'append' | 'replace'; tags_auto_create: boolean; } interface TagSettingsData { tags_auto_create: boolean; }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
app_http_handlers.go(2 hunks)app_llm.go(1 hunks)docker-compose.yml(1 hunks)web-app/src/components/CustomFieldsEditor.tsx(2 hunks)web-app/src/components/Settings.tsx(2 hunks)web-app/src/components/TagSettings.tsx(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- web-app/src/components/Settings.tsx
🧰 Additional context used
📓 Path-based instructions (2)
web-app/src/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Fix all frontend lint/TypeScript issues by running npm run lint in web-app before committing
Files:
web-app/src/components/CustomFieldsEditor.tsxweb-app/src/components/TagSettings.tsx
**/*.go
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Ensure all Go source files are formatted with gofmt before committing
Files:
app_llm.goapp_http_handlers.go
🔇 Additional comments (9)
docker-compose.yml (1)
10-16: LGTM! Development workflow enhancement.The addition of
develop.watchenables automatic rebuilds during development, improving developer experience. The ignore patterns fornode_modules/and.git/are appropriate.app_llm.go (1)
153-161: LGTM! Correct implementation of TagsAutoCreate gating.The function properly reads
TagsAutoCreateunder a read lock and conditionally skips filtering when auto-creation is enabled. The early return pattern is appropriate and maintains backward compatibility when the setting is disabled.web-app/src/components/CustomFieldsEditor.tsx (2)
13-13: LGTM! Type definition extended appropriately.The
tags_auto_createfield is correctly added to theSettingsDatainterface for type consistency with the backend settings model.
65-73: LGTM! Correct partial update implementation.The component correctly sends only the fields it manages (
custom_fields_enable,custom_fields_selected_ids,custom_fields_write_mode), avoiding overwriting thetags_auto_createfield managed by the TagSettings component. This follows the partial update pattern that was previously discussed and resolved.app_http_handlers.go (2)
131-164: LGTM! Partial update implementation with proper type handling.The handler correctly:
- Parses incoming JSON as a map to support partial updates
- Type-checks each field with safe existence checks
- Converts JSON number types (float64) to integers appropriately for array indices
- Includes the new
tags_auto_createfield handlingThe approach enables concurrent editing of different settings sections without overwrites.
173-181: POST /api/settings consumer ignores response body, so safe
The only consumer (CustomFieldsEditor.tsx) checksresponse.okand never callsresponse.json()on success, so returning the full settings object won’t break existing code.web-app/src/components/TagSettings.tsx (3)
51-70: LGTM! Partial update correctly addresses past review concerns.The component correctly implements partial updates by:
- Sending only
tags_auto_createin the POST body (lines 53-55)- Updating local state from the server response to maintain consistency (lines 58-60)
This prevents overwriting settings managed by other components, addressing the concern raised in previous reviews.
Based on past review comments.
84-119: LGTM! Well-designed user interface with appropriate warnings.The UI implementation is clear and user-friendly:
- Checkbox properly labeled and accessible
- Descriptive text explains the feature clearly
- Warning box appropriately alerts users about side effects
- Good use of dark mode styling
122-142: LGTM! Proper save flow with user feedback.The save button and feedback implementation is correct:
- Button disabled when no changes or saving in progress
- Clear visual states for enabled/disabled
- Success and error messages displayed appropriately
- Message cleared after 3 seconds
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
web-app/src/components/TagSettings.tsx(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
web-app/src/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Fix all frontend lint/TypeScript issues by running npm run lint in web-app before committing
Files:
web-app/src/components/TagSettings.tsx
| setSaving(true); | ||
| setMessage(''); | ||
| setError(''); | ||
|
|
||
| try { | ||
| // Only send the field this component manages (partial update) | ||
| const response = await axios.post('./api/settings', { | ||
| tags_auto_create: settings.tags_auto_create | ||
| }); | ||
| setMessage('Settings saved successfully'); | ||
| // Update both settings and initialSettings with the server response | ||
| const settingsData = response.data.settings as SettingsData; | ||
| setSettings(settingsData); | ||
| setInitialSettings(settingsData); | ||
|
|
||
| // Clear message after 3 seconds | ||
| setTimeout(() => setMessage(''), 3000); | ||
| } catch (err) { | ||
| console.error('Error saving settings:', err); | ||
| setError('Failed to save settings'); | ||
| } finally { | ||
| setSaving(false); | ||
| } |
There was a problem hiding this comment.
Clear the success-message timeout to avoid state updates after unmount.
If the user navigates away before the 3 s timer fires (or saves repeatedly), the pending setTimeout still runs and calls setMessage on an unmounted component, triggering React warnings and leaking timers. Store the timeout ID in a ref, clear it before scheduling a new one, and tear it down on unmount.
@@
-import { useEffect, useState } from 'react';
+import { useEffect, useRef, useState } from 'react';
@@
const [message, setMessage] = useState('');
const [error, setError] = useState('');
+ const messageTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
+
+ useEffect(() => {
+ return () => {
+ if (messageTimeoutRef.current) {
+ clearTimeout(messageTimeoutRef.current);
+ }
+ };
+ }, []);
@@
- // Clear message after 3 seconds
- setTimeout(() => setMessage(''), 3000);
+ // Clear message after 3 seconds
+ if (messageTimeoutRef.current) {
+ clearTimeout(messageTimeoutRef.current);
+ }
+ messageTimeoutRef.current = setTimeout(() => setMessage(''), 3000);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| setSaving(true); | |
| setMessage(''); | |
| setError(''); | |
| try { | |
| // Only send the field this component manages (partial update) | |
| const response = await axios.post('./api/settings', { | |
| tags_auto_create: settings.tags_auto_create | |
| }); | |
| setMessage('Settings saved successfully'); | |
| // Update both settings and initialSettings with the server response | |
| const settingsData = response.data.settings as SettingsData; | |
| setSettings(settingsData); | |
| setInitialSettings(settingsData); | |
| // Clear message after 3 seconds | |
| setTimeout(() => setMessage(''), 3000); | |
| } catch (err) { | |
| console.error('Error saving settings:', err); | |
| setError('Failed to save settings'); | |
| } finally { | |
| setSaving(false); | |
| } | |
| // In web-app/src/components/TagSettings.tsx | |
| import { useEffect, useRef, useState } from 'react'; | |
| // … other imports | |
| function TagSettings(/* props */) { | |
| - const [message, setMessage] = useState(''); | |
| const [message, setMessage] = useState(''); | |
| const [error, setError] = useState(''); | |
| const messageTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null); | |
| useEffect(() => { | |
| return () => { | |
| if (messageTimeoutRef.current) { | |
| clearTimeout(messageTimeoutRef.current); | |
| } | |
| }; | |
| }, []); | |
| const saveSettings = async () => { | |
| setSaving(true); | |
| setMessage(''); | |
| setError(''); | |
| try { | |
| // Only send the field this component manages (partial update) | |
| const response = await axios.post('./api/settings', { | |
| tags_auto_create: settings.tags_auto_create | |
| }); | |
| setMessage('Settings saved successfully'); | |
| // Update both settings and initialSettings with the server response | |
| const settingsData = response.data.settings as SettingsData; | |
| setSettings(settingsData); | |
| setInitialSettings(settingsData); | |
| - // Clear message after 3 seconds | |
| // Clear message after 3 seconds | |
| if (messageTimeoutRef.current) { | |
| clearTimeout(messageTimeoutRef.current); | |
| } | |
| messageTimeoutRef.current = setTimeout(() => setMessage(''), 3000); | |
| } catch (err) { | |
| console.error('Error saving settings:', err); | |
| setError('Failed to save settings'); | |
| } finally { | |
| setSaving(false); | |
| } | |
| }; | |
| // … rest of component | |
| } |
🤖 Prompt for AI Agents
In web-app/src/components/TagSettings.tsx around lines 47 to 69, the success
message timeout is scheduled with setTimeout but never cleared, which can cause
state updates after unmount and timer leaks; change the implementation to store
the timeout ID in a ref, clear any existing timeout before scheduling a new one,
and clear the timeout in a useEffect cleanup on unmount (use window.clearTimeout
with the ref-held ID and set the ref to null after clearing) so repeated saves
or navigation away won't trigger setMessage on an unmounted component.
|
Hi, where do I see the Auto-Generate Tag button in the frontend? **"Frontend improvements: |
|
@seafish1972 There's a new ui where you can set it, you can also use app/config/settings.json 
|
|
Thanks. I'm pretty new to github. I cloned your forked version. There I can't see this new GUI. |
You need to have a valid You should be able to navigate to |
|
THX. It's working. But maybe I'm wrong but my expectation was placing the tag "paperless-gpt-ocr-auto" would do both tagging, title generating, finding creation, date, ..... AND doing a OCR. Am I wrong? Placing this tag "only" doing OCR and afterwards setting a tag "paperless-gpt-ocr-complete". No title generated, no creation date, no correspondent. |
|
I don't remember the variable names, but if you want them all, there's 2 way: put 2 tags in paperless for inbox, or setup papergpt to use the tag that triggers the title and stuff as the completed tag of ocr completion |
2 participants
TL;DR:
ensureSystemTagsExistfunction to create critical system tags if they don't exist.UpdateDocumentsmethod to support auto-creation of tags based on settings.TagSettingscomponent for managing tag auto-creation settings.This pull request introduces robust validation and management for document tags in both backend and frontend, ensuring that every document always has at least one tag and that system tags are reliably present. It also adds a user-facing setting for automatic tag creation and improves the UI to prevent accidental removal of all tags from a document.
Backend improvements:
ensureSystemTagsExistinmain.goto guarantee all required system tags exist in Paperless-ngx before processing documents. The app now fails fast if any system tag cannot be created. [1] [2]UpdateDocumentsinpaperless.goto auto-create missing tags when enabled, always auto-creating system tags, and to prevent documents from having zero tags by applying a fallback system tag. [1] [2]TagsAutoCreateto the backend settings (types.go,settings.go) and documented its usage in tag creation logic. [1] [2] [3]Frontend improvements:
TagSettingscomponent, integrated into the main settings page, allowing users to enable/disable automatic tag creation. [1] [2] [3] [4]Summary by CodeRabbit
New Features
Bug Fixes
Chores