[ENG-8064] Add New Notifications Data Model #11151

Johnetordoff · 2025-05-20T14:09:45Z

Purpose

This is Phase 2 of the Notifications Refactor it creates a new notification system and migrates the existing data into that model.

Changes

adds new data model for Notification, NotificationTypes and Subscriptions
creates a notifications.yaml for data dependency notificationtypes
add migrations
updates notifications to use NotificationTypes
updates Admin app to control email templates
updates metrics reporter to count notifications sent etc.
updates tests to all use capture_notifications mocking util

QA Notes

Please make verification statements inspired by your code and what your code touches.

Verify
Verify

What are the areas of risk?

Any concerns/considerations/questions that development raised?

Documentation

Side Effects

Ticket

https://openscience.atlassian.net/browse/ENG-8064

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: [ENG-7965] Add v2 email token confirmation endpoints (CenterForOpenScience#11139)

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: fix issue where not having any external identities caused a 500

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: [ENG-7966] Add "collected-in" relationship for Nodes (CenterForOpenScience#11140)

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: fix issue where trying another already confirmed email threw an uncaught exception (CenterForOpenScience#11161) [ENG-8148] Add ArtifactOutcome in annotations to linked nodes (CenterForOpenScience#11158)

…OpenScience/osf.io into add-new-notifications-data-model * 'refactor-notifications' of https://github.com/CenterForOpenScience/osf.io: flake8 fixed tests remove quickfiles update mails mock Update send_mail mocks Clean up tests Clean up tests Clean up imports Remove Meetings, Comments and OSF Groups Notifications remove osf groups remove osf groups remove osf groups remove osf groups remove osf groups # Conflicts: # tests/test_notifications.py

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: Update hybrid values for new workflow check (CenterForOpenScience#11166)

…OpenScience/osf.io into add-new-notifications-data-model * 'refactor-notifications' of https://github.com/CenterForOpenScience/osf.io: ignore Django maintenance state outside block

…OpenScience/osf.io into add-new-notifications-data-model * 'refactor-notifications' of https://github.com/CenterForOpenScience/osf.io: fixed bug with contributors remove superfluildous `groups` from serializer fix new and noteworth nodes bug going to Sentry

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: fixed None issue when iterate (CenterForOpenScience#11192) [ENG-8048] Remove caching to avoid incorrect results for ascendants (CenterForOpenScience#11169) [ENG-7870] Crossref DOIs not minting with _v1, OSF is displaying DOI versions with _v1 (CenterForOpenScience#11154) Update changelog and package.json [ENG-8145] [ENG-8147] Manual DOI and GUID for Preprints & Registrations - BE (CenterForOpenScience#11174)

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: Revert "[ENG-8048] Remove caching to avoid incorrect results for ascendants (…" Revert "fixed None issue when iterate (CenterForOpenScience#11192)"

…OpenScience/osf.io into add-new-notifications-data-model * 'refactor-notifications' of https://github.com/CenterForOpenScience/osf.io:

into add-new-notifications-data-model * 'develop' of https://github.com/CenterForOpenScience/osf.io: Bump version no. Add CHANGELOG move CROSSREF_UNAVAILABLE_DELAY to settings.py handle and 5xx status code from crossref handle 500 error from crossref Update changelog and package.json

Johnetordoff · 2025-07-25T13:44:05Z

addons/base/views.py

-        file_signals.file_updated.send(target=node, user=user, event_type=action, payload=payload)
+    file_signals.file_updated.send(target=node, user=user, event_type=action, payload=payload)
+
+    match f'node_{action}':


Improve this line, I think we can map actions better.

Johnetordoff · 2025-07-25T13:45:38Z

api/nodes/serializers.py

@@ -1256,11 +1256,15 @@ def create(self, validated_data):

        try:
            contributor_dict = {
-                'auth': auth, 'user_id': id, 'email': email, 'full_name': full_name, 'send_email': send_email,
-                'bibliographic': bibliographic, 'index': index, 'save': True,
+                'auth': auth,


revert this

Johnetordoff · 2025-07-25T13:46:13Z

api/nodes/views.py

+                user=user,
+                event_context={
+                    'guid': node._id,
+                    'title': node._id,


Johnetordoff · 2025-07-25T13:46:26Z

api/nodes/views.py

+                user=user,
+                event_context={
+                    'guid': fork._id,
+                    'title': node._id,


Johnetordoff · 2025-07-25T13:46:51Z

api/providers/serializers.py


        if bool(get_perms(user, provider)):
            raise ValidationError('Specified user is already a moderator.')
        if 'claim_url' in context:
-            template = mails.CONFIRM_EMAIL_MODERATION(provider)
+            template = NotificationType.Type.PROVIDER_CONFIRM_EMAIL_MODERATION


refactor template

Johnetordoff · 2025-07-25T13:47:01Z

api/providers/serializers.py

-            user.username,
-            template,
-            **context,
+        print(template, context)


Johnetordoff · 2025-07-25T13:47:30Z

api/providers/tasks.py

@@ -636,88 +639,78 @@ def bulk_upload_finish_job(upload, row_count, success_count, draft_errors, appro
    approval_errors.sort()
    if not dry_run:
        upload.save()
+        notification_type = None


Johnetordoff · 2025-07-25T13:47:53Z

api/providers/tasks.py

        upload.email_sent = timezone.now()
        upload.save()
-        logger.info(f'Email sent to bulk upload initiator [{initiator._id}]')
+        logger.info(f'Notification sent to bulk upload initiator [{initiator._id}]')


… notifications

Johnetordoff · 2025-07-25T19:33:18Z

api/requests/serializers.py

+                message_frequency='instantly',
+                event_context={
+                    'sender': sender.username,
+                    'bcc_addr': [sender.username] if validated_data['bcc_sender'] else None,


Consider improving bcc and reply_to support.

Does mailhog test for this?

Johnetordoff force-pushed the add-new-notifications-data-model branch 3 times, most recently from ecc8e88 to 3a8b414 Compare May 21, 2025 14:28

add new data model for notifications

57b0bd1

Johnetordoff force-pushed the add-new-notifications-data-model branch from 3a8b414 to 57b0bd1 Compare May 21, 2025 14:42

Johnetordoff added 8 commits May 22, 2025 09:39

Merge branch 'feature/pbs-25-10' of https://github.com/CenterForOpenS…

929b9fd

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: [ENG-7965] Add v2 email token confirmation endpoints (CenterForOpenScience#11139)

add new notificationsubscription class to views

69231e9

Merge branch 'feature/pbs-25-10' of https://github.com/CenterForOpenS…

c324716

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: fix issue where not having any external identities caused a 500

Merge branch 'feature/pbs-25-10' of https://github.com/CenterForOpenS…

40b38f7

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: [ENG-7966] Add "collected-in" relationship for Nodes (CenterForOpenScience#11140)

Merge branch 'feature/pbs-25-10' of https://github.com/CenterForOpenS…

2ab3589

…cience/osf.io into add-new-notifications-data-model * 'feature/pbs-25-10' of https://github.com/CenterForOpenScience/osf.io: Update hybrid values for new workflow check (CenterForOpenScience#11166)

Merge branch 'refactor-notifications' of https://github.com/CenterFor…

c449599

…OpenScience/osf.io into add-new-notifications-data-model * 'refactor-notifications' of https://github.com/CenterForOpenScience/osf.io: ignore Django maintenance state outside block

Johnetordoff force-pushed the add-new-notifications-data-model branch from 2b8ba0d to c449599 Compare June 9, 2025 13:40

Johnetordoff changed the base branch from feature/pbs-25-10 to refactor-notifications June 13, 2025 14:40

Johnetordoff added 5 commits June 16, 2025 10:46

fix absolute url issue

f550c61

fix up unit test issues

458fbfd

Merge branch 'refactor-notifications' of https://github.com/CenterFor…

bb145f4

…OpenScience/osf.io into add-new-notifications-data-model * 'refactor-notifications' of https://github.com/CenterForOpenScience/osf.io:

Johnetordoff force-pushed the add-new-notifications-data-model branch 2 times, most recently from dfe88a2 to ad18e9d Compare July 3, 2025 13:48

fix backward compat issues and remove old tests

300524c

Johnetordoff force-pushed the add-new-notifications-data-model branch from ad18e9d to 300524c Compare July 3, 2025 14:20

Johnetordoff added 4 commits July 7, 2025 10:27

split notification models into 3 files and improve interval choices

85e1342

clean-up tests and pass frequency data properly

f2e5309

update management commands and tests for notification migration

0471b76

Johnetordoff marked this pull request as ready for review July 8, 2025 20:05

Johnetordoff added 5 commits July 22, 2025 16:47

fix up user claim message notification tests

b3ca1b8

add bulk registration upload notifications to tests

2ac9ef9

fix issues with notifications when adding nodes to instituions

9815f1f

fix issues with collection submissions notifications

7c08559

fix reviewable and contributor notifications

86a9466

Johnetordoff force-pushed the add-new-notifications-data-model branch from 51d2685 to 86a9466 Compare July 23, 2025 17:20

Johnetordoff added 2 commits July 24, 2025 10:05

fix preprint versioning tests

0e2f169

update withdraw request declined notification

0139a5f

Johnetordoff force-pushed the add-new-notifications-data-model branch from 8495ecd to 0139a5f Compare July 24, 2025 14:22

update registration detail

b21bf71

Johnetordoff force-pushed the add-new-notifications-data-model branch from 012a455 to b4899ee Compare July 25, 2025 13:22

Johnetordoff commented Jul 25, 2025

View reviewed changes

api/nodes/views.py Outdated

user=user,

event_context={

'guid': node._id,

'title': node._id,

Copy link

Contributor Author

Johnetordoff Jul 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

redo

Johnetordoff commented Jul 25, 2025

View reviewed changes

api/nodes/views.py Outdated

user=user,

event_context={

'guid': fork._id,

'title': node._id,

Copy link

Contributor Author

Johnetordoff Jul 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

redo

Johnetordoff commented Jul 25, 2025

View reviewed changes

api/providers/serializers.py Outdated

user.username,

template,

**context,

print(template, context)

Copy link

Contributor Author

Johnetordoff Jul 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove

Johnetordoff commented Jul 25, 2025

View reviewed changes

fix invite and institutional admin contributor tests

7e96d6f

Johnetordoff force-pushed the add-new-notifications-data-model branch from b4899ee to 7e96d6f Compare July 25, 2025 14:27

Johnetordoff added 4 commits July 25, 2025 12:11

clean up test_addons code

7dc8375

fix up auth_views tests

fb6086b

fix up contributor and desk notifications

53b89ac

deletes detect_duplicate notifications as that only applied to legacy…

46b6944

… notifications

Johnetordoff force-pushed the add-new-notifications-data-model branch from 21ce32a to 46b6944 Compare July 25, 2025 17:40

fix spam ban notification type and split up webtests into smaller files

dcdbdc7

Johnetordoff commented Jul 25, 2025

View reviewed changes

more clean-up from self-CR

b5bdebe

Johnetordoff force-pushed the add-new-notifications-data-model branch from 616d17b to b5bdebe Compare July 25, 2025 20:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ENG-8064] Add New Notifications Data Model #11151

[ENG-8064] Add New Notifications Data Model #11151

Uh oh!

Johnetordoff commented May 20, 2025 •

edited

Loading

Uh oh!

Johnetordoff Jul 25, 2025

Uh oh!

Johnetordoff Jul 25, 2025

Uh oh!

Johnetordoff Jul 25, 2025

Uh oh!

Johnetordoff Jul 25, 2025

Uh oh!

Johnetordoff Jul 25, 2025

Uh oh!

Johnetordoff Jul 25, 2025

Uh oh!

Johnetordoff Jul 25, 2025

Uh oh!

Johnetordoff Jul 25, 2025

Uh oh!

Johnetordoff Jul 25, 2025

Uh oh!

Johnetordoff Jul 25, 2025

Uh oh!

Uh oh!

[ENG-8064] Add New Notifications Data Model #11151

Are you sure you want to change the base?

[ENG-8064] Add New Notifications Data Model #11151

Uh oh!

Conversation

Johnetordoff commented May 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Changes

QA Notes

Documentation

Side Effects

Ticket

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Johnetordoff commented May 20, 2025 •

edited

Loading