Skip to content

feat: implement feed url updated notification#1739

Draft
davidgamez wants to merge 16 commits into
mainfrom
notification_feed_url_updated
Draft

feat: implement feed url updated notification#1739
davidgamez wants to merge 16 commits into
mainfrom
notification_feed_url_updated

Conversation

@davidgamez

@davidgamez davidgamez commented Jun 12, 2026

Copy link
Copy Markdown
Member

Summary:

This PR introduces the feed URL update notification. The URL update notification has two variants: URL replacement and feed redirect. Added helpers to emit notifications, rate limiting, and Brevo integration.

Expected behavior:

  • When a URL is replaced on automated imports/operations API, a feed.url_updated => feed_redirected event is logged into the notification_event table
  • When a feed is redirected using the MD catalog import/operations API, a feed.url_updated => url_replaced event is logged into the notification_event table.
  • Events are sent every day on schedule for daily cadence and on Monday for Weekly cadence

Testing tips:

[Internal team]
For integrated tests, use DEV environment. As the notification endpoints are not implemented, notification-related entities need to be created via SQL.

  • Subscribe a user to a notification by inserting a row in notification_subscription table; replace with the target user id(REPLACE_ME_WITH_USER_ID)
 INSERT INTO notification_subscription (id, user_id, notification_type_id, cadence, digest, active, active_since)
 VALUES (gen_random_uuid()::text, REPLACE_ME_WITH_USER_ID, 'feed.url_updated',
'weekly', false, true, now() - interval '1 day');
  • Simulate a notification_event:
WITH ev AS ( INSERT INTO notification_event (id, notification_type_id, event_subtype, source, payload, created_at) VALUES (gen_random_uuid()::text, 'feed.url_updated', 'url_replaced', 'manual_test',
  '{"old_url":"https://old/feed.zip","new_url":"https://new/feed.zip"}'::jsonb, now()) RETURNING id)
 INSERT INTO notification_event_feed (id, notification_event_id, feed_stable_id, role)
 SELECT gen_random_uuid()::text, ev.id, 'mdb-10', 'subject' FROM ev;
  • Call the notifications_dispatch_plan task from retool(DEV)
  • Verify that the user receives a raw email(not from a template)

From our AI friend

This pull request introduces a new notification event system for tracking and emitting feed URL changes and feed redirections, as well as a reusable rate limiter utility. The main changes include the addition of a generic rate limiter, the implementation of fire-and-forget notification event writers, and the integration of these notification triggers into the GTFS and GBFS database population scripts. This enhances observability and user notification when critical feed properties change.

Notification event system:

  • Added notification_event_service module with emit_url_replaced and emit_feed_redirected functions for writing notification events to the users DB in a fire-and-forget manner. Includes helpers for URL normalization and difference detection.
  • Introduced notification_constants module containing namespaced string constants for notification types, event subtypes, roles, cadences, statuses, and sources, ensuring consistency across the notification system.
  • Added __init__.py to the shared.notifications package to document and expose notification utilities.

Integration with feed population scripts:

  • Updated populate_db_gtfs.py and populate_db_gbfs.py to emit notification events when:
    • A feed's producer URL changes in-place (emit_url_replaced).
    • A feed is redirected to another (emit_feed_redirected), including relevant metadata. [1] [2] [3] [4] [5]

Utility and infrastructure:

  • Added a generic, thread-safe, reusable token-bucket RateLimiter class and a process-wide registry for managing outbound API call rates, with full documentation and testability hooks.
    Please make sure these boxes are checked before submitting your pull request - thanks!
  • Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
  • Add or update any needed documentation to the repo
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)


logger = logging.getLogger(__name__)

_DEFAULT_SENDER_EMAIL = "noreply@mobilitydatabase.org"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[out-of-scope]: Configurations should be more dynamic, avoiding the need for redeployment when they are changed.

return get_rate_limiter(_BREVO_RATE_LIMITER_NAME, _configured_brevo_rps())


_DIGEST_EMAIL_SUBJECT_DICTIONARY = {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[out-of-scope]: internationalization messages

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant