Alex Vandiver c5200e8b05 deliver_scheduled_emails: Use a queue, instead of infinite retries. ... `deliver_scheduled_emails` tries to deliver the email synchronously, and if it fails, it retries after 10 seconds. Since it does not track retries, and always tries the earliest-scheduled-but-due message first, the worker will not make forward progress if there is a persistent failure with that message, and will retry indefinitely. This can result in excessive network or email delivery charges from the remote SMTP server. Switch to delivering emails via a new queue worker. The `deliver_scheduled_emails` job now serves only to pull deferred jobs out of the table once they are due, insert them into RabbitMQ, and then delete them. This limits the potential for head-of-queue failures to failures inserting into RabbitMQ, which is more reasonable than failures speaking to a complex external system we do not control. Retries and any connections to the SMTP server are left to the RabbitMQ consumer. We build a new RabbitMQ queue, rather than use the existing `email_senders` queue, because that queue is expected to be reasonably low-latency, for things like missed message notifications. The `send_future_email` codepath which inserts into ScheduledEmails is also (ab)used to digest emails, which are extremely bursty in their frequency -- and a large burst could significantly delay emails behind it in the queue. The new queue is explicitly only for messages which were not initiated by user actions (e.g., invitation reminders, digests, new account follow-ups) which are thus not latency-sensitive. Fixes: #32463.		2025-03-04 16:09:25 -08:00
..
__init__.py
base.py	worker: Address sentry_sdk deprecations.	2024-10-22 10:05:01 -07:00
deferred_email_senders.py	deliver_scheduled_emails: Use a queue, instead of infinite retries.	2025-03-04 16:09:25 -08:00
deferred_work.py	realm: Add option to schedule data deletion while deactivating.	2024-12-18 23:06:12 -08:00
digest_emails.py	ruff: Fix UP035 Import from collections.abc, typing instead.	2024-07-13 22:28:22 -07:00
email_mirror.py	send_email: Parse emails in a way mypy 1.14 understands.	2024-12-21 21:06:53 -08:00
email_senders_base.py	deliver_scheduled_emails: Use a queue, instead of infinite retries.	2025-03-04 16:09:25 -08:00
email_senders.py	deliver_scheduled_emails: Use a queue, instead of infinite retries.	2025-03-04 16:09:25 -08:00
embed_links.py	embed_links: Add savepoint=False to avoid creating savepoints.	2024-11-21 14:55:15 -08:00
embedded_bots.py	worker: Flag messages processed by embedded bot.	2025-01-24 17:56:44 -08:00
missedmessage_emails.py	ruff: Fix PLC0206 Extracting value from dictionary without calling .items().	2024-12-21 21:06:53 -08:00
missedmessage_mobile_notifications.py	ruff: Fix PLR6104 Use += to perform an augmented assignment directly.	2024-07-14 13:49:51 -07:00
outgoing_webhooks.py	worker: Flag messages processed by outgoing bot.	2025-01-24 17:56:44 -08:00
queue_processors.py	ruff: Fix UP007 Use X | Y for type annotations.	2024-07-13 22:28:22 -07:00
test.py	ruff: Fix UP035 Import from collections.abc, typing instead.	2024-07-13 22:28:22 -07:00
thumbnail.py	models: Add content_type to ImageAttachment.	2025-01-31 14:29:57 -08:00
user_activity_interval.py	ruff: Fix UP035 Import from collections.abc, typing instead.	2024-07-13 22:28:22 -07:00
user_activity.py	ruff: Fix UP006 Use list instead of List for type annotation.	2024-07-13 22:28:22 -07:00