With the backfill migration complete, all 1:1 and self DMs now use
DirectMessageGroup recipients. Remove the PREFER_DIRECT_MESSAGE_GROUP
setting and all code paths that conditionally used personal recipients
when the setting was False.
This simplifies recipient_users.py to always use
get_or_create_direct_message_group, updates all data importers
(Slack, Mattermost, Rocket.Chat) to always create DM groups for
1:1 messages, and removes all @override_settings decorators and
tests that only exercised the personal-recipient code path.
Some tests see query count changes from removing the redundant
get_direct_message_group lookup in get_recipient_from_user_profiles
(which now falls through directly to
get_or_create_direct_message_group), and from tests that previously
pinned PREFER_DIRECT_MESSAGE_GROUP=False now producing DM group
recipients instead of personal ones.
This commit updates the invite users email input to correctly parse and
convert email addresses with attached names into pills.
The `email-addresses` library is now used to parse email addresses in
RFC5322 format, ensuring accurate extraction of names and emails.
A new `Invitee` dataclass is introduced in the backend to store invitee
details.
Fixes part of #32874.
Co-authored-by: Maneesh Shukla <shuklamaneesh24@gmail.com>
Since commit a352f2e10, the "delivery_email" field that's set here
hasn't been used.
The default value for bot_owner is None, so we don't need to set
that here, and we can set the bot_type field directly as a param
to create_user.
In preparation for implementing automated two-tier billing on plans,
renames BillingSession.current_count_for_billed_licenses to be
current_counts_for_billed_users. The function now returns a dataclass
object, BillingUserCounts, with fields for workplace and non-workplace
users.
Currently, the non-workplace user count is hardcoded to zero for all
billing session cases, and the previously returned integer for billed
license counts is returned as the workplace user count.
This commit exists to ease the deployment of the new 1:1 direct
message groups. Specifically, with PREFER_DIRECT_MESSAGE_GROUP=False,
we now respect (and make use of) direct message groups if they are
found for a pair; however, they are never created. With
PREFER_DIRECT_MESSAGE_GROUP=True, they are created if they do not
exist already.
Regardless of the setting, the NarrowBuilder merges both personal
recipients and 1:1 DM group recipients.
This is done because setting `PREFER_DIRECT_MESSAGE_GROUP` may not
happen atomically; in large instances doing this switch without
dropping services, changing `PREFER_DIRECT_MESSAGE_GROUP` will happen
in some processes before others. As such, some processes may begin
making 1:1 DM group recipients while other processes still have
PREFER_DIRECT_MESSAGE_GROUP=False. To always show both old and new
messages, we must include both flavors of DM recipient, and we must
ratchet towards the 1:1 DM group recipients.
A subsequent commit will handle the migration of old messages, which
can be applied after this commit is deployed, and
PREFER_DIRECT_MESSAGE_GROUP=True.
Co-authored-by: Mohammad Reza Kianifar <mohammad.76kiani@gmail.com>
When this command is run in isolation, the attachment object added for
the realm summary statistics needs to be deleted first, so that when
the cache is flushed the UserProfile object still exists. See
flush_used_upload_space_cache.
The PREFER_DIRECT_MESSAGE_GROUP setting controls the recipient
type for 1:1 and self DMs. It defaults to False, maintaining legacy
behavior. Tests can override this with
@override_settings(PREFER_DIRECT_MESSAGE_GROUP=True) to test the new
DirectMessageGroup recipient type behavior.
While calling `get_apns_alert_title`, we already have
the channel name and topic display name available.
Instead of doing a DB query to get channel name and
preparing topic display name again within `get_apns_alert_title`,
we pass these two values as arguments.
Signed-off-by: Prakhar Pratyush <prakhar@zulip.com>
If a user had both android and apple device registered,
preparing legacy payload resulted in executing
`get_message_payload` twice - which includes DB queries
and other operations.
This commit extracts out the function to avoid duplicate
call while preparing legacy FCM and APNs payload.
Signed-off-by: Prakhar Pratyush <prakhar@zulip.com>
For get and filter queries of NamedUserGroup, realm_for_sharding
field is used instead of realm field, as directly using
realm_for_sharding field on NamedUserGroup makes the query faster
than using realm present on the base UserGroup table.
To maintain API compatibility during and after the migration to direct
message groups for 1:1 and self messages, update the analytics count
query to treat DM groups of size 2 or less as private messages.
APNs tokens are provided by the client in hex, and we store them in
hex. The existing code which attempts to "validate" them by parsing
them as base64 only works because base64 is a superset of hex.
Enforce that APNs tokens are hex, and remove all of the pieces of test
code which were incorrectly passing them in as base64 strings.
This is most important for `send_zulip_update_announcements`, which
can race with the version run as a post-deploy hook. However, all of
these crons can tolerate being slightly delayed, and there's little
benefit to them taking CPU or possibly hitting odd borderline race
conditions when the deploy is in progress.
For safety, we only trust the deploy lockfile if it was created
within the last hour -- deploys should not take more than an hour, and
failing to ever run hourly crons is much worse than perhaps running
them during a real very-long deploy.
In the analytics tests, where users are created as part of the set up
of the test, we want to get users by the delivery email and not the
visible-to-other-users email field.
Prep for setting default new user email visibility in realm creation
by the organization type.
Updates the labels in the"Messages set by client" chart so that
the Flutter app is no longer labeled as "beta".
The React Native app is now labeled as "old". And the older
Android and iOS apps are now labeled as "ancient".
This was confusingly doing an assertion about the subscription being
active, not the channel. We could rename it to
require_active_subscription. But it was only passed with a non-default
value in b2cb443d24, and that call was
removed in 378062cc83.
When these exceptions are thrown from the request-to-bouncer functions
inside of manage.py register_server/update_analytics_counts, they
shouldn't be silenced, merely calling maybe_mark_pushes_disabled in the
background.
This results in the occurrence of the error not being shown to the
user. Failure to upload analytics data when running these commands
should result in a loud, obvious error.
Failure of running register_server before this change:
```
./manage.py register_server
This command registers your server for the Mobile Push Notifications Service.
Doing so will share basic metadata with the service's maintainers:
* This server's configured hostname: zulipdev.com:9991
* This server's configured contact email address: desdemona+admin@zulip.com
* Metadata about each organization hosted by the server; see:
<https://zulip.com/doc-permalinks/basic-metadata>
Use of this service is governed by the Zulip Terms of Service:
<https://zulip.com/policies/terms>
Do you want to agree to the Zulip Terms of Service and proceed? [Y/n]
Mobile Push Notification Service registration successfully updated!
```
The occurrence of the error is not revealed to the user. Same concern
applies to the update_analytics_counts command.
After this change:
```
./manage.py register_server
This command registers your server for the Mobile Push Notifications Service.
Doing so will share basic metadata with the service's maintainers:
<...>
Do you want to agree to the Zulip Terms of Service and proceed? [Y/n]
Traceback (most recent call last):
File "/srv/zulip/./manage.py", line 150, in <module>
execute_from_command_line(sys.argv)
File "/srv/zulip/./manage.py", line 115, in execute_from_command_line
utility.execute()
File "/srv/zulip-venv-cache/bb36fc1fcb6d8c70a9a0bcb7bac45d78623a9ff4/zulip-py3-venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/srv/zulip-venv-cache/bb36fc1fcb6d8c70a9a0bcb7bac45d78623a9ff4/zulip-py3-venv/lib/python3.10/site-packages/django/core/management/base.py", line 413, in run_from_argv
self.execute(*args, **cmd_options)
File "/srv/zulip/zerver/lib/management.py", line 97, in execute
super().execute(*args, **options)
File "/srv/zulip-venv-cache/bb36fc1fcb6d8c70a9a0bcb7bac45d78623a9ff4/zulip-py3-venv/lib/python3.10/site-packages/django/core/management/base.py", line 459, in execute
output = self.handle(*args, **options)
File "/srv/zulip/zerver/management/commands/register_server.py", line 137, in handle
send_server_data_to_push_bouncer(consider_usage_statistics=False, raise_on_error=True)
File "/srv/zulip/zerver/lib/remote_server.py", line 453, in send_server_data_to_push_bouncer
response = send_to_push_bouncer(
File "/srv/zulip/zerver/lib/remote_server.py", line 233, in send_to_push_bouncer
raise JsonableError(msg)
zerver.lib.exceptions.JsonableError: Duplicate registration detected.
```
If this is called on a user without is_mirror_dummy=True, that seems
certain to be a bug. Therefore, an assert is preferable in order to
catch this, rather than returning early with noop like some other
function such as do_deactivate_user.
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm, --test-custom-db, Debian 12 production install with custom db name and user, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:jammy, , Ubuntu 22.04 production install and PostgreSQL upgrade with pgroonga, jammy) (push) Has been cancelled
We were using admins group as a hardcoded value for the default of
`can_remove_subscribers_group`, now we use a function to get the value
of the default group.
Created using manage.py squashmigrations, with my patch to the Django
migration optimizer to correctly collapse
AddConstraints/RemoveConstraints operations.
This commit adds a new `group_size` field to the `DirectMessageGroup`
model, and backfills its value to each of the existing direct message
groups.
Fixes part of #25713
analytics_usercount_property_realm_id_end_time_591dbec1_idx, added
back in b7df84d5a8, makes this lookup actually indexed.
195defb031 rewrote this query in a way which stopped using the
index.
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
