With the backfill migration complete, all 1:1 and self DMs now use
DirectMessageGroup recipients. Remove the PREFER_DIRECT_MESSAGE_GROUP
setting and all code paths that conditionally used personal recipients
when the setting was False.
This simplifies recipient_users.py to always use
get_or_create_direct_message_group, updates all data importers
(Slack, Mattermost, Rocket.Chat) to always create DM groups for
1:1 messages, and removes all @override_settings decorators and
tests that only exercised the personal-recipient code path.
Some tests see query count changes from removing the redundant
get_direct_message_group lookup in get_recipient_from_user_profiles
(which now falls through directly to
get_or_create_direct_message_group), and from tests that previously
pinned PREFER_DIRECT_MESSAGE_GROUP=False now producing DM group
recipients instead of personal ones.
This commit updates the invite users email input to correctly parse and
convert email addresses with attached names into pills.
The `email-addresses` library is now used to parse email addresses in
RFC5322 format, ensuring accurate extraction of names and emails.
A new `Invitee` dataclass is introduced in the backend to store invitee
details.
Fixes part of #32874.
Co-authored-by: Maneesh Shukla <shuklamaneesh24@gmail.com>
Since commit a352f2e10, the "delivery_email" field that's set here
hasn't been used.
The default value for bot_owner is None, so we don't need to set
that here, and we can set the bot_type field directly as a param
to create_user.
In preparation for implementing automated two-tier billing on plans,
renames BillingSession.current_count_for_billed_licenses to be
current_counts_for_billed_users. The function now returns a dataclass
object, BillingUserCounts, with fields for workplace and non-workplace
users.
Currently, the non-workplace user count is hardcoded to zero for all
billing session cases, and the previously returned integer for billed
license counts is returned as the workplace user count.
This commit exists to ease the deployment of the new 1:1 direct
message groups. Specifically, with PREFER_DIRECT_MESSAGE_GROUP=False,
we now respect (and make use of) direct message groups if they are
found for a pair; however, they are never created. With
PREFER_DIRECT_MESSAGE_GROUP=True, they are created if they do not
exist already.
Regardless of the setting, the NarrowBuilder merges both personal
recipients and 1:1 DM group recipients.
This is done because setting `PREFER_DIRECT_MESSAGE_GROUP` may not
happen atomically; in large instances doing this switch without
dropping services, changing `PREFER_DIRECT_MESSAGE_GROUP` will happen
in some processes before others. As such, some processes may begin
making 1:1 DM group recipients while other processes still have
PREFER_DIRECT_MESSAGE_GROUP=False. To always show both old and new
messages, we must include both flavors of DM recipient, and we must
ratchet towards the 1:1 DM group recipients.
A subsequent commit will handle the migration of old messages, which
can be applied after this commit is deployed, and
PREFER_DIRECT_MESSAGE_GROUP=True.
Co-authored-by: Mohammad Reza Kianifar <mohammad.76kiani@gmail.com>
The PREFER_DIRECT_MESSAGE_GROUP setting controls the recipient
type for 1:1 and self DMs. It defaults to False, maintaining legacy
behavior. Tests can override this with
@override_settings(PREFER_DIRECT_MESSAGE_GROUP=True) to test the new
DirectMessageGroup recipient type behavior.
While calling `get_apns_alert_title`, we already have
the channel name and topic display name available.
Instead of doing a DB query to get channel name and
preparing topic display name again within `get_apns_alert_title`,
we pass these two values as arguments.
Signed-off-by: Prakhar Pratyush <prakhar@zulip.com>
If a user had both android and apple device registered,
preparing legacy payload resulted in executing
`get_message_payload` twice - which includes DB queries
and other operations.
This commit extracts out the function to avoid duplicate
call while preparing legacy FCM and APNs payload.
Signed-off-by: Prakhar Pratyush <prakhar@zulip.com>
To maintain API compatibility during and after the migration to direct
message groups for 1:1 and self messages, update the analytics count
query to treat DM groups of size 2 or less as private messages.
APNs tokens are provided by the client in hex, and we store them in
hex. The existing code which attempts to "validate" them by parsing
them as base64 only works because base64 is a superset of hex.
Enforce that APNs tokens are hex, and remove all of the pieces of test
code which were incorrectly passing them in as base64 strings.
In the analytics tests, where users are created as part of the set up
of the test, we want to get users by the delivery email and not the
visible-to-other-users email field.
Prep for setting default new user email visibility in realm creation
by the organization type.
Updates the labels in the"Messages set by client" chart so that
the Flutter app is no longer labeled as "beta".
The React Native app is now labeled as "old". And the older
Android and iOS apps are now labeled as "ancient".
If this is called on a user without is_mirror_dummy=True, that seems
certain to be a bug. Therefore, an assert is preferable in order to
catch this, rather than returning early with noop like some other
function such as do_deactivate_user.
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm, --test-custom-db, Debian 12 production install with custom db name and user, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:jammy, , Ubuntu 22.04 production install and PostgreSQL upgrade with pgroonga, jammy) (push) Has been cancelled
We were using admins group as a hardcoded value for the default of
`can_remove_subscribers_group`, now we use a function to get the value
of the default group.
This commit adds a new `group_size` field to the `DirectMessageGroup`
model, and backfills its value to each of the existing direct message
groups.
Fixes part of #25713
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
This commit renames the "Huddle" Django model class to
"DirectMessageGroup", while maintaining the same table --
"zerver_huddle".
Fixes part of #28640.
With `realm_active_humans` no longer dependent on the per-user rows,
there is no reason to preserve them -- any measure of "was a user
active" should look directly at the much richer RealmAuditLog. This
removes the bulk of the UserCount table, since the remaining rows all
require user interaction of some sort to produce rows.
This makes it no longer dependent on active_users_audit:is_bot:day,
which subsequent commits will make a RealmCount, not UserCount, query.
This folds the same behaviour of `active_users_audit` directly into
the query; however, only running over active users, using the index
from the earlier commit, and using the new `DISTINCT ON` formulation
make this a fast query compared to `active_users_audit:is_bot:day` +
the old `realm_active_humans::day`.
Updates the labels in the "Messages sent by client" analytics chart
for the user-agent/client names for the Flutter mobile app, which
can be "ZulipFlutter" or "ZulipMobile/flutter".
Fixes#28220.
This commit adds include_realm_default_subscriptions parameter
to the invite endpoints and the corresponding field in
PreregistrationUser and MultiuseInvite objects. This field will
be used to subscribe the new users to the default streams at the
time of account creation and not to the streams that were default
when sending the invite.
Fixes#29632.
The issue description explains this well:
We currently recalculate `currently_used_upload_space_bytes` every file
upload, by dint of calling `flush_used_upload_space_cache` on
save/delete, and then immediately calling
`user_profile.realm.currently_used_upload_space_bytes()` in
`notify_attachment_update`. Since this walks the Attachments table,
recalculating this can take seconds in large realms.
Switch this to using a CountStat, so we don't need to walk significant
chunks of the Attachment table when we upload an attachment. This will
also give us a historical daily graph of usage.
The previous implementation using Django's `get_or_create` for
`do_increment_logging_stat` involved two separate database queries,
potentially leading to race conditions.
Use an `ON CONFLICT ... DO UPDATE` (aka "upsert") query, which
eliminates race conditions and improves performance. This is mildly
complicated due to the different unique indexes across the various
tables, and the need for bug-for-bug compatibility with the previous
implementation.
Fixes#28947.
Co-authored-by: Alex Vandiver <alexmv@zulip.com>
The "invites" worker exists to do two things -- make a Confirmation
object, and send the outgoing email. Making the Confirmation object
in a background process from where the PreregistrationUser is created
temporarily leaves the PreregistrationUser in invalid state, and
results in 500's, and the user not immediately seeing the sent
invitation. That the "invites" worker also wants to create the
Confirmation object means that "resending" an invite invalidates the
URL in the previous email, which can be confusing to the user.
Moving the Confirmation creation to the same transaction solves both
of these issues, and leaves the "invites" worker with nothing to do
but send the email; as such, we remove it entirely, and use the
existing "email_senders" worker to send the invites. The volume of
invites is small enough that this will not affect other uses of that
worker.
Fixes: #21306Fixes: #24275
This prevents users from hammering the invitation endpoint, causing
races, and inviting more users than they should otherwise be allowed
to.
Doing this requires that we not raise InvitationError when we have
partially succeeded; that behaviour is left to the one callsite of
do_invite_users.
Reported by Lakshit Agarwal (@chiekosec).
Updates translated JsonableError strings that relate to streams
to use channel instead of stream. Separated from other error string
updates as this is a dense area of changes for this rename.
Part of stream to channel rename project.
Updates the labels for the "Messages sent by recipient type" chart
to use "channel", and updates the error message that would be sent
for the "messages_sent_by_stream" chart (that has not yet been
implemented) for a missing channel/stream.
Part of the stream to channel rename project.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.
Fixes part of #28640.
Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
The previous logic incorrectly used the server-level number of users
even when a (presumably smaller) realm-level count was available.
Fixes a bug introduced in 2e1ed4431a.
View functions in `analytics/views/support.py` are moved to
`corporate/views/support.py`.
Shared activity functions in `analytics/views/activity_common.py`
are moved to `corporate/lib/activity.py`, which was also renamed
from `corporate/lib/analytics.py`.
