Commit Graph

9430 Commits

Author SHA1 Message Date
Anders Kaseorg
8a35871217 events: Remove realm_is_zephyr_mirror_realm from API.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-16 11:23:10 -07:00
Anders Kaseorg
40a022dcc3 zephyr: Remove Zephyr mirroring support.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-16 11:18:18 -07:00
bedo
bc1ed82620 rate_limiter: Block IPv6 by /64 instead of per-single-IP.
Fixes #21544.

The network prefix (e.g. 64) identifies the network portion
which determines which bucket should that IPv6 belong to.

Implemented as a generic approach, so we have the option
to use different prefix sizes (leading to different buckets)
in the future.

Add comprehensive assertions for which IPv6s
are expected to belong to the same bucket and which are not.
2025-09-12 13:06:29 -07:00
Alex Vandiver
db89874bb0 cache: Write cache_key_prefixes without any compression.
This ensures that the new choice of zstd compression does not break
the previous, still-running, install during upgrades when the list of
cache_key_prefixes is updated.
2025-09-09 19:28:52 -07:00
Alex Vandiver
839135d007 users: Allow 1-character names.
Real people have 1-character names.  As seen by the previous attempt
to enforce length, in b7779667fc, real
people have names with all sorts of lengths.  All that we should care
about is that they're non-empty.
2025-09-08 14:30:59 -07:00
Evy Kassirer
31505e90f7 subscription_info: Improve subscriber fetch preformance.
Discussed here:
https://chat.zulip.org/#narrow/channel/141-kandra-ops/topic/partial.20subscribers.20query.20does.20users.20sequential.20scan/near/2255421
2025-09-08 11:47:59 -07:00
Alex Vandiver
257cfad667 outgoing_webhook: Lump chunked-encoding errors with connection errors. 2025-09-05 10:35:01 -07:00
Anders Kaseorg
1a05b2c10f tabbed-instructions: Delete activate_correct_tab.
We no longer have any OS-specific tabs; we can just activate the first
one on the server side.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-04 09:37:34 -07:00
Anders Kaseorg
35ce90d4f8 help_relative_links: Delete.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-04 09:37:34 -07:00
Shubham Padia
a766c092fc help: Restore broken link checks for help center documentation.
We add a step to build help center and then test the broken links as we
used to before removing the test temporarily.

This commit focuses on just adding back the broken link checks for the
help center. We skip the fragment check since that is in-built in
starlight and starlight tests account for that already. For the image
check we can add it back in a followup issue.
2025-09-03 09:28:15 -07:00
Shubham Padia
d333ddb961 help: Remove current help center.
This commit removes the current help center markdown files and any logic
that was used to host those files at help/.

We also remove a bunch of tests, we should the equivalent of those tests
for the new help center. Issues to track: #35649, #35647. These issues
track adding back tests for redirects and broken links.

We had a symlink from templates/zerver/integrations/email.md pointing to
help/using-zulip-via-email.md. We can no longer have that symlink since
the latter has been converted to an MDX file. We have deleted the
symlink and put a markdown file in it's place. Both the files have
comments to edit the other in case of changes.

This commit also makes changes in astro config, astro component paths
and other places to move the starlight help center docs base path from
/starlight_help to /help.

The change to rename /starlight_help/ to /help/ in MDX files is done in
the next commit. If we squash these commits, this line should be
removed.

`./tools/build-help-center` no longer does the conversion step.

We also remove some dead code related to the old help center in
documentation.py.
2025-09-03 09:28:15 -07:00
Shubham Padia
8f16f1139e help: Move URL redirects from Django to Astro.
Fixes #35649.

We do not serve the astro build via Django and thus we have to move the
help center redirects. We get a warning by pagefind on all these
redirects that they will be ignored in the search results as they don't
have an HTML body. We can ignore this warning till the cutover and
create a followup issue to solve that warning if this commit goes
through main.
2025-09-03 09:28:15 -07:00
Mateusz Mandera
6ab30fcced import: Set Stream.subscriber_count for imported channels.
The import code wasn't setting subscriber_count at all, resulting in a
value of 0 when dealing with imports from 3rd party apps.
We run this unconditionally, also for imports from Zulip, since this
ensures we won't inherit incorrect values, if the data we're import has
them - e.g. due to some bugs that affected the server the data came from.
2025-08-29 11:58:17 -07:00
Prakhar Pratyush
c604ecb902 push_notifications: Log roundtrip time since worker decided to send.
Adds a log for the end-to-end latency from when the worker decided
to send push notifications & received a success response from bouncer.

Fixes part of #35368.
2025-08-28 11:08:02 -07:00
Mohammad Reza Kianifar
e2d1c233ba get_recipient_ids: Include user ID for self-messages using DM group.
When using direct message groups as the preferred method for 1:1 or
self-messages, the get_recipient_ids function should include the
user ID for self-messages. Without this, the UI cannot detect who
was the recipient of the message.
2025-08-28 10:04:37 -07:00
Vector73
92fbe47a3b events: Add support for processing modern presence event in web client. 2025-08-27 17:16:31 -07:00
Alex Vandiver
575dd10f99 reminders: Stop interpolating user-provided values in format string.
We must not intermix Markdown strings which are ready for the message,
with format strings which we intend to interpolate on.
2025-08-27 12:07:21 -07:00
Prakhar Pratyush
5a37942c61 push_notifications: Parse push/e2ee/notify response using Pydantic.
In `send_push_notifications`, we were manually asserting types of the
fields returned by the `remotes/push/e2ee/notify` endpoint at runtime
and constructed the `response_data` data structure for further use.

Pydantic is the perfect tool for this task.

This commit updates the concerned code to use Pydantic's `TypeAdapter`.

No functional change.

Fixes part of #35368.
2025-08-27 10:58:10 -07:00
Alex Vandiver
c68be03df7 markdown: Remove vestigial Twitter code and cache.
Contrary to what the comment implied, the remaining `fetch_tweet_data`
function would never return a cache hit, as it is namespaced by the
current deploy's cache key.
2025-08-27 09:22:56 -07:00
Alex Vandiver
9715f6c104 cache: Use zstd compression level 9 for bmemcached compression. 2025-08-27 09:22:56 -07:00
Alex Vandiver
33e6d78f3b memcached: Switch to zstd compression.
This cannot go in `CACHES` because a module cannot be serialized via
pickle.
2025-08-27 09:22:56 -07:00
Alex Vandiver
52671fd486 message_cache: Skip pickling JSON-encoded string in cache. 2025-08-27 09:22:56 -07:00
Alex Vandiver
7f325108a3 message_cache: bmemcache already applies compression.
This will appear to _increase_ time spent in memcached, according to
our access logs.  This is because we used to apply zlib compression
twice -- once, outside of our performance logging, and a
second (ineffectual) time inside.  We now only compress once, which
reduces the overall time, and accounts for it more correctly.
2025-08-27 09:22:56 -07:00
Alex Vandiver
9c92fe8f34 streams: Skip pickling the recipient string in cache. 2025-08-27 09:22:56 -07:00
Alex Vandiver
0d6cfc2a63 cache: Add pickled_tupled=False to allow skipping pickling for str/bytes. 2025-08-27 09:22:56 -07:00
Mateusz Mandera
51cef01c29 message: Use .is_channel_message column instead of is_stream_message().
This avoids a potential unnecessary message.recipient fetch required by
is_stream_message(). is_stream_message() methods precedes the addition
of the denormalized is_channel_message column and is now unnecessary.

In practice, we usually fetch Message objects with `.recipient` already,
so I don't expect any notable performance impact here - but it's still a
useful change to make.
2025-08-19 23:37:07 -07:00
Mateusz Mandera
4e3ec77a0f retention: Eliminate join with Recipient table when archiving DMs.
We can use the is_channel_message column instead of doing the join to
filter on recipient type.
2025-08-19 23:36:10 -07:00
Alex Vandiver
331d210dac push_notifications: Revert parallel-device sending.
This reverts #26594, due to unexpected ConnectionClosed errors observed in Django.
2025-08-14 07:36:20 -07:00
Tim Abbott
9b2e270a49 zulip_updates: Add an update for the 11.0 release. 2025-08-13 14:48:11 -07:00
Tim Abbott
3e84eaab5d onboarding: Improve welcome bot custom message wrapper.
The previous text was rather wordy, and using a silent mention makes
it clear who has the ability to control this group, and who one can
potentially contact with questions.
2025-08-13 14:09:20 -07:00
Prakhar Pratyush
3cbf0e70a2 push_notification: Add support to send E2EE test push notification.
This commit adds an endpoint `/mobile_push/e2ee/test_notification`
to send an end-to-end encrypted test push notification to the user's
selected mobile device or all of their mobile devices.
2025-08-13 00:13:50 -07:00
Prakhar Pratyush
f034a6c3b4 push_notification: Remove is_removal param from send_push_notifications.
We can determine whether the request is meant to revoke an already
sent push notification using the "type" field of the payload.

Passing `is_removal` parameter explicitly to `send_push_notifications`
is not required.
2025-08-13 00:13:50 -07:00
Alya Abbott
a05c76fba3 updates: Edit update announcement (level=20).
Describe additional features; the prior version has not been
deployed to Cloud.
2025-08-12 17:25:42 -07:00
Vector73
f1d1d5f1a4 events: Add support for sending presence events in modern format.
If the client has passed `simplified_presence_events` as true
in the `client_capabilities` parameter of the `POST /register`
request, then the server will send `presence` events with the
`presences` field, which has the user presence data in the
modern API format. When that client capability is false, the
`presence` event will be unchanged and sent with the user
presence data in the legacy format.
2025-08-12 12:37:54 -07:00
Vector73
88761f70a2 event_schema: Rename "presence" event to "legacy_presence". 2025-08-12 12:14:56 -07:00
Sahil Batra
5675860707 channel-folders: Send event when reordering channel folders. 2025-08-12 11:17:17 -07:00
Sahil Batra
bcdfcf909d welcome-bot: Only send message with custom text when testing.
We now only send the custom message when using
"Send me a test message" button and not send the
standard welcome bot message.
2025-08-12 00:12:52 -07:00
apoorvapendse
8f14b717ff zerver: Drop support for Zulip API emails in channel creation.
Also write the description a bit better.

Discussion:
https://chat.zulip.org/#narrow/channel/378-api-design/topic/Channel.20creation.20should.20return.20the.20channel.20ID/near/2240040.

Signed-off-by: apoorvapendse <apoorvavpendse@gmail.com>
2025-08-11 23:40:12 -07:00
apoorvapendse
6203861529 zerver: API to create channel.
Some checks failed
Check feature level updated / check-feature-level-updated (push) Has been cancelled
Code scanning / CodeQL (push) Has been cancelled
Zulip production suite / Ubuntu 22.04 production build (push) Has been cancelled
Zulip CI / ${{ matrix.name }} (zulip/ci:bookworm, true, false, Debian 12 (Python 3.11, backend + documentation), bookworm) (push) Has been cancelled
Zulip CI / ${{ matrix.name }} (zulip/ci:jammy, false, true, Ubuntu 22.04 (Python 3.10, backend + frontend), jammy) (push) Has been cancelled
Zulip CI / ${{ matrix.name }} (zulip/ci:noble, false, false, Ubuntu 24.04 (Python 3.12, backend), noble) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm, --test-custom-db, Debian 12 production install with custom db name and user, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:jammy, , Ubuntu 22.04 production install and PostgreSQL upgrade with pgroonga, jammy) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:noble, , Ubuntu 24.04 production install, noble) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm-7.0, 7.0 Version Upgrade, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm-8.0, 8.0 Version Upgrade, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:jammy-6.0, 6.0 Version Upgrade, jammy) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:noble-10.0, 10.0 Version Upgrade, noble) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:noble-9.0, 9.0 Version Upgrade, noble) (push) Has been cancelled
Fixes #16206.

Co-authored-by: Sahil Batra <sahil@zulip.com>
Co-authored-by: Steve Howell <showell@zulip.com>
Signed-off-by: apoorvapendse <apoorvavpendse@gmail.com>
2025-08-08 19:29:17 -07:00
Saubhagya Patel
bb5c87e306 bots: Add a setting to customize the Welcome Bot message.
This commit includes the following changes:
- Add an administrator setting to customize the Welcome Bot
message when sending an invitation.
- Add an API endpoint to test the customized Welcome Bot message
by sending a copy of the message to the administrator.

Fixes #27663.

Co-authored-by: Akarsh Jain <akarsh.jain.790@gmail.com>
2025-08-08 18:59:37 -07:00
apoorvapendse
4d49fa6fbb streams: Extract zephyr realm invite check.
This will be reused later in the
`/channels/create` view.

Signed-off-by: apoorvapendse <apoorvavpendse@gmail.com>
2025-08-08 18:41:47 -07:00
Mateusz Mandera
254b0ff477 scim: Add support for Groups. 2025-08-08 17:29:11 -07:00
Shubham Padia
5f9860776f test_docs: Remove help specific tests and port rest over to /api.
We do not test relative links here since relative links were only
used in the old help pages. We should probably remove the logic
for relative links after we have done the official cutover to the
starlight help center: #35654.
2025-08-08 11:59:29 -07:00
Shubham Padia
9ec15e99bd test_urls: Use /api instead of /help pages for some tests.
We are going to move the help center to starlight soon and the tests
won't be able to access that help center until we solve things in a
followup issue. We move some of the urls being tested from /help to /api
to not loose coverage on those tests.
2025-08-08 11:59:29 -07:00
Shubham Padia
3e60b16ac1 starlight_help: Move help-beta over to starlight_help.
We are starting the cutover process and starlight_help is the directory
we have agreed on to place our new help center project. We do not want
to use `starlight_help` as the URL for the project, but this commit
changes the url from `help-beta` to `starlight_help` temporarily since
we can only change URL once we get rid of the current help center
project. That will be done in a future commit.
2025-08-08 11:59:29 -07:00
Alya Abbott
e712f02c91 help: Document channel folder management functionality.
Updates the left sidebar and relative settings link in both the
current and new help center documentation.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2025-08-08 07:19:57 -07:00
Tim Abbott
f287a82743 webhooks: Add nocoverage comment for unused MAPPING.
This code path isn't used yet; I expect we'll get to remove this when
we start using it for a specific integration in the near-ish future.
2025-08-07 17:39:05 -07:00
Niloth P
7022811349 integrations: Add URL option and UI for mapping messages to Zulip channels.
This commit adds a "mapping" URL option preset that adds "Matching Zulip
channel" option to the stream dropdown widget. When that option is
chosen from the dropdown, it adds another parameter to the integration
URL -- "&mapping=channels".

This "mapping" parameter is meant to be used by integrations like Slack
to identify whether the user wants to map Slack channels to different
Zulip channels or different topics within a single channel.

This adds an icon for the `mapping`s' drop down option in the "Where to
send notification" drop down field.

Co-authored-by: Pieter CK <pieterceka123@gmail.com>
Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2025-08-07 17:19:03 -07:00
PieterCK
3863f4d6a5 integrations: Add IGNORE_PRIVATE_REPOSITORIES preset WebhookUrlOption.
This refactors the `ignore_private_repositories` URL option to be a
preset because it can be used by other "version-control" integrations.
2025-08-07 17:19:03 -07:00
PieterCK
57ff908af9 webhook_common: Add a method to build preset WebhookUrlOption.
This adds `WebhookUrlOption.build_preset_config` method which builds
pre-configured WebhookUrlOptions objects. It can be used to abstract
commonly used WebhookUrlOption settings or to construct special
settings that have additional logic and UI in the web-app modal for
generating an incoming webhook URL.

Currently, one such setting is the "branches" url option. This setting
is meant to be used by "versioncontrol" integrations such as GitHub,
Gitea, etc. It adds UI that lets the user to choose which branches of
their repository can trigger notifications. So, we refactor those
integrations to use `build_preset_config` for the "branches" option.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2025-08-07 17:19:03 -07:00