Commit Graph

1611 Commits

Author SHA1 Message Date
Anders Kaseorg
6096907263 install-uv: Upgrade uv from 0.9.12 to 0.9.21.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2026-01-02 19:09:18 -08:00
Anders Kaseorg
6cd0a354ef install-node: Upgrade Node.js from 24.11.1 to 24.12.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-12-28 00:49:26 -08:00
Alex Vandiver
13243be3e8 migrations: Move schema creation and search_path into pre_migrate hook. 2025-12-16 13:03:27 -08:00
Alex Vandiver
3ee7690b0b migrations: Fix default search_path.
PostgreSQL's `search_path` sets the order in which schemas are
searched, and the first one is the one in which unqualified objects
are created[^1].

Lower in the file, we create an explicit `zulip` schema -- so we
should not use the database name in the `search_path`.  Switch to the
correct constant `zulip`.

[^1]: https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH
2025-12-16 13:03:27 -08:00
Alex Vandiver
359e5c2b78 install: Support PostgreSQL 18. 2025-12-10 21:37:07 -08:00
Alex Vandiver
1f3aed14a5 migrations: Degrade gracefully with missing remote dictionaries.
Rather than require a configuration option, detect the `tsearch_data`
error, in conjunction with a remote PostgreSQL host, and warn and
continue.  It is pretty much impossible to arrive at that combination
and then _not_ want to try again with the flag, so we should handle
that degradation gracefully, instead of requiring an explicit
install and configuration flag.
2025-12-03 11:44:14 -08:00
Alex Vandiver
f22592151a setup-certbot: Puppet guarantees the certbot package.
It has been installed on all hosts since 01e8f752a8.
2025-11-26 11:33:27 -08:00
Alex Vandiver
0c953f9870 setup-certbot: Run hooks, instead of "service nginx reload".
`certbot certonly`, when obtaining a new certificate, does not
run deploy hooks that were installed into
`/etc/letsencrypt/renewal-hooks` (certbot/certbot#9978).  Because of
this, we added an explicit `service nginx reload`.

This turns out to be awkward for a few reasons -- firstly, this does
not handle the restarting of our SMTP server, which also needs to pick
up the new certificates.  It requires that Docker installs explicitly
restart nginx their own way.  And it also means that if certbot
already had a certificate, we reload nginx twice.

We would ideally just have certbot just run all of the deploy hooks,
as usual, no matter if the certificate is new or a renewal; however,
it has no flag to do so.

We move to approximating running the deploy hooks, if the certificate
is new.  This makes it easier to have identical behaviour between
initial install and later renewals.

This requires moving the symlinking step itself to a deploy hook, so
it can run before nginx and the email server are restarted.  Our one
consumer of the `--skip-symlink` flag, Docker, can work around its
removal.  The symlink step is a no-op if `$ZULIP_DOMAIN` is unset,
i.e. during automated renewals.  This ensures that system that
installed certbot, but has since changed to manual certificate
management, does not have those certificates overridden next time
certbot renews itself.
2025-11-26 09:10:40 -08:00
Alex Vandiver
52b1bd69a9 setup-certbot: More reasonable non-interactive behaviour.
The use of `--force-interactive` was because it "suppresses a
warning", which current certbot shows no sign of displaying without
the flag.

We remove `--force-interactive`, and instead add
`--keep-until-expiring` if we are non-interactive, to allow
`setup-certbot` to default to not unnecessarily renewing certificates.
When run interactively, it may still prompt if a not-near-expiry
certificate is to be force-renewed.
2025-11-26 09:10:40 -08:00
Alex Vandiver
150b81f3ad letsencrypt: Remove danging fixes from Zulip Server 5.0.
It is no longer possible to upgrade directly from 5.0, so these steps
are no longer necessary.
2025-11-26 09:10:40 -08:00
Anders Kaseorg
9235ccfc99 install-node: Upgrade Node.js from 24.11.0 to 24.11.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-11-25 21:02:51 -08:00
Anders Kaseorg
6916c33186 install-uv: Upgrade uv from 0.9.5 to 0.9.12.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-11-25 16:29:32 -08:00
Alex Vandiver
6c9705a6bb checks: Move checkconfig to standard check framework. 2025-11-19 11:50:18 -08:00
Anders Kaseorg
35d20b04d1 install-node: Upgrade Node.js from 22.20.0 to 24.11.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-11-01 07:24:57 -07:00
Anders Kaseorg
69f2e95e49 install-uv: Upgrade uv from 0.8.22 to 0.9.5.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-10-23 15:52:47 -07:00
Alex Vandiver
a8f0cb2cf9 restart-server: All realm Tornado ports need a restart when they change. 2025-10-21 11:38:03 -07:00
Alex Vandiver
21f08265de restart-server: --tornado-reshard implies --skip-client-reloads.
The flags are marked mutually exclusive, so don't pass both; have the
former imply the latter.
2025-10-21 11:38:03 -07:00
Anders Kaseorg
4192d46a0e upgrade-zulip-from-git: Fix fully broken refname parsing.
Commit c903128eb7 (#36214) was evidently
never tested.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-10-12 16:47:53 -07:00
Alex Vandiver
c903128eb7 upgrade-zulip-from-git: Provide better error message on a bad refname.
Some checks failed
Code scanning / CodeQL (push) Has been cancelled
Zulip production suite / Ubuntu 22.04 production build (push) Has been cancelled
Zulip CI / ${{ matrix.name }} (zulip/ci:bookworm, true, false, Debian 12 (Python 3.11, backend + documentation), bookworm) (push) Has been cancelled
Zulip CI / ${{ matrix.name }} (zulip/ci:jammy, false, true, Ubuntu 22.04 (Python 3.10, backend + frontend), jammy) (push) Has been cancelled
Zulip CI / ${{ matrix.name }} (zulip/ci:noble, false, false, Ubuntu 24.04 (Python 3.12, backend), noble) (push) Has been cancelled
Zulip CI / ${{ matrix.name }} (zulip/ci:trixie, false, false, Debian 13 (Python 3.13, backend), trixie) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm, --test-custom-db, Debian 12 production install with custom db name and user, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:jammy, , Ubuntu 22.04 production install and PostgreSQL upgrade with pgroonga, jammy) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:noble, , Ubuntu 24.04 production install, noble) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:trixie, , Debian 13 production install, trixie) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm-7.0, 7.0 Version Upgrade, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm-8.0, 8.0 Version Upgrade, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:jammy-6.0, 6.0 Version Upgrade, jammy) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:noble-10.0, 10.0 Version Upgrade, noble) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:noble-9.0, 9.0 Version Upgrade, noble) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:trixie-11.0, 11.0 Version Upgrade, trixie) (push) Has been cancelled
2025-10-07 22:44:53 -07:00
Anders Kaseorg
88a7fde8bd install-node: Upgrade Node.js from 22.19.0 to 22.20.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-10-07 22:42:50 -07:00
Alex Vandiver
01664a1a10 sharding: Swap new config into place during restart-server.
This allows restart-server, before moving the new config into place,
to perform a diff and only restart the affected Tornado ports.
2025-10-07 13:19:05 -07:00
Alex Vandiver
e18b0fcd16 sharding: Use restart-server after setting up new sharding config.
restart-server has grown more complex since
efresh-sharding-and-restart was written -- use that complexity.  For
instance, workers are stopped one-at-a-time, rather than requiring
that they all stop sequentially before being started again.

It also means that Django restarts will be rolling, which trades off
500s during login for realms which are moving, in exchange for leaving
other realms unaffected.
2025-10-07 13:19:05 -07:00
Alex Vandiver
2bd4c07fdb sharding: Move chown/chmod into where the files are written. 2025-10-07 13:19:05 -07:00
Alex Vandiver
6249c2f970 sharding: Make nginx shard map owned by zulip:zulip.
This will make it easier to manipulate in restart-server.
2025-10-07 13:19:05 -07:00
Alex Vandiver
970ff73c5f restart-server: Fix comment to be sensical. 2025-10-07 09:40:53 -07:00
Anders Kaseorg
890ccec8d4 timestamp: Use localized formatting in format_datetime_to_string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-10-03 10:43:16 -07:00
Anders Kaseorg
0b400a66cc install-uv: Upgrade uv from 0.8.15 to 0.8.22.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-30 16:47:54 -07:00
Anders Kaseorg
270c50651c install-uv: Upgrade uv from 0.8.10 to 0.8.15.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-04 19:48:55 -07:00
Anders Kaseorg
fc32adaa38 install-node: Upgrade Node.js from 22.18.0 to 22.19.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-09-04 15:01:37 -07:00
Alex Vandiver
fc962b8b0e run-hooks: Handle hooks after OS upgrade on fresh install. 2025-09-03 15:20:53 -07:00
Alex Vandiver
bef864251b reload-clients: Ensure that Smokescreen does not interfere with reloads.
When run from cron, reload-server (and thus reload-clients) picks up
the `HTTP_proxy` environment variable, which redirects HTTP requests
through Smokescreen -- which prevents localhost requests.  This
results in clients never getting sent reload events.

Explicitly unset proxies when talking to localhost in reload-clients.
2025-08-19 23:39:38 -07:00
Anders Kaseorg
8de55e7cc1 install-node: Upgrade Node.js from 22.17.1 to 22.18.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-08-14 21:37:50 -07:00
Anders Kaseorg
26dc355bf5 install-uv: Upgrade uv from 0.7.21 to 0.8.10.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-08-13 23:40:33 -07:00
Anders Kaseorg
ff15d746c3 install: Support Debian 13.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2025-08-13 14:11:05 -07:00
Anders Kaseorg
fd1543a67c apt-repos: Use signed-by instead of globally trusted keys.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2025-08-13 14:11:05 -07:00
Anders Kaseorg
60fe9d438f build-pgroonga: Use msgpack-c on Debian 13.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2025-08-13 14:11:05 -07:00
Anders Kaseorg
2e97637d8e build-pgroonga: Upgrade PGroonga from 3.1.8 to 4.0.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-08-13 14:11:05 -07:00
Alex Vandiver
e132af28fc start-server: Warn on deploy mismatch if it is a noop. 2025-08-12 10:50:30 -07:00
Alex Vandiver
f6eba9c396 log-search: Add --extra filter, for [dm] and similar filters. 2025-08-12 10:25:44 -07:00
Shubham Padia
3e60b16ac1 starlight_help: Move help-beta over to starlight_help.
We are starting the cutover process and starlight_help is the directory
we have agreed on to place our new help center project. We do not want
to use `starlight_help` as the URL for the project, but this commit
changes the url from `help-beta` to `starlight_help` temporarily since
we can only change URL once we get rid of the current help center
project. That will be done in a future commit.
2025-08-08 11:59:29 -07:00
Alex Vandiver
fd390b9eb1 settings: Enable Altcha in development if on localhost.
Some checks failed
Code scanning / CodeQL (push) Has been cancelled
Zulip production suite / Ubuntu 22.04 production build (push) Has been cancelled
Zulip CI / ${{ matrix.name }} (zulip/ci:bookworm, true, false, Debian 12 (Python 3.11, backend + documentation), bookworm) (push) Has been cancelled
Zulip CI / ${{ matrix.name }} (zulip/ci:jammy, false, true, Ubuntu 22.04 (Python 3.10, backend + frontend), jammy) (push) Has been cancelled
Zulip CI / ${{ matrix.name }} (zulip/ci:noble, false, false, Ubuntu 24.04 (Python 3.12, backend), noble) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm, --test-custom-db, Debian 12 production install with custom db name and user, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:jammy, , Ubuntu 22.04 production install and PostgreSQL upgrade with pgroonga, jammy) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:noble, , Ubuntu 24.04 production install, noble) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm-7.0, 7.0 Version Upgrade, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:bookworm-8.0, 8.0 Version Upgrade, bookworm) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:jammy-6.0, 6.0 Version Upgrade, jammy) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:noble-10.0, 10.0 Version Upgrade, noble) (push) Has been cancelled
Zulip production suite / ${{ matrix.name }} (zulip/ci:noble-9.0, 9.0 Version Upgrade, noble) (push) Has been cancelled
We only do this on localhost, because users of remote development
servers will run afoul of "Secure context is required"[^1].

[^1]: https://altcha.org/docs/troubleshooting/#secure-context
2025-07-25 22:29:51 -07:00
Anders Kaseorg
c911dcf279 install-node: Upgrade Node.js from 22.17.0 to 22.17.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-22 14:19:56 -07:00
Alex Vandiver
43034a79b6 run_hooks: Make post-deploy hooks be best-effort. 2025-07-16 18:00:47 -07:00
Alex Vandiver
e9a2ee56c3 log-search: Add --stats flag. 2025-07-16 09:22:02 -07:00
Alex Vandiver
2616b7d030 log-search: Add a missing new filter flag to -L. 2025-07-16 09:22:02 -07:00
Alex Vandiver
6d80d4c77f restart-server: Skip weekly restart-server if a deploy is in process.
This is unlikely, but cron running `restart-server` at the random
wrong time mid-deploy could be bad.
2025-07-15 09:07:36 -07:00
Anders Kaseorg
996eb72e2a install-uv: Upgrade uv from 0.7.15 to 0.7.21.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-07-15 07:32:44 -07:00
Prakhar Pratyush
8b3cef554b settings: Add push_registration_encryption_keys map.
The `push_registration_encryption_keys` map stores the
assymetric key pair generated on bouncer.

The public key will be used by the client to encrypt
registration data and the bouncer will use the corresponding
private key to decrypt.

- Updated the `generate_secrets.py` script to generate the map
during installation in dev environment.
- Added a management command to add / remove key i.e. use it
for key rotation while retaining the older key-pair for a period
of time.
2025-07-06 21:11:26 -07:00
Anders Kaseorg
7959a1853c install-node: Upgrade Node.js from 22.16.0 to 22.17.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-06-25 16:53:09 -07:00
Anders Kaseorg
9f8f6e60d9 install-uv: Upgrade uv from 0.7.11 to 0.7.15.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2025-06-25 16:52:39 -07:00