Commit Graph

4695 Commits

Author SHA1 Message Date
Zixuan James Li
b3f27b1843 mypy: Enable local_partial_types.
dmypy is innately incompatible with `local_partial_types` being
`False` as it uses fined-grained incremental mode for caching.

We need to enable `local_partial_types` for mypy as well so that the two
will behave the same way.

This requires us to add additional type annotation in certain siuations
but that's fine.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-06 17:33:13 -07:00
Zixuan James Li
bf9f9c8b5d tools: Support running mypy daemon for better performance.
mypy daemon performs significantly better than running the regular
mypy cli tool when we type check the entire codebase multiple
times locally.

This adds running mypy daemon as an option for both
`tools/run-mypy` and `tools/lint`.

To ensure daemon messages like "Daemon started", "Daemon stopped"
won't get printed we filter any output that starts with "Daemon".

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-06 17:33:13 -07:00
Zixuan James Li
ea3478e377 tools: Extract ROOT_DIR.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-06 17:33:13 -07:00
Anders Kaseorg
e4e51b0eb4 install-shfmt: Upgrade shfmt from 3.5.0 to 3.5.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-06 17:23:16 -07:00
Anders Kaseorg
feff1d0411 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-05 17:54:17 -07:00
Alex Vandiver
aabbb2ad10 bootstrap-awscli: Leave CWD unchanged. 2022-06-28 09:39:31 -04:00
Alex Vandiver
1b5e058d3b install-aws-server: Add a --debug-key to allow logins during bootstrap. 2022-06-28 09:39:31 -04:00
Alex Vandiver
1a1061e77e install-aws-server: Switch to getopt. 2022-06-28 09:39:31 -04:00
Alex Vandiver
1be9ab2690 install-aws-server: Assume zulip_ops::profile:: prefix on all roles.
This will require that any profile-specific sections of
`$HOME/.zulip-install-server.conf` be renamed to their short form.
2022-06-28 09:39:31 -04:00
Alex Vandiver
9a3be2af64 install-aws-server: Fix default_branch config name. 2022-06-28 09:39:31 -04:00
Alex Vandiver
f5d4dea2f0 bootstrap-aws-installer: Leave ubuntu user deletion until the last.
This makes it easier to log in and figure out what went wrong if
something failed during the bootstrapping process.
2022-06-28 09:39:31 -04:00
Anders Kaseorg
ef3510fa6d nginx: Remove legacy X-XSS-Protection header.
Support for this header was removed in Chrome 78, Safari 15.4, and
Edge 17.  It was never supported in Firefox.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 17:38:18 -07:00
Anders Kaseorg
25f6b53e90 docs: Capitalize Handlebars consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 13:35:10 -07:00
Anders Kaseorg
d5c302feef run-dev: Remove unnecessary finish calls.
Tornado finishes the request automatically.  Avoids this error, hidden
until commit 81f7192ca3 (#22301), when
the browser tab is closed:

Traceback (most recent call last):
  File "/srv/zulip-py3-venv/lib/python3.8/site-packages/tornado/web.py", line 1683, in _execute
    result = await result
  File "tools/run-dev.py", line 280, in prepare
    await self.finish()
tornado.iostream.StreamClosedError: Stream is closed

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:38:43 -07:00
Anders Kaseorg
0dfde9db94 python: Simplify unnecessary getattr calls.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:38:22 -07:00
Anders Kaseorg
3bf8ee2156 python: Unquote some unnecessarily quoted type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:37:41 -07:00
Anders Kaseorg
e069330084 check-thirdparty: License under GPLv2+.
Since python-debian is GPL licensed, our script that imports it should
arguably be GPL licensed as well.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:33:11 -07:00
Alex Vandiver
6cd4d907ba upload-release: Only copy full releases to zulip-server-latest.tar.gz.
`zulip-server-6.0-pre0.tar.gz` should not be copied to
`zulip-server-latest.tar.gz`.
2022-06-23 19:24:54 -07:00
Anders Kaseorg
81f7192ca3 tornado: Add missing await for finish calls.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-23 19:22:12 -07:00
Anders Kaseorg
c944adfcc6 mypy: Enable redundant-expr errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-23 19:22:12 -07:00
Alex Vandiver
41deef40cf nagios: Switch to generic check_cron_file for queues and consumers.
These share a common root; 91da4bd59b duplicated the code, but
didn't move the existing uses to the new utility.
2022-06-22 12:07:38 -07:00
Alex Vandiver
91379fd67e ci: Update upgrade test to 5.3, from 5.2. 2022-06-21 17:40:33 -07:00
Alex Vandiver
a35af3f38b install/upgrade: Allow new packages during apt-get upgrade.
`postgresql-14.4` is a notable upgrade in the PostgreSQL series, as it
fixes potential database corruption from `CREATE INDEX CONCURRENTLY`
statements which are run while rows are modified[1].  However, it also
requires an upgrade from `libllvm9` to `libllvm10`, which means it is
not installed by a mere `apt-get upgrade`.

Add the `--with-new-pkgs` flag to all of the potentially relevant
`apt-get upgrade` calls, so that this (and similar) packages are
upgraded successfully.

[1]: https://www.postgresql.org/docs/release/14.4/
2022-06-21 11:21:49 -07:00
Anders Kaseorg
a7f9c4f958 logging: Pass more format arguments to logging.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-03 12:27:23 -07:00
Anders Kaseorg
79842fd228 install-shfmt: Upgrade shfmt from 3.4.3 to 3.5.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-02 12:03:49 -07:00
Adam Sah
ba5cf331a2 testing: 100% coverage for zerver/tests/test_digest.py. 2022-06-01 16:09:13 -07:00
Anders Kaseorg
ecb900bd03 requirements: Update zulint with multiline pattern support.
https://github.com/zulip/zulint/pull/28

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-01 14:34:58 -07:00
Zixuan James Li
63e9ae8389 typing: Apply trivial fixes to adjust edge cases in typing.
Add none-checks, rename variables (to avoid redefinition of
the same variable with different types error), add necessary
type annotations.

This is a part of #18777.

Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-30 12:03:51 -07:00
Alex Vandiver
fb338f22d7 ci: Stop rabbitmq when making CI images.
The production CI image starts `rabbitmq-server` but does not stop it,
which leaves a stale `/var/run/rabbitmq/pid` file in the image.

`rabbitmqctl wait --timeout 600 /var/run/rabbitmq/pid`, which is run
after starting the rabbitmq node, reads the PID file and waits for the
PID to be running, and for rabbitmq's port to be responding to pings.

If it reads an old PID file before the new PID is written, it
aborts (all but the first and last lines are output from `rabbitmqctl
wait` that is hidden by `/etc/init.d/rabbitmq-server`):

```
 * Starting RabbitMQ Messaging Server rabbitmq-server
Waiting for pid file '/var/run/rabbitmq/pid' to appear
pid is 341
Waiting for erlang distribution on node 'rabbit@fc8f64d6acdb' while OS process '341' is running
Error:
process_not_running
 * FAILED - check /var/log/rabbitmq/startup_\{log, _err\}
 ```

If it failed, the `production-upgrade` script tried to start
`rabbitmq` again -- despite it already still starting in the
background.  These two attempts conflicted, and often one or both
failed.

Stop `rabbitmq-server` when building the image, which removes the
stale PID file.
2022-05-19 13:30:59 -07:00
Adam Sah
04ccd8c6d8 testing: 100% code coverage for zerver/lib/outgoing_webhook.py. 2022-05-17 13:51:00 -07:00
Alex Vandiver
c93024cd5b oneclick: Fail if the fab command fails. 2022-05-17 13:41:12 -07:00
Alex Vandiver
ff647dff03 oneclick: Do not use a stale Zulip client.
Initializing the Zulip client opens a long-lived TCP connection due to
connection pooling in urllib3.  In Github Actions, the network kills
such requests after ~270s, making the later `send_message` call fail.

Use a singular call to `zulip.Client()` early on to verify the
credentials, and do not cache the resulting client object.  Instead,
re-create it during the final step when it is needed, so we do not run
afoul of bad TCP connection state.

This would ideally be fixed via connection keepalive or retry at the
level of the Zulip module.
2022-05-17 13:41:12 -07:00
Adam Sah
534754442a test-backend: Improve performance by disabling XML report.
We don't use the XML report ourselves. We add options to make this
easy to control if specific circumstances indicate doing so.
2022-05-16 16:38:02 -07:00
Adam Sah
492272d597 test-backend: Display test function in HTML coverage reports. 2022-05-16 16:38:02 -07:00
Alex Vandiver
9e42cf435b tail-ses: Add a tool to tail SES delivery/bounce/complaint queues. 2022-05-06 17:49:00 -07:00
Ganesh Pawar
59e810790f user_groups: Convert inline form to modal.
Fixes part of #21298.
2022-05-04 17:46:30 -07:00
Raghav Luthra
fb89c14f34 lint: Improve data-tippy-allowHTML error message.
The error message a user gets from the linter when using the
data-tippy-allowHTML attribute now conveys the fact that the
<template> tag is supposed to hold the tooltip content. This
might make understanding the correct workflow easier for
someone who encounters this error.
2022-05-04 17:38:30 -07:00
Adam Sah
a9ce02348b testing: Improve backend code coverage for semaphore webhook. 2022-05-04 16:13:24 -07:00
Alex Vandiver
bf562f8fff ci: Update upgrade test to 5.2, from 5.1. 2022-05-04 11:37:15 -07:00
Tim Abbott
25999b7056 lint: Ban Help Center relative links without leading /.
Such links would likely end up broken if we change our URL scheme to
no longer be based around URL fragments.
2022-05-04 11:18:27 -07:00
patricklauer
d86b7c4ef7
provision: Add missing ENDC to avoid colouring all output. 2022-05-03 12:41:43 -07:00
Anders Kaseorg
e952641013 install: Resupport Ubuntu 22.04.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-03 09:41:08 -07:00
Anders Kaseorg
7acb642fa5 requirements: Upgrade to Tornado 6.
Fixes #8913.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg
f23bfe91c0 run-dev: Fix types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg
c9faefd50e cache: Instantiate only one BMemcached cache backend.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg
0ef9309e92 run-dev: Switch to asyncio event loop.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg
bded7180f7 tornado: Unfork tornado.autoreload.
We previously forked tornado.autoreload to work around a problem where
it would crash if you introduce a syntax error and not recover if you
fix it (https://github.com/tornadoweb/tornado/issues/2398).

A much more maintainable workaround for that issue, at least in
current Tornado, is to use tornado.autoreload as the main module.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg
1c7954b452 run-dev: Avoid deprecated tornado.gen.engine.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg
080a806d60 build-pgroonga: Update PGroonga to 2.3.6.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-29 16:02:45 -07:00
Tim Abbott
cd011b1a94 ci: Verify production servers using /api/v1/server_settings.
The previous commit changed our response for the root domain on a
self-hosted Zulip server containing no realms to be a 404.

This made the HTTP headers verification done here less robust than
would be ideal, in that one could imagine a bug that makes every URL
404 passing that test.

So we instead change the check to verify the /server_settings API,
which doesn't have that problem.
2022-04-28 12:34:59 -07:00