Commit Graph

65 Commits

Author SHA1 Message Date
Anders Kaseorg
dbdf67301b memcached: Switch from pylibmc to python-binary-memcached.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-06 12:51:14 -07:00
Priyank Patel
9504d403b3 github-actions: Fix incorrect hash key in production install job.
The hash keys were missing hash for package.json and yarn.lock
because they were not present since we don't do a full checkout
in this job. We fix this by sending over those files and generating
hashes from them.

I usally verify these cache keys by clicking the Restore <cache>
step dropdown menu and then clicking the Run ... dropdown menu again
to see the generated hash.
2020-08-03 12:37:20 -07:00
Priyank Patel
dda7e938f4 puppeteer: Fix flake with compose box input.
There were a lots of flakes in CI recently because typeahead didn't
appear when Enter was pressed and real emails are not accepted as
valid inputs. To fix this we wait for typeahead to appear and then
click that instead of Enter. We also use delay option to type the
email (100ms delay between keypresses) since without we'd also get
flakes.

Re-enable puppeteer test in CI after this fix too.
2020-07-30 11:32:32 -07:00
Tim Abbott
cee7e8e89d ci: Disable puppeteer tests. 2020-07-28 17:12:49 -07:00
Steve Howell
da8691d7d7 ci: Add new check-node-fixtures tool to CI.
This tool only takes a couple seconds to run,
so it's pretty low risk to add to CI.
2020-07-26 16:07:40 -07:00
Priyank Patel
51199878ed tools/ci: Fix KeyError when running tools/ci/generate-dockerfiles.
This KeyError occurs because we remove extra_packages from
images.yml because it was no longer needed in a486872a8e.
2020-07-26 16:03:35 -07:00
Priyank Patel
6a4792e844 tools/ci: Adjust ZULIP_PATH so it works for both CIs.
Until, we remove the circleci.
2020-07-24 10:36:55 -07:00
Priyank Patel
75ab26771b ci: Store production built assets to /tmp/production-build.
Previously, we copied them to /tmp and from there we specified those
assets we copied in circleci config in presist_to_workspace step.
Copying it to a directory allows us to get rid of list in circleci
config and GitHub Actions's upload artifact (their version of
presist to workspace) doesn't allow us to specify indivivual files
so only is this cleaner but required.
2020-07-24 10:36:55 -07:00
Mohit Gupta
308643cace tests: Rename test_narrow.py to test_message_fetch.py.
This commit renames test_narrow.py and also all its references to
test_message_fetch.py.
2020-07-07 17:14:10 -07:00
arpit551
4f6cd6577c http_headers: Substitute Nginx version based on OS.
success-http-headers-bionic.txt and success-http-headers-focal.txt
differ only in the nginx version so this substitution will allow
us to have single file for both of them. Also this change helps
to avoid CI failure if Nginx version is updated in the OS.
2020-07-07 11:20:05 -07:00
Priyank Patel
29dd0c485a ci: Move upgrading git for bionic to docker image.
Just reduces the clutter in configuration file for GitHub Actions.
2020-07-06 18:09:15 -07:00
Priyank Patel
c908278237 ci: Install moreutils in the docker image. 2020-07-06 18:09:15 -07:00
Aman Agrawal
a486872a8e requirements: Upgrade Thumbor to 7.0.0a5 on Python 3.
Co-authored-by: Anders Kaseorg <anders@zulip.com>
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 16:09:53 -07:00
arpit551
834abefdb9 http_headers: Update Nginx version in success-http-headers-focal. 2020-07-02 22:42:22 +05:30
Anders Kaseorg
fa89d1b266 generate-self-signed-cert: Correct subjectAltName for an IP address.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-29 22:19:47 -07:00
Alex Vandiver
2540b678b0 ci: Test upgrading PostgreSQL 10 -> 12. 2020-06-29 17:18:47 -07:00
Alex Vandiver
f621e1b7e4 ci: Split out installation testing. 2020-06-29 17:18:47 -07:00
Alex Vandiver
36335ecd39 ci: Simplify tool-copying in production-build. 2020-06-29 17:18:47 -07:00
Priyank Patel
92f98292c1 tools/ci: Create a github user instead of a circleci user.
While we are making this change, we create a ARG USERNAME variable
that behaves like ENV but just for the lifetime of the Dockerfile.
2020-06-29 16:31:43 -07:00
Priyank Patel
a2c2c6ceff tools: Move CI docker images files into tools/ci. 2020-06-29 16:31:43 -07:00
Aman Agrawal
6950d8d769 provision: Don't run migrations on zulip db in CircleCI.
The automated tests running in CircleCI don't actually use the `zulip`
db, so we can skip running migrations on it in some CircleCI shards to
save time.

NOTE: This only effects build jobs that run provision, except the
`production-build` job where we skip building the dbs altogether.
Migrations still run on `focal-backend` build job to ensure
we are testing all our development setup code.
2020-06-16 15:37:32 +05:30
Anders Kaseorg
80b991fc44 production-helper: Clarify log output on check_rabbitmq_consumers failure.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 16:49:32 -07:00
arpit551
662f4902f8 ci: Setup production job for Focal.
Run production suites on Ubuntu Focal.
Added separate success-http-headers files for Focal and Bionic.
Also excluded them from whitespace rules in lint.

memcached 1.5.22 in Ubuntu 20.04 has a bug where it looks for its SASL
configuration at /etc/sasl2/memcached.conf/memcached.conf instead of
/etc/sasl2/memcached.conf.
We already use a workaround for this while applying puppet configurations in
99e71f3786 but for docker builds we used
do memcached hack since we can not use systemd in docker containers.
2020-06-08 21:59:57 -07:00
arpit551
291bce93d0 provision: Rename --production-test-suite option in provision.
Since we use this option in our docker-zulip project also
so rather than using it as a test suite option we made it
more specific i.e. --build-release-tarball-only.
2020-06-04 14:26:23 -07:00
Dinesh
d429b7145e tests: Make puppeteer tests run in full suites and CircleCI. 2020-06-03 17:39:35 -07:00
Aman Agrawal
c9bff71410 ci: Don't remove .coverage file & set empty suffix.
Fixes #14962
* codecov needs `.coverage` file in the pwd to upload coverage
results.
* concurrency='multiprocessing' forces `.coverage` file to have a
data_suffix, we explicity set it to "" for the file to have no suffix.
Filed issue as https://github.com/nedbat/coveragepy/issues/989
2020-05-27 17:54:54 -07:00
Aman Agrawal
1a8798d9c5 ci: Disable casper tests in ci due to flaky behaviour. 2020-05-13 14:02:06 -07:00
Aman Agrawal
3442182d7b ci: Use absolute path to zulip folder.
Strangely focal resolves `~/zulip` as `/root/zulip`, this makes
sure we are in the correct folder.
2020-04-29 10:54:53 -07:00
Tim Abbott
28cb0aa81a docs: Use consistent spelling of CircleCI. 2020-04-28 11:26:58 -07:00
Aman Agrawal
1324e7798c ci: Directly run production install script. 2020-04-28 11:11:00 -07:00
Aman Agrawal
b5e31c04c9 ci: Rename production install script. 2020-04-28 11:11:00 -07:00
Aman Agrawal
70f7d04bf1 ci: Rename production builder script file. 2020-04-28 11:11:00 -07:00
Aman Agrawal
cf14cc211f ci: Separate prod tarball extraction and install process.
This allows to run scripts between extraction and install
process.

It will be used to restore npm caches for production install jobs.

We extract the tarball in the working directory so that yarn.lock and
package.json are available to restore cache.

(And also so the path is deterministic).
2020-04-25 16:10:41 -07:00
Aman Agrawal
0aa18e88a5 ci: Stop cloning repo as it is not required for prod install.
Move prod installer and test files to persistent dir so that
they can be utilized.
2020-04-25 16:07:55 -07:00
Aman Agrawal
0e544ec40b ci: Add job to run production script to tests production install.
Only restore npm cache, venv cache not required.
Restart rabbitmq-server for production install.
2020-04-24 12:40:31 -07:00
Aman Agrawal
7c99555107 ci: Copy https headers to persistent dir for comparison. 2020-04-24 12:40:31 -07:00
Aman Agrawal
b7c50034b8 ci: Remove unnecessary setup-production steps.
This steps won't be used since production install will run in
a separate job.
2020-04-24 12:40:26 -07:00
Aman Agrawal
1a096a9a10 ci: Move built production tarball to /tmp/ directory.
This directory will persist across build jobs.
2020-04-24 12:40:20 -07:00
Pranav
93b09d44d6 ci: Add test to check for untracked files.
Runs a test at the end of tools/ci/backend to check if any untracked
files have been created during this ci tests. Exits with error code 1
if untracked files are found, otherwise exits successfully.

Fixes #14691.
2020-04-24 12:35:47 -07:00
Puneeth Chaganti
d5f5a99b07 tools: Verify that all integrations have bot avatars. 2020-04-22 17:45:30 -07:00
arpit551
aabad1c0bb test: run test-locked-requirements only for Bionic.
Currently our compiled requirements files will differ for different
python versions. So we will run test-locked-requirements only for Bionic.
2020-04-22 16:10:59 -07:00
arpit551
56a5beddc0 ci: Fixed errors related to running services in ci production build.
Restart postgres service if provision is called in production test suite.
This is required because terminate-psql-sessions script (used
in tools/ci/setup-production) throws error if postgres service is not running.

Restart rabbitmq service if provision is called in production test suite.
This is done to start the node as Circle CI don't start services on installation.

Removed memcached restart as flush-memcached script (which is furthur
used in tools/ci/production) throws UNKNOWN READ FAILURE if memcached is restarted
in development.
2020-04-21 14:46:40 -07:00
arpit551
e6edf469ee ci: Remove the need of using TRAVIS in env.
Since now we want to use production suites on Circle CI so there
is no need to set TRAVIS in env while running scripts.

CIRCLECI is set default in the enviroment of Circle CI builds
so we can use it directly.

Also Travis CI had rabbitmq-server installed so we had to add workaround
in install script to avoid the error. That workaround is removed.
2020-04-21 14:46:40 -07:00
arpit551
4946405e3a ci: Change production setup in ci according to bionic and Circle CI.
Used postgres 10 inplace of postgres 9.5 as it is used in Bionic.

Upgraded nginx version in success-http-headers which is in CircleCI
Bionic enviroment.
2020-04-21 14:46:40 -07:00
arpit551
dee8de3b88 ci: Replaced travis with test suite while setting production setup on ci. 2020-04-21 14:46:40 -07:00
arpit551
80af201ef9 ci: Removed hacky code required for travis CI.
Removed hacky code required for travis CI in production-helper
and setup production.
2020-04-21 14:46:40 -07:00
Anders Kaseorg
15d68c40dd nginx: Set X-XSS-Protection: 1; mode=block.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-05 16:13:53 -07:00
Anders Kaseorg
79c215626e nginx: Set X-Content-Type-Options: nosniff globally.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-05 16:13:53 -07:00
Stefan Weil
d2fa058cc1
text: Fix some typos (most of them found and fixed by codespell).
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2020-03-27 17:25:56 -07:00
Anders Kaseorg
006a69c912 ci: Disable test-run-dev.
This test doesn’t catch many bugs and is known to be flaky.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 21:38:19 -07:00