Commit Graph

2124 Commits

Author SHA1 Message Date
PIG208
49eab4efef Revert "request: Refactor to record rate limit data using ZulipRequestNotes."
This reverts commit 3f9a5e1e17.
2021-07-16 09:01:20 -07:00
Alex Vandiver
928dc4bafd sentry: Set environment from machine.deploy_type config.
This allows for greater flexibility in values for "environment," and
avoids having to have duplicate definitions of STAGING in
`zproject/config.py` and `zproject/default_settings.py` (due to import
order restrictions).

It does overload the "deploy type" concept somewhat.

Follow-up to #19185.
2021-07-15 15:01:43 -07:00
Alex Vandiver
3e57d66632 sentry: Drop unused integrations argument. 2021-07-15 15:01:43 -07:00
PIG208
c03b9c95ad request: Store client information using ZulipRequestNotes.
This concludes the HttpRequest migration to eliminate arbitrary
attributes (except private ones that are belong to django) attached
to the request object during runtime and migrated them to a
separate data structure dedicated for the purpose of adding
information (so called notes) to a HttpRequest.
2021-07-14 12:01:07 -07:00
PIG208
8eb2c3ffdb request: Move realm from the request to ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208
3f9a5e1e17 request: Refactor to record rate limit data using ZulipRequestNotes.
We will no longer use the HttpRequest to store the rate limit data.
Using ZulipRequestNotes, we can access rate_limit and ratelimits_applied
with type hints support. We also save the process of initializing
ratelimits_applied by giving it a default value.
2021-07-14 12:01:07 -07:00
Tim Abbott
0bf13ce5db portico: Add Technical University of Munich case study.
This isn't linked yet, since the main place we'll link to is isn't
added yet.
2021-07-09 13:04:38 -07:00
Alex Vandiver
a985284b5d sentry: Set environment to staging on staging hosts.
Splitting exceptions out between staging and production provides
useful filtering.
2021-07-09 10:54:19 -07:00
Mateusz Mandera
85cbdc8904 rate_limit: Add rate limiting of ZulipRemoteServer. 2021-07-08 15:55:02 -07:00
Mateusz Mandera
b9056d193d rate_limit: Implement IP-based rate limiting.
If the user is logged in, we'll stick to rate limiting by the
UserProfile. In case of requests without authentication, we'll apply the
same limits but to the IP address.
2021-07-08 15:46:52 -07:00
Mateusz Mandera
3b4f8cc85b saml: Add setting to skip the "continue to registration" page.
It's a smoother Just-In-Time provisioning process to allow
creating the account and getting signed in on the first login by the
user.
2021-07-08 15:21:20 -07:00
Anders Kaseorg
98194b9316 blueslip: Remove SAVE_FRONTEND_STACKTRACES setting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-07 09:26:02 -07:00
Anders Kaseorg
07fef56c74 logging_handlers: Remove STAGING_ERROR_NOTIFICATIONS setting.
Running notify_server_error directly from the logging handler can lead
to database queries running in a random context.  Among the many
potential problems that could cause, one actual problem is a
SynchronousOnlyOperation exception when running in an asyncio event
loop.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-07 09:26:02 -07:00
Mateusz Mandera
cee4da64fa saml: Don't raise AssertionError if no name is provided in SAMLResponse.
This is an acceptable edge case for SAML and shouldn't raise any errors.
2021-07-07 09:23:21 -07:00
Anders Kaseorg
ee8724e436 docs: Correct Big Blue Button to BigBlueButton.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 16:59:00 -07:00
Anders Kaseorg
37f98fdec8 zproject: Add prod_settings mypy stub, aliasing prod_settings_template.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 09:53:41 -07:00
Damian Parrino
caea7f167d help: Reorganize articles on editing topics.
* Move content on moving topics between streams to a dedicated
  article. We advertise it as "move content" to hint that one can move
  messages or split topics, and link to it.

* This deletes change-the-topic-of-a-message, because the same content
  is already covered in rename-a-topic.

* This commit mostly just moves content between articles. Most of that
  content was redundant with the first few paragraphs of the surviving
  "rename a topic" article. The former "This is useful for" se ntence
  was adapted to the remaining article.

* This commit also adds a redirect for the removed article, and
  updates related links.
2021-06-29 13:52:06 -07:00
Tim Abbott
06121e1f69 urls: Add redirect for community_topic_edits rename. 2021-06-16 15:07:16 -07:00
Mateusz Mandera
b1da5d9d76 ldap: Prevent creation of users with malformed email value. 2021-06-11 17:30:46 -07:00
Suyash Vardhan Mathur
001435a297 refactor: Rename and move app_filters.py.
Moved the code in zerver/templatetags/app_filters.py
to more intuitive location and name at
zerver/lib/templates.py.
2021-06-11 07:43:22 -07:00
Mateusz Mandera
c54b48452d saml: Support syncing custom profile fields with SAML attributes.
Fixes #17277.

The main limitation of this implementation is that the sync happens if
the user authing already exists. This means that a new user going
through the sign up flow will not have their custom fields synced upon
finishing it. The fields will get synced on their consecutive log in via
SAML in the future. This can be addressed in the future by moving the
syncing code further down the codepaths to login_or_register_remote_user
and plumbing the data through to the user creation process.

We detail that limitation in the documentation.
2021-06-08 14:47:33 -07:00
Mateusz Mandera
00c7ac15df saml: Define a TypedDict for the type of IdP settings dict.
The old type in default_settings wasn't right - limit_to_subdomains is a
List[str]. We define a TypeDict for capturing the typing of the settings
dict more correctly and to allow future addition of configurable
attributes of other non-str types.
2021-06-08 14:30:21 -07:00
Anders Kaseorg
61e1e38a00 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-07 17:57:51 -07:00
Gaurav Pandey
9b696cf212 api: Expose event_queue_longpoll_timeout_seconds in /register.
Rename poll_timeout to event_queue_longpoll_timeout_seconds
and change its value from 90000 ms to 90 sec. Expose its
value in register api response when realm data is fetched.
Bump API_FEATURE_LEVEL to 74.
2021-06-05 07:37:19 -07:00
akshatdalton
7df62ebbaf settings: Make MAX_MESSAGE_LENGTH a server-level setting.
This will offer users who are self-hosting to adjust
this value. Moreover, this will help to reduce the
overall time taken to test `test_markdown.py` (since
this can be now overridden with `override_settings`
Django decorator).

This is done as a prep commit for #18641.
2021-06-03 09:26:28 -07:00
Gaurav Pandey
f67fc37f7d settings: Rename MAX_EMOJI_FILE_SIZE.
Rename MAX_EMOJI_FILE_SIZE to
MAX_EMOJI_FILE_SIZE_MIB reflecting
the size in mebibytes.
2021-06-02 13:55:15 -07:00
Gaurav Pandey
73059dd0c7 settings: Rename MAX_LOGO_FILE_SIZE.
Rename MAX_LOGO_FILE_SIZE to
MAX_LOGO_FILE_SIZE_MIB reflecting
the size in mebibytes.
2021-06-02 13:55:15 -07:00
Gaurav Pandey
2ddc62e9b3 settings: Rename MAX_ICON_FILE_SIZE.
Rename MAX_ICON_FILE_SIZE to
MAX_ICON_FILE_SIZE_MIB reflecting
size in mebibytes.
2021-06-02 13:55:15 -07:00
Gaurav Pandey
0f6bb210a6 settings: Rename MAX_AVATAR_FILE_SIZE.
Rename MAX_AVATAR_FILE_SIZE to
MAX_AVATAR_FILE_SIZE_MIB reflecting
size in mebibytes.
2021-06-02 13:55:15 -07:00
Mateusz Mandera
b202116594 auth: Extract sync_user_profile_custom_fields function from ldap sync.
This logic will be reusable for analogical sync in other auth codepaths.
2021-06-01 08:32:50 -07:00
Mateusz Mandera
47a4a9193f auth: Make full_name_validated attr of generic OIDC configurable.
This will be useful for deployments that want to just use the full name
provided by the IdP and thus skip the registration form. Also in
combination with disabling name changes in the organization, can force
users to just use that name without being able to change it.
2021-05-28 09:50:23 -07:00
Mateusz Mandera
e17758f8ad auth: Implement a generic OpenID Connect backend.
Fixes #11939.
2021-05-28 09:43:06 -07:00
Alex Vandiver
54c222d3f8 settings: Support arbitrary database user and dbname.
This adds basic support for `postgresql.database_user` and
`postgresql.database_name` settings in `zulip.conf`; the defaults if
unspecified are left as `zulip`.

Co-authored-by: Adam Birds <adam.birds@adbwebdesigns.co.uk>
2021-05-25 13:46:58 -07:00
Mateusz Mandera
1df725e6f1 settings: Silence CryptographyDeprecationWarning spam from a dependency. 2021-05-23 13:31:55 -07:00
Mateusz Mandera
cd88a6e6fb tests: Rename APPLE_JWK to a generic name for re-use in other tests.
This can be re-used for the JWK in OIDC implementation in the follow-up
commits.
2021-05-23 13:30:17 -07:00
Alex Vandiver
670c7e7ba4 settings: Remove now-unnecessary EMAIL_DELIVERER_DISABLED setting. 2021-05-18 12:39:28 -07:00
Alex Vandiver
82797dd53c settings: Standardize the name of the deliver_scheduled_messages logs.
This makes it match its command name, and other logfile name.
2021-05-18 12:39:28 -07:00
Anders Kaseorg
79c087471c jinja2: Serialize page-params with orjson.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-18 12:36:15 -07:00
Anders Kaseorg
e015f3ed7d docs: Correct “webapp” to “web app”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-14 13:15:38 -07:00
Vishnu KS
051dab58ea navbar: Add gear menu advertisement for sponsoring zulip. 2021-05-12 10:21:48 -07:00
Alex Vandiver
0f1611286d management: Rename the deliver_email command to deliver_scheduled_email.
This makes it parallel with deliver_scheduled_messages, and clarifies
that it is not used for simply sending outgoing emails (e.g. the
`email_senders` queue).

This also renames the supervisor job to match.
2021-05-11 13:07:29 -07:00
Anders Kaseorg
544bbd5398 docs: Fix capitalization mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-10 09:57:26 -07:00
Anders Kaseorg
d0c6f4f400 python: Strip leading and trailing spaces from docstrings.
This is enforced by Black ≥ 21.4b0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-07 22:42:39 -07:00
Mateusz Mandera
4006bb5153 auth: Improve display of errors when user needs to reset password.
Raising jsonableError in the authentication form was non-ideal because
it took the user to an ugly page with the returned json.

We also add logging of this rare occurence of the scenario being
handled here.
2021-05-07 09:10:45 -07:00
Dinesh
00e0b63c6c auth: Handle user_profile.check_password raising PasswordTooWeakError.
user_profile.check_password(password) in authenticate of
EmailAuthBackend can raise PasswordTooWeakError; this happens when the
user's password is weaker than the current required policies and needs
to be rehashed (E.g. because, as in Django 3.2, the minimum salt
entropy increased).

This is a very rare case, but still needs a good user-facing error
message. We raise a json error to handle this with a user-facing error
message.

See this comment by Mateusz Mandera for a detailed explanation
about this case along with a traceback it generates.
https://github.com/zulip/zulip/pull/15449#discussion_r448308614
2021-05-07 09:10:45 -07:00
Dinesh
c5ab01656d auth: Remove Optional for request param of authenticate().
The authenticate function of EmailAuthBackend had request param
type set Optional[HttpRequest] had `None` as default. This
function is never called without a request. So this changes it to
require an HttpRequest parameter.

It was made `Optional` in bc062e1c4d,
because this parameter was new in Django at the time.

We're safe to make it a required argument as everything worked well
before that recent commit and Mateusz Mandera and I checked if it gets
`None` anywhere and found only authenticate of non EmailAuthBackend
gets `None` in some places like `dev_direct_login`.

All the places in tests where this function got `None` as request
were fixed in previous commit.
2021-05-07 09:10:45 -07:00
Alex Vandiver
af26849554 outgoing_webhook: Set a default timeout of 10s.
Support for the timeouts, and tests for them, was added in
53a8b2ac87 -- though no code could have set them after 31597cf33e.

Add a 10-second default timeout.  Observationally, p99 is just about
5s, with everything else being previously being destined to meet the
30s worker timeout; 10s provides a sizable buffer between them.

Fixes #17742.
2021-05-07 08:39:36 -07:00
Anders Kaseorg
405bc8dabf requirements: Remove Thumbor.
Thumbor and tc-aws have been dragging their feet on Python 3 support
for years, and even the alphas and unofficial forks we’ve been running
don’t seem to be maintained anymore.  Depending on these projects is
no longer viable for us.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-06 20:07:32 -07:00
Flávio Prado
0c78131139 settings: Import LDAP settings in prod_settings_template.
Currently users that try to deploy Zulip through docker has errors
because LDAP group search configuration can't be automated.

Reverts a hunk of f5197518a9.
2021-05-05 10:55:27 -07:00
Mateusz Mandera
cc96f02947 presence: Disable live presence updates in larger realms.
As discussed in the comment, this is a critical scalability
optimization for organizations with thousands of users.

With substantial comment updates by tabbott.
2021-05-05 08:50:42 -07:00