Commit Graph

13898 Commits

Author SHA1 Message Date
PIG208
49eab4efef Revert "request: Refactor to record rate limit data using ZulipRequestNotes."
This reverts commit 3f9a5e1e17.
2021-07-16 09:01:20 -07:00
Abhijeet Prasad Bodas
2545f00ad8 do_update_embedded_data: Remove unnecessary transaction.atomic.
There isn't any attachments code involved here.
This was added in c93f1d4eda, probably accidentally.
2021-07-16 09:00:02 -07:00
Abhijeet Prasad Bodas
9901b72690 tests: Purge queue before starting tests, not at tearDown.
This is more robust towards reruning failed tests (which ran
partially and added some events to a queue before failing).

The tearDown code was added in 571f8b8664.
2021-07-16 09:00:02 -07:00
Anders Kaseorg
a0e3de302f test_reactions: Fix ineffective mock due to indentation mistake.
Introduced by commit 90b6fa7881.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-15 17:44:59 -07:00
akshatdalton
6b5812082e markdown: Fix shebang line eliminating behaviour of Codehilite.
See the block comment explaining the motivation for this change, but
basically, the shebang feature of Python-Markdown's Codehilite
extension could be really confusing and is not part of the CommonMark
standard.

1. https://python-markdown.github.io/extensions/code_hilite/#shebang-no-path
2. eacff473a2/markdown/extensions/codehilite.py (L164-L180)

Fixes: #18591.
2021-07-15 15:18:33 -07:00
Priyansh Garg
a21a280054 data_import: Rename mattermost_user to user_handler.
This logic can be readily reused for new data import tools.
2021-07-15 14:28:36 -07:00
Anders Kaseorg
7ccefa5242 push_notifications: Fix APNs message generation.
This emulates the previous PyAPNs2 behavior of moving the contents of
the ‘custom’ key to top level.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-15 13:49:56 -07:00
Eeshan Garg
0ac7d7d21d corporate: Store sponsorship request data in the database.
We are starting to run into situations where this data could be
quite useful for making future decisions, so it makes to store it
in the database, not just in an email.
2021-07-15 10:31:03 -07:00
Eeshan Garg
434c262365 analytics: Display org_type on /activity.
Moving forward we are hoping to collect data on org types from our
users, so it makes sense to display the org type on the "Counts"
tab of our /activity page.
2021-07-15 10:14:38 -07:00
Eeshan Garg
faa695e86d registration: Collect organization type on sign-up. 2021-07-15 09:58:27 -07:00
Alex Vandiver
43f3f9221d slack_incoming: Handle explicit None text payload in webhook. 2021-07-15 09:37:39 -07:00
sahil839
2f44a08b37 settings: Convert settings_overlay.html to handlebars template.
Fixes #18796.
2021-07-15 08:59:23 -07:00
Tim Abbott
4fb6f089e5 api: Remove error for no settings changes requested.
While this error message was vaguely useful, it wasn't reachable via
the UI at all, and it prevents merging this endpoint with related
endpoints.
2021-07-14 15:54:05 -07:00
sahil839
e0faa9699d api: Remove incorrect realm setting in 'realm/update_dict' schema.
We incorrectly include many realm settings in the data section of
'realm/update_dict' schema. It should only contain the settings
related to message edit, realm icon, realm logo and authentication
methods and not other settings, becausea all the other settings send
'realm/update' event and not 'realm/update_dict' event.

This commit only removes 'add_emoji_by_admins_only' and others will
be removed separately.
2021-07-14 14:57:09 -07:00
sahil839
61acf68aa4 emoji: Allow emoji authors to delete the emoji always.
Previously, non-admin emoji authors were allowed to
delete the emoji only if add_emoji_by_admins_only
was false. But, as add_emoji_by_admins_only setting
is for who can add emoji and not delete emojis, it
should not affect the behavior of deleting emojis
and users should always be allowed to delete the
emojis which. they added themselves
2021-07-14 14:57:08 -07:00
sahil839
50240ca71b models: Use COMMON_POLICY_TYPES for user_group_edit_policy.
This commit adds moderators and full members options for
user_group_edit_policy by using COMMON_POLICY_TYPES.

Moderators do not require to be a member of user group in
order to edit or remove the user group if they are allowed
to do so according to user_group_edit_policy.
But full members need to be a member of user group to edit
or remove the user group.
2021-07-14 14:38:44 -07:00
sahil839
93a1479286 user_group: Change error message to "Insufficient permission".
There is no need to have a error message which specifies the
roles having permission to edit user-groups, we can simply
have error message as "Insufficient permission" as we already
show the roles having permission clearly in UI.
2021-07-14 14:38:43 -07:00
sahil839
2355e66850 models: Move UserProfile.notification_setting_types to UserBaseSettings.
Since all the notifications settings are defined in UserBaseSettings, we
should shift the notification_setting_types dict to UserBaseSettings.
2021-07-14 14:37:57 -07:00
sahil839
2f386aed0f models: Move UserProfile.property_types to UserBaseSettings.
Since all the display settings are defined in UserBaseSettings,
we should shift the property_types dict to UserBaseSettings.
2021-07-14 14:37:57 -07:00
sahil839
d7dfe80454 models: Add RealmUserDefault table for realm-level default of settings.
This table will be used to store the realm-level default of display
and notification settings for new users.
2021-07-14 14:35:04 -07:00
PIG208
c03b9c95ad request: Store client information using ZulipRequestNotes.
This concludes the HttpRequest migration to eliminate arbitrary
attributes (except private ones that are belong to django) attached
to the request object during runtime and migrated them to a
separate data structure dedicated for the purpose of adding
information (so called notes) to a HttpRequest.
2021-07-14 12:01:07 -07:00
PIG208
26875cdf0b tests: Migrate mocked Request to HostMockRequest.
This migrates some mocked Request class and mocked request achieved
with namedtuple in test_decorators and test_mirror_users to use the
refactored HostMockRequest.

Since weakref cannot be used with namedtuple, this old way of mocking a
request object should be migrated to using HostRequestMock. Only after
this change we can extract client from the request object and store it
via ZulipRequestNotes.
2021-07-14 12:01:07 -07:00
PIG208
8eb2c3ffdb request: Move realm from the request to ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208
5167a93229 request: Move tornado_handler to ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208
742c17399e request: Move miscellaneous attributes to ZulipRequestNotes.
This includes the migration of fields that require trivial changes
to be migrated to be stored with ZulipRequestNotes.

Specifically _requestor_for_logs, _set_language, _query, error_format,
placeholder_open_graph_description, saveed_response, which were all
previously set on the HttpRequest object at some point. This migration
allows them to be typed.
2021-07-14 12:01:07 -07:00
PIG208
5475334b16 request: Refactor to store requestor_for_logs in ZulipRequestNotes. 2021-07-14 12:01:07 -07:00
PIG208
3f9a5e1e17 request: Refactor to record rate limit data using ZulipRequestNotes.
We will no longer use the HttpRequest to store the rate limit data.
Using ZulipRequestNotes, we can access rate_limit and ratelimits_applied
with type hints support. We also save the process of initializing
ratelimits_applied by giving it a default value.
2021-07-14 12:01:07 -07:00
PIG208
da6e5ddcae request: Move log_data from HttpRequest to ZulipRequestNotes. 2021-07-14 12:01:05 -07:00
PIG208
75cde8ebcf request: Refactor HostRequestMock to subclass HttpRequest. 2021-07-14 11:52:43 -07:00
PIG208
03693cd27e request: Map HttpRequest to ZulipRequestNotes for typing.
We create a class called ZulipRequestNotes as a new home to all the
additional attributes that we add to the Django HttpRequest object.
This allows mypy to do the typecheck and also enforces type safety.

Most of the attributes are added in the middleware, and thus it is
generally safe to assert that they are not None in a code path that
goes through the middleware. The caller is obligated to do manual
the type check otherwise.

This also resolves some cyclic dependencies that zerver.lib.request
have with zerver.lib.rate_limiter and zerver.tornado.handlers.
2021-07-14 11:52:42 -07:00
akshatdalton
f5c4d51ed2 resolve topic: Add is:resolved search keyword/filtering support.
This commit adds the backend support for `is:resolved` search keyword.
In the next commit, I will add the frontend support for the same.
2021-07-13 23:18:41 -07:00
akshatdalton
7ec406f39d refactor: Extract RESOLVED_TOPIC_PREFIX in topic.py.
This is a prep commit for #18990.
2021-07-13 23:18:41 -07:00
Abhijeet Prasad Bodas
ebb4eab0f9 worker: Rewrite MissedMessageWorker to not be lossy.
Previously, we stored up to 2 minutes worth of email events in memory
before processing them. So, if the server were to go down we would lose
those events.

To fix this, we store the events in the database.

This is a prep change for allowing users to set custom grace period for
email notifications, since the bug noted above will aggravate with
longer grace periods.
2021-07-13 17:21:38 -07:00
Abhijeet Prasad Bodas
1709428cff models: Create MissedMessageEmailEntry table.
This will be used to store the missedmessage events received
during the waiting time for email notifications (which is currently
2 minutes, hardcoded).

The change in `test_retention` is because we've set `on_delete=CASCADE`
for the message field this table.
The new query is like so:
```
DELETE FROM "zerver_missedmessageemailentry"
    WHERE "zerver_missedmessageemailentry"."message_id" IN (
        1545, 1546, 1547, 1548, 1549, 1550, 1551, 1552, 1553
        )
```
2021-07-13 17:21:37 -07:00
Abhijeet Prasad Bodas
c3319a5231 notification_data: Create common source for trigger strings.
This reduces loose strings in the codebase, and allows us to not worry
about the exact naming (`stream_email_enabled` or `stream_emails_enabled`?)
and tense (`mentioned` or `mention`?).

Ideally this new class should have been in `lib/notification_data.py`,
which is our file for things like this. But, the next commit requires
using this data in `models.py`, and importing from `notification_data.py`
to `models.py` causes recursive imports.
2021-07-13 17:16:32 -07:00
PIG208
e373df88e0 webhooks: Make github webhook support event filtering system. 2021-07-13 16:47:18 -07:00
Suyash Vardhan Mathur
309f4ba145 openapi: Make endpoint operationId dash-separated.
The operationId is directly used in URLs of API doc pages
to find the OpenAPI data to render. However, this is dash-
separated in the URLs, and having underscore_separated IDs
in OpenAPI data doesn't allow direct comparison of the two.

This commit changes all OperationIDs from underscore_separated
to dash-separated.
2021-07-13 16:46:22 -07:00
akshatdalton
3ea1ff7665 refactor: Use assertGreaterEqual helper instead of assertTrue. 2021-07-13 13:03:38 -07:00
akshatdalton
0b469e9e4e refactor: Use assertGreater helper instead of assertTrue. 2021-07-13 13:03:38 -07:00
akshatdalton
e203112fd4 refactor: Use assert_length helper instead of assertTrue/assertEqual. 2021-07-13 13:03:38 -07:00
Alex Vandiver
0b7cff2f22 push_notifications: Hide "error" level messages from aioapns.
Work around Fatal1ty/aioapns#15, by silencing error-level logging from
the aioapns logger.  We deal with the results of failed
send_notification calls by examining the `result.description` and
handling them; the extra logging message merely clutters the Sentry
logs.
2021-07-13 09:28:05 -07:00
Suyash Vardhan Mathur
981e4f8946
openapi: Render all responses of an operation.
Previously, one needed to specifying all the HTTP status
codes that we want to render along with the operation,
but the primary use case just needs the responses of
all the status codes, and not just one.

This commit modifies the Markdown extension to render
all the responses of all status codes of a specified
operation in a loop.
2021-07-13 08:33:43 -07:00
Mateusz Mandera
db7d2ee713 test_upload: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera
af29bd77a0 test_sessions: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera
91b3ba6d66 test_queue_worker: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera
ccd961b797 test_reactions: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Mateusz Mandera
6ec5a9698f test_markdown: Fix unnecessarily hard-coded user id. 2021-07-13 08:31:11 -07:00
Abhijeet Prasad Bodas
e63e86dcb2 worker: Ensure complete coverage for PushNotificationsWorker.
The `# nocoverage` was unnecessary apart from for the compatibility code,
so add a test for that code and remove the `# nocoverage`.

The `message_id` -> `message_ids` conversion was done in
9869153ae8.
2021-07-13 08:30:31 -07:00
aryanshridhar
7dc56fdcf2 settings_emoji: Allow only admins to override existing emojis.
Previously, even non-admins had the option to override built-in
emojis in the `Settings Emoji` UI.

This commits essentially limits the functionality of overriding
custom and allows only realm administrators to
override built-in emojis with their custom emojis by adding an
authorization check in the backend.

It also adds relevant tests in `test_realm_emoji` which tests
for the cases where an admin and non admin tries to override
the built-in emoji.

Fixes #18860.
2021-07-13 08:28:28 -07:00
Mateusz Mandera
ab380b122b python_examples: Use ensure_users where appropriate.
We added this function in 8e1a7cfb52
in order to make things more readable in example which hard-code user
ids. The point is to validate that the id indeed refers to the user that
the person writing the example expects, while providing information to
readers of the code so they don't have to do db queries to figure out
the user. As mentioned in the commit referred to above, this is
particularly useful when some db changes cause renumbering of user ids -
because then all these ids have to be adjusted and it's nice to know the
intended user.
2021-07-13 08:24:06 -07:00