Commit Graph

310 Commits

Author SHA1 Message Date
Tim Abbott
bfc30e4214 api docs: Improve headings and organization for endpoints.
Mostly, this is a change in ordering to make more sense, but we also
fix several names that were clearly confusing.

We restore the convention that each endpoint has the same title at the
top of the page as what we have in the sidebar menu, which appears to
have been violated in many recent updates to API documentation.
2020-06-18 22:38:05 -07:00
orientor
d8ef853545 openapi: Make api-docs filename match with operationIds.
api docs filenames are basically the operationId of their endpoint
in zulip.yaml with `_` replaced by `-`. But some operationIds have
changed, so change the affected filenames. Make changes in other
files accordingly.
2020-06-18 22:38:05 -07:00
Hashir Sarwar
5200598a31 events: Don't send avatar URLs of long term idle users.
This adds a new client_capability that clients such as the mobile apps
can use to avoid unreasonable network bandwidth consumed sending
avatar URLs in organizations with 10,000s of users.

Clients don't strictly need this data, as they can always use the
/avatar/{user_id} endpoint to fetch the avatar if desired.

This will be more efficient especially for realms with
10,000+ users because the avatar URLs would increase the
payload size significantly and cost us more bandwidth.

Fixes #15287.
2020-06-18 21:35:16 -07:00
sahil839
c488a35f10 streams: Add API for changing stream-level message_retention_days.
This commit adds backend support for setting message_retention_days
while creating streams and updating it for an existing stream. We only
allow organization owners to set/update it for a stream.

'message_retention_days' field for a stream existed previously also, but
there was no way to set it while creating streams or update it for an
exisiting streams using any endpoint.
2020-06-18 17:00:15 -07:00
Tim Abbott
f8ea5f3769 pointer: Remove pointer from GET /users/me.
This cleans up a bit of mess in the Zulip API.
2020-06-18 12:55:59 -07:00
orientor
817ee5c6b2 openapi_docs: Replace argument with parameter.
The term `parameter` is a better word than `argument` for data passed
to an API endpoint; this is why OpenAPI uses in their terminology.
Replace `argument` with `parameter` in the API docs to improve their
readability.

Fixes #15435.
2020-06-18 09:18:27 -07:00
akashaviator
ce571048b9 openapi: Document GET /attachments endpoint. 2020-06-17 18:35:44 -07:00
sahil839
791e5de5de api: Remove is_old_stream property from the stream objects.
This commit removes is_old_stream property from the stream objects
returned by the API. This property was unnecessary and is essentially
equivalent to 'stream_weekly_traffic != null'.

We compute sub.is_old_stream in stream_data.update_calculated_fields
in frontend code and it is used to check whether we have a non-null
stream_weekly_traffic or not.

Fixes #15181.
2020-06-16 10:26:33 -07:00
Aman Agrawal
cda7b2f539 deletion: Add support for bulk message deletion events.
This is designed to have no user-facing change unless the client
declares bulk_message_deletion in its client_capabilities.

Clients that do so will receive a single bulk event for bulk deletions
of messages within a single conversation (topic or PM thread).

Backend implementation of #15285.
2020-06-14 22:34:00 -07:00
orientor
61c64b3df9 tests: Use validate_against_openapi_schema in testing.
This adds a powerful end-to-end test for Zulip's API documentation:
For every documented API endpoint (with a few declared exceptions that
we hope to remove), we verify that every API response received by our
extensive backend test suite matches the declared schema.

This is a critical step towards being able to have complete, high
quality API documentation.

Fixes #15340.
2020-06-14 15:05:52 -07:00
Anders Kaseorg
57a80856a5 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format.

Now including %d, %i, %u, and multi-line strings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:39:00 -07:00
Anders Kaseorg
edf411718c openapi: Fix documentation of narrow parameter.
The narrow parameter was incorrectly documented as a one-level array
rather than an array of arrays, and the tests incorrectly expected
this due to a combination of design and implementation bugs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg
91a86c24f5 python: Replace None defaults with empty collections where appropriate.
Use read-only types (List ↦ Sequence, Dict ↦ Mapping, Set ↦
AbstractSet) to guard against accidental mutation of the default
value.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg
69c0959f34 python: Fix misuse of Optional types for optional parameters.
There seems to have been a confusion between two different uses of the
word “optional”:

• An optional parameter may be omitted and replaced with a default
  value.
• An Optional type has None as a possible value.

Sometimes an optional parameter has a default value of None, or None
is otherwise a meaningful value to provide, in which case it makes
sense for the optional parameter to have an Optional type.  But in
other cases, optional parameters should not have Optional type.  Fix
them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg
365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg
69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Kartik Srivastava
8c39ddfd28 api: Add GET /users/{user_id}/subscription/{stream_id} endpoint.
This new endpoint returns a 'user' dictionary which, as of now,
contains a single key 'is_subscribed' with a boolean value that
represents whether the user with the given 'user_id' is subscribed
to the stream with the given 'stream_id'.

Fixes #14966.
2020-06-10 17:59:14 -07:00
Steve Howell
d5cc29755e typing: Accept only user_ids in typing endpoint.
The only clients that should use the typing
indicators endpoint are our internal clients,
and they should send a JSON-formatted list
of user_ids.

We now enforce this, which removes some
complexity surrounding legacy ways of sending
users, such as emails and comma-delimited
strings of user_ids.

There may be a very tiny number of mobile
clients that still use the old emails API.

This won't have any user-facing effect on
the mobile users themselves, but if you type
a message to your friend on an old mobile
app, the friend will no longer see typing
indicators.

Also, the mobile team may see some errors
in their Sentry logs from the server rejecting
posts from the old mobile clients.

The error messages we report here are a bit
more generic, since we now just use REQ
to do validation with this code:

    validator=check_list(check_int)

This also allows us to remove a test hack
related to the API documentation.  (We changed
the docs to reflect the modern API in an
earlier commit, but the tests couldn't be
fixed while we still had the more complex
semantics for the "to" parameter.)
2020-06-10 17:50:32 -07:00
orientor
e989c87a18 openapi: Fix response validation bug.
Currently there are no checks in validate_against_openapi_schema
to check whether the `content` it received actually matched with
the `response code`. For example during testing if a certain endpoint
was returning 400 but it was expected to return 200, then it would
pass schema validation as it would only have `msg` and `result` keys.
Add this validation and fix the wrong response returning points.
2020-06-10 16:45:36 -07:00
orientor
03ef5e0b31 openapi: Add response example validation.
Add test to validate example responses in zulip.yaml. Also change
zulip.yaml for some wrong examples or for cases which were not
covered by `test-api`. Also enhance `validate_against_openapi_schema`.
2020-06-10 15:45:50 -07:00
Anders Kaseorg
67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg
6480deaf27 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, with more
restrictions patched out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 14:48:09 -07:00
Dinesh
dc90d54b08 auth: Add Sign in with Apple support.
This implementation overrides some of PSA's internal backend
functions to handle `state` value with redis as the standard
way doesn't work because of apple sending required details
in the form of POST request.

Includes a mixin test class that'll be useful for testing
Native auth flow.

Thanks to Mateusz Mandera for the idea of using redis and
other important work on this.

Documentation rewritten by tabbott.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-06-09 17:29:35 -07:00
Tim Abbott
71078adc50 docs: Update URLs to use https://zulip.com.
We're migrating to using the cleaner zulip.com domain, which involves
changing all of our links from ReadTheDocs and other places to point
to the cleaner URL.
2020-06-08 18:10:45 -07:00
sahil839
7de23b8b5c users: Remove short_name and client_id from get_profile_backend.
This commit removes short_name and client_id fields from the user
objects returned by get_profile_backend because neither of them
had a purpose.

* short_name hasn't been present anywhere else in the Zulip API for
  several years, and isn't set through any coherent algorithm.
* client_id was a forgotten 2013-era predecessor to the queue_id field
  returned by the register_event_queue process.

The combination of these changes gets us close to having `get_profile`
have the exact same format as other endpoints fetching a user object.
2020-06-08 17:01:08 -07:00
sahil839
1f5778bad7 users: Refactor get_profile_backend to be based on format_user_row.
This commit changes get_profile_backend to be based on format_user_row
such that it's a superset of the fields for our other endpoints for
getting data on a user.

To be clear, this does not removes any of the exisiting fields, that
were returned by this endpoint.

This change adds some fields to the User object returned by the
endpoint. API docs are updated accordingly for the added fields.
2020-06-08 16:57:44 -07:00
Anders Kaseorg
8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Kartik Srivastava
5cb1edcadc openapi/python_examples: Update examples for add/remove subscriptions.
This updates the examples to use user ids to add or remove subscriptions
from a stream.
2020-06-05 16:11:04 -07:00
Aman Agrawal
a096f34cab move_topic_to_stream: Add option to disable breadcrumb messages.
Option to disable breadcrumb messages were given in both message edit
form and topic edit stream popover.

User now has the option to select which stream to send the notification
of stream edit of a topic via checkboxes in the UI.
2020-06-05 12:28:51 -07:00
orientor
1227b65066 openapi: Forbid opaque objects from OpenAPI documentation.
Objects whose properties are not described were validated by the
current validator. Edit it so that objects with no `properties`
or `additionalProperties` attribute i.e. opaque objects get
invalidated.

Also make changes in zulip.yaml to fix any opaque objects (tweaked by
tabbott to edit the documentation for better clarity).
2020-06-02 16:45:06 -07:00
orientor
8224400166 openapi_responses: Improve response validation tests.
Currently, `validate_against_openapi_schema` checks only the top
level of the response dictionary. Improve it so that it can
validate objects and arrays at all levels. Also edit zulip.yaml
accordingly. And for new response keys which were not defined
before add VERY basic documentation.
2020-06-02 16:34:54 -07:00
Tim Abbott
f93c19ec62 openapi: Fix various errors in OpenAPI declarations.
Based on original work by Aditya Verma to tag places where our API
documentation was inaccurate / incomplete.
2020-06-02 11:54:19 -07:00
sahil839
2187c84ed9 stream: Add support for sending user_id to endpoint when subscribing users.
This commit modifies the backend to accept user ids when subscribing
users to streams.

It also migrates all existing tests to use this API, aside from a
small set of tests for the legacy API.
2020-06-01 15:59:04 -07:00
sahil839
9ef1c5b1a6 users: Add is_owner field to user objects returned by get endpoints.
This commit adds 'is_owner' field to the user object returned by
'/users', 'users/{user_id}', and '/users/me' endpoints.
2020-06-01 15:33:51 -07:00
sahil9001
328caf1ad5 api: Simplify format of realm_emoji author data.
There's no reason to send data beyond the user `id` of the uploader,
and reason not to, as the previous model was both awkward when
`author=None` and resulted in unecessary parsing complexity for
clients.

Modified by tabbott to add the frontend changes and API documentation.

Fixes #15115.
2020-05-31 17:44:50 -07:00
orientor
5b5095ad9f openapi: Use correct operationId atrribute for endpoints.
Update `operationId` of some endpoints to their correct version.
2020-05-29 15:04:50 -07:00
orientor
d2ee99a2fd openapi: Add markdown extension for rendering return values in API docs.
Currently response return values have to be written twice, once in
the docs and once in zulip.yaml. Create a markdown extension so
that the return values in api docs are rendered using content from
zulip.yaml
2020-05-29 15:02:56 -07:00
orientor
98903df639 openapi: Fix some zulip.yaml return values description.
There are several places where the return values description is better
in the api docs than in the zulip.yaml file. Import such changes to
zulip.yaml before we replace the current hacky API docs with the
zulip.yaml content.
2020-05-29 15:02:43 -07:00
sahil839
9fa60672e6 users: Modify update user API endpoint to accept role as parameter.
This commit changes the update user API endpoint to accept role
as parameter instead of the bool parameters is_guest and is_admin.

User role dropdown in user info modal is also modified to use
"dropdown_options_widget".

Modified by tabbott to document the API change.
2020-05-29 14:29:17 -07:00
orientor
7e1bc9bf2f openapi_responses: Fix get_user_presence. 2020-05-27 13:23:37 -07:00
orientor
cffb927f07 openapi_responses: Fix get_own_user. 2020-05-27 13:23:36 -07:00
orientor
1273545637 openapi_responses: Fix get_messages. 2020-05-27 13:21:04 -07:00
orientor
4096e0e6b9 openapi_responses: Fix list_subscriptions. 2020-05-27 13:15:51 -07:00
orientor
a4cf77e5dc openapi_responses: Fix get_message_history. 2020-05-27 13:15:51 -07:00
orientor
29ef17c999 openapi_responses: Fix get_realm_emoji. 2020-05-27 13:15:51 -07:00
orientor
b9c31942b5 openapi_responses: Fix get_user_groups. 2020-05-27 13:15:51 -07:00
orientor
74e9019922 openapi_responses: Fix get_stream_topics. 2020-05-27 11:27:27 -07:00
orientor
ae8167cd09 openapi_responses: Fix get_users and get_user. 2020-05-27 11:27:26 -07:00
orientor
bc50308799 openapi_responses: Fix send_message. 2020-05-27 11:23:14 -07:00
orientor
f50aab6740 openapi_responses: Fix update_subscription_settings. 2020-05-27 11:23:12 -07:00