Commit Graph

876 Commits

Author SHA1 Message Date
Tim Abbott
013a8886b2 styles: Rename dark.css to night_mode.css. 2018-03-28 10:42:28 -07:00
Steve Howell
4f05cefb32 Remove obsolete code for stream subscribe buttons.
In 18e43895ff we replaced
stream subscribe buttons with stream links.  The new feature
has been well tested and well received for over a year now,
so it's safe to remove the older feature at this point.

Older sites will have super old messages that still have the
rendered markup; this commit does not attempt to address those
situations.  Most likely, clicking on an old button in the old
message will either do nothing or look like a message reply.
2018-03-24 12:31:36 -04:00
Greg Price
b42a7b1701 digest: Add a server setting, and disable by default.
This feature isn't really ready yet -- the relevance isn't good, so
the emails aren't a great experience.  More work needed; pending that,
just don't send them.

There's already a per-realm setting, which doesn't have a control in
the org settings UI but does suppress it in the per-user settings UI.
Piggyback on that to suppress that UI control when the feature is
disabled at the server level too.

Also cut a comment that hasn't really made sense since the logic was
changed months ago -- the comment originally explained why we sent
digests on Tuesday, Wednesday, and Thursday, and doesn't correspond to
why we dialled back to weekly on Tuesdays.
2018-03-23 14:12:01 -07:00
Greg Price
e44a8b8de9 logging: Add a setting to aid manual testing of error-notif changes.
This is nicer than the "For manual testing ..." comment. :-)
Also as a proper setting we can have it control some logging I
added locally while testing my recent changes to pika logging.
2018-03-21 18:03:05 -07:00
Greg Price
fe0f1edddb settings: Fix double negative in LOGGING_NOT_DISABLED.
Saying "not disabled" just makes it more work to read than it needs to
be -- instead say ENABLED.
2018-03-21 18:03:05 -07:00
Gooca
2363c9955a ui scrollbar: Update perfect-scrollbar to 1.3.0.
Update perfect-scrollbar to fix stutter space-scrolling in #8544. Also
reworked deprecated `element.perfectScrollbar` to `new
PerfectScrollbar(element)`.  Lastly, updated provision version and
changed node module path to new path.

This also refactors perfect-scrollbar in help.js to work with updated
version of perfect-scrollbar. Because the update also changed
perfect-scrollbar's css selectors for all scrollbars in zulip, we
update those too.

Fixes #8544.
2018-03-20 21:59:11 -07:00
Greg Price
2a178b52fb logging: Suppress error mail from routine pika connection failures.
In some environments, these exceptions happen regularly on upgrading
Zulip.  They're harmless because we reconnect, so avoid complaining
noisily about them.
2018-03-20 16:49:05 -07:00
Nikhil Kumar Mishra
f29a1918f3 hotspots: Add ALWAYS_SEND_ALL_HOTSPOTS in dev_settings.
Replace the local variable SEND_ALL from get_next_hotspots.
Add unit test for the same.
2018-03-19 10:39:43 -07:00
Greg Price
22071a44a7 antispam: Add a sitewide ratelimit on invites by new realms.
This applies only on a server open for anyone to create a realm.
Moreover, if the server admins have granted any given realm a
max_invites greater than the default, that realm is exempt too.
2018-03-16 18:00:11 -07:00
Greg Price
dc1eeef30a antispam: Make a setting for default Realm.max_invites.
This makes this value much easier for a server admin to change than it
was when embedded directly in the code.  (Note this entire mechanism
already only applies on a server open for anyone to create a realm.)

Doing this also means getting the default out of the database.
Instead, we make the column nullable, and when it's NULL in the
database, treat that as whatever the current default is.  This better
matches anyway the likely model where there are a few realms with
specially-set values, and everything else should be treated uniformly.

The migration contains a `RenameField` step, which sounds scary
operationally -- but it really does mean just the *field*, in
the model within the Python code.  The underlying column's name
doesn't change.
2018-03-16 18:00:11 -07:00
Steve Howell
fef2840531 Add FetchStatus class.
We will use this in subsequent commits to replace the
`load_more_enabled` flag.
2018-03-09 14:20:58 -08:00
Steve Howell
8063f73f6d Extract message_scroll.js.
This mostly moves code from ui.js.

We change the arguments to `message_fetch.load_more_messages()`
to be `opts` with callbacks for `show_loading` and `hide_loading`.

We also defer starting the scroll handler until `message_fetch.js`
has been initialized.
2018-03-08 15:03:47 -08:00
Steve Howell
3a1bf04a56 compose: Add pills for typing in PM recipients.
@brockwhittaker wrote the original prototype for having
pills in the recipient box when users compose PMs (either
1:1 or huddle).  The prototype was test deloyed on our
main realm for several weeks.

This commit includes all the original CSS and HTML from
the prototype.

After some things changed with the codebase after the initial
test deployment, I made the following changes:

    * In prior commits I refactored out a module called
      `user_pill.js` that implemented some common functions
      against a more streamlined version of `input_pill.js`,
      and this commit largely integrates with that.

    * I made changes in a prior commit to handle Zephyr
      semantics (emails don't get validated) and tested
      this commit with zephyr.

    * I fixed a reload bug by extracting code out to
      `compose_pm_pill.js` and re-ordering some
      calls to `initialize`.

There are still two flaws related to un-pill-ified text in the
input:

    * We could be more aggressive about trying to pill-ify
      emails when you blur or tab away.

    * We only look at the pills when you send the message,
      instead of complaining about the un-pill-ified text.
      (Some folks may consider that a feature, but it's
      probably surprising to others.)
2018-03-07 15:53:11 -08:00
Steve Howell
f0d101edf5 pills: Streamline input pills (for user groups).
The main point of this change is to streamline the core
code for input pills, and we use also modify user groups.

The main change to input_pill.js is that you now
configure a function called `create_item_from_text`, and
that can return an arbitrary object, and it just needs
a field called `display_value`.

Other changes:
    * You now call `input.create(opts)` to create the
      widget.
    * There is no longer a cache, because we can
      write smarter code in typeahead `source` functions
      that exclude ids up front.
    * There is no value/optinalKey complexity, because
      the calling code can supply arbitrary objects and
      do their own external data management on the pill
      items.
    * We eliminate `prependPill`.
    * We eliminate `data`, `keys`, and `values`, and just
      have `items`.
2018-03-07 15:53:11 -08:00
YJDave
f5a24efdd8 static/js: Add settings_ui.js file for common settings UI functions. 2018-03-03 08:38:54 -08:00
Greg Price
6d35a697af antispam: Don't let new accounts in open realms immediately send invites.
We haven't had a problem with this yet, but this should help prevent it.
2018-03-01 14:56:19 -08:00
Steve Howell
1f6ddf0110 refactor: Extract transmit.js from compose.js.
We now isolate the code to transmit messages into transmit.js.
It is stable code that most folks doing UI work in compose.js don't
care about the details of, so it's just clutter there.  Also, we may
soon have other widgets than the compose box that send messages.

This change mostly preserves test coverage, although in some cases
we stub at a higher level for the compose path (this is a good thing).
Extracting out transmit.js allows us to lock down 100% coverage on that
file.
2018-02-20 09:29:26 -08:00
Tim Abbott
b3d37b5412 sendfile: Clean up sendfile settings code. 2018-02-16 05:10:19 +05:30
Aditya Bansal
efe8545303 local-uploads: Start running authentication checks on file requests.
From here on we start to authenticate uploaded file request before
serving this files in production. This involves allowing NGINX to
pass on these file requests to Django for authentication and then
serve these files by making use on internal redirect requests having
x-accel-redirect field. The redirection on requests and loading
of x-accel-redirect param is handled by django-sendfile.

NOTE: This commit starts to authenticate these requests for Zulip
servers running platforms either Ubuntu Xenial (16.04) or above.

Fixes: #320 and #291 partially.
2018-02-16 05:06:37 +05:30
Aman Jain
a0b58b1560 activity: Add a link to copy list of realm admins.
Tweaked by tabbott to do the database queries properly.

This should help user to copy realm admin emails in a go.
Fixes: #7885
2018-02-12 08:55:45 -08:00
Greg Price
12d878c785 APNs: Drop disused setting from old implementation. 2018-02-09 17:16:21 -08:00
Tim Abbott
afda05adc4 rate limiting: Increase default rate limits.
It's too easy to go over the rate limits when using the webapp.

The correct fix for this probably involves some changes to which
routes get covered by what sort of rate limit, but for now, just
increase the limits.
2018-02-09 11:55:08 -08:00
Brock Whittaker
fe3a5665da panels: Rename "desktop_notifications_panel" => "panels".
The original code made a 3/4-hearted effort to generically accommodate
more banners/"panels" later, but named itself after the first one made.

[greg: expanded commit message.]
2018-02-05 16:00:52 -08:00
Aditya Bansal
0fcf0c5052 thumbor: Add thumbor on port 9995 in development.
For now, this does nothing in a production environment, but it should
simplify the process of doing testing on the Thumbor implementation,
by integrating a lot of dependency management logic.
2018-01-29 13:10:29 -08:00
Greg Price
3230d35d7f settings: Clean up EMAIL_HOST documentation a bit.
Gmail is a bad example for outbound email; use a generic example.

Also leave the `= None` default out of the config file, as it's
redundant with DEFAULT_SETTINGS in our internal settings.py ; and
explain in the latter why we don't mention the other SMTP settings.
2018-01-24 14:58:21 -08:00
Greg Price
6da9ecef88 settings: Set a flag when email isn't configured, so we can warn. 2018-01-24 14:34:30 -08:00
Greg Price
08bbd7e61d settings: Slightly simplify EMAIL_BACKEND logic.
DEVELOPMENT is defined as just `not PRODUCTION`, but this code made
it look like things might be more complicated than that.
2018-01-24 14:34:30 -08:00
Tim Abbott
56395f40d2 static: Restore bundling KaTeX and zxcvbn via django-pipeline.
Since we need KaTeX to be available for zerver/lib/tex.py and
static/third/katex/cli.js to be able to shell out to it.  However, for
some reason, the KaTeX we bundle using Webpack doesn't seem to be
importable by Node (and it's also kinda a pain to find its filename
from `cli.js`).

So, we work around this by just using the legacy system for KaTeX.

Something similar is needed for zxcvbn.js, in order to support the
settings_account.js use case (basically deferred loading of this
file); that requires JS code to have access to the correct path for
zxcvbn.
2018-01-23 12:30:28 -08:00
Aditya Bansal
76f6f7cb47 datetimepicker: Add flatpickr lib as dependancy. 2018-01-19 11:33:11 -05:00
Aditya Bansal
c770bdaa3a reminder_bot: Add infra for adding reminder bot to every realm. 2018-01-19 11:33:11 -05:00
Vishnu Ks
0bca0286a1 billing: Integrate Stripe, using Stripe Checkout.
Stripe Checkout means using JS code provided by Stripe to handle
almost all of the UI, which is great for us.

There are more features we should add to this page and changes we
should make, but this gives us an MVP.

[greg: expanded commit message; fixed import ordering and some types.]
2018-01-17 16:43:54 -08:00
Aditya Bansal
ec1297c1e8 schedulemessages: Add delivery system for scheduled message. 2018-01-10 09:18:02 -05:00
Cynthia Lin
7790e2b23c org settings: Add framework for User groups page. 2018-01-06 11:50:52 -05:00
Greg Price
ca5c991994 errors: Simplify manual testing of error emails.
There are two different things you need to patch in order to get error
emails (at `/emails`) in dev.  Flip one of them in dev all the time,
and make the comment on the other a bit more explicit.
2017-12-22 11:52:13 -05:00
Greg Price
b15231dfc2 logging: Rename AdminZulipHandler to AdminNotifyHandler.
This name hasn't been right since f7f2ec0ac back in 2013; this handler
sends the log record to a queue, whose consumer will not only maybe
send a Zulip message but definitely send an email.  I found this
pretty confusing when I first worked on this logging code and was
looking for how exception emails got sent; so now that I see exactly
what's actually happening here, fix it.
2017-12-22 11:52:13 -05:00
Greg Price
b6491fd52a logging: Eliminate log_level from create_logger.
Because calls to `create_logger` generally run after settings are
configured, these would override what we have in `settings.LOGGING` --
which in particular defeated any attempt to set log levels in
`test_settings.py`.  Move all of these settings to the same place in
`settings.py`, so they can be overridden in a uniform way.
2017-12-12 17:17:08 -08:00
Brock Whittaker
e0236646bf night-mode: Add custom CSS through JS.
This adds custom CSS through JavaScript for things that do not
scope well and will override other inherited styles.

This should ONLY be used for problematic CSS that has no obvious
or easy CSS-only solution.

(Specifically, we need this for the "default link" styling, which is
hard to override because we don't want to start winning ties due to
specificity that we would not have won in the light theme).
2017-11-29 23:06:11 -08:00
Tim Abbott
f9347962f6 settings: Only include two-factor apps if two-factor auth is enabled. 2017-11-29 11:09:16 -08:00
Tim Abbott
586e18b237 auth: Allow accounts with the same email in different realms.
[Modified by greg to (1) keep `USERNAME_FIELD = 'email'`,
(2) silence the corresponding system check, and (3) ban
reusing a system bot's email address, just like we do in
realm creation.]
2017-11-28 16:23:10 -08:00
Greg Price
b6cc21b438 debug: Add facility to dump tracemalloc snapshots.
Originally this used signals, namely SIGRTMIN.  But in prod, the
signal handler never fired; I debugged fruitlessly for a while, and
suspect uwsgi was foiling it in a mysterious way (which is kind of
the only way uwsgi does anything.)

So, we listen on a socket.  Bit more code, and a bit trickier to
invoke, but it works.

This was developed for the investigation of memory-bloating on
chat.zulip.org that led to a331b4f64 "Optimize query_all_subs_by_stream()".

For usage instructions, see docstring.
2017-11-28 15:52:07 -08:00
Umair Khan
274bba82b9 two_factor: Add configuration and URLs.
This adds django-two-factor to the project, but held behind
settings.TWO_FACTOR_AUTHENTICATION_ENABLED, so that this has no effect
by default.
2017-11-28 15:21:40 -08:00
Tommy Ip
ddaff4cd2a refactor: Extract upload mechanics to new JS module.
Tweaked by tabbott to move changes from the next commit that are
required for this to pass tests into this commit.

Note that this exports a few items that were not previously exported.
2017-11-27 21:31:51 -08:00
Tim Abbott
8b935f4e99 settings: Add setting for SYSTEM_BOT_REALM.
This fixes some subtle JavaScript exceptions we've been getting in
zulipchat.com, caused by the system bot realm there not being "zulip"
interacting with get_cross_realm_users.
2017-11-27 14:46:07 -08:00
Garvit
52069998eb compose: Preserve links when pasting html.
This uses the to-markdown.js library to do all the hard work of
parsing HTML and turning it into markdown and not e.g. uploaded files.

Tweaked by tabbott to better scope when it activates to just include
pastes of HTML content.

Fixes #5853.
2017-11-21 19:52:47 -08:00
rht
bff736868e Generate custom-icon-webfont on each provision or update-prod-static.
Fixes #7354.
2017-11-20 16:36:49 -08:00
derAnfaenger
395f1e9270 embedded bots: Add database config storage.
Storage limititations are only set on the value of
a config entry, since this is the only user-accessible
part of the schema. Keys are statically set by each
embedded bot.
2017-11-16 23:06:38 -08:00
Aditya Bansal
79576797b0 thumbor: Auto generate thumbor_key and add it to zulip settings. 2017-11-16 22:34:14 -08:00
Rishi Gupta
f0807353ae settings: Remove ABOUT_LINK_DISABLED.
Was introduced in 0f4bbc8, I think because the /team page was unfinished at
the time.
2017-11-16 21:15:24 -08:00
Tim Abbott
054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Brock Whittaker
b76578ca13 Add an experimental dark mode stylesheet.
Note from tabbott: While this initial version is experimental and
definitely incomplete, we expect to have a solid version done over the
next few weeks (after more refactoring).  We're merging this now to
make it easy to test both versions when refactoring our CSS.

Fixes #267.
2017-11-15 16:45:34 -08:00
Steve Howell
ae5ba7f4fd Add a couple cross-realm bots.
These are new:

    new-user-bot
    emailgateway

Our cross-realm bots are hard coded to have email addresses
in the `zulip.com` domain, and they're not part of ordinary
realms.

These have always been cross-realm, but new enforcement in the
frontend code of all messages having been sent by a known user means
that it's important to add these properly.
2017-11-14 16:02:19 -08:00
Umair Khan
0e21cbc5d9 user-groups: Create static/js/user_groups.js. 2017-11-13 13:08:15 +05:00
derAnfaenger
d707e3bc1d bots: Add USER_STATE_SIZE_LIMIT setting. 2017-11-10 18:56:26 -08:00
Brock Whittaker
00d11fb0af input-pill: Add styling.
This adds some styling for the default case of the input pills and some
custom styling.
2017-11-10 14:14:03 -08:00
Brock Whittaker
7b00736fa2 input-pill: Add "input_pill" class and documentation. 2017-11-10 14:14:03 -08:00
Cynthia Lin
0b800b0a7d icons: Create framework for custom icons and add new bot icon. 2017-11-10 11:18:42 -08:00
rht
ccf2792c1c refactor: Remove six.moves.configparser import. 2017-11-07 10:51:44 -08:00
Steve Howell
f105c480b3 Calculate gravatar hashes on the webapp client.
This change prepares us to have the server send avatar_url
of None when somebody wants a gravatar avatar (as opposed
to a user-uploaded one).

Subsequent commits will change behavior on both the server
and client to have this happen.  So this commit has no-op
code for now, but it will soon use the fallback-to-gravatar
logic.
2017-11-07 10:36:02 -08:00
rht
26b6b893e6 zproject: Use python 3 syntax for typing. 2017-11-04 19:58:03 -07:00
Greg Price
119bc14182 alias domains: Add a v1 of this feature.
The main limitation of this version is that it's controlled entirely
from settings, with nothing in the database and no web UI or even
management command to control it.  That makes it a bit more of a
burden for the server admins than it'd ideally be, but that's fine
for now.

Relatedly, the web flow for realm creation still requires choosing a
subdomain even if the realm is destined to live at an alias domain.

Specific to the dev environment, there is an annoying quirk: the
special dev login flow doesn't work on a REALM_HOSTS realm.  Also,
in this version the `add_new_realm` and `add_new_user` management
commands, which are intended for use in development environments only,
don't support this feature.

In manual testing, I've confirmed that a REALM_HOSTS realm works for
signup and login, with email/password, Google SSO, or GitHub SSO.
Most of that was in dev; I used zulipstaging.com to also test
 * logging in with email and password;
 * logging in with Google SSO... far enough to correctly determine
   that my email address is associated with some other realm.
2017-11-03 13:38:59 -07:00
Greg Price
b33d981d7b settings: Simplify setting ALLOWED_HOSTS, by handling EXTERNAL_HOST.
This means one fewer thing the admin typically needs to read, absorb,
and make a decision about at install time.

The one way this change could hypothetically cause trouble is if the
admin wants to keep subdomains of EXTERNAL_HOST out of ALLOWED_HOSTS.
But while the subdomains often won't exist as domain names, it's hard
to imagine the situation in which they would exist but be under
someone else's control, or be doing something other than serving
Zulip realms.
2017-11-03 13:38:59 -07:00
Greg Price
271a9f0da7 settings: Eliminate disused variable EXTERNAL_API_URI.
This was part of the logic to handle EXTERNAL_API_PATH varying.
But also it was already no longer used -- it was only ever passed
into template contexts, as `external_api_uri`, and it'd been
overtaken there by `external_api_uri_subdomain`.

So, update our dev docs to reflect that, and eliminate the variable.
2017-10-30 18:29:29 -07:00
Greg Price
7366a4ca7a settings: Eliminate the disused EXTERNAL_API_PATH.
This setting isn't documented at all, and I believe nobody has used it
since the end of api.zulip.com in 2016.  So we get to complete the
cleanup of this logic.
2017-10-30 18:29:29 -07:00
Harshit Bansal
56f26c2d6f zproject/settings.py: Correct emoji sprite CSS files. 2017-10-28 10:38:33 -07:00
Henrik Pettersson
09cd47c6ad Add UI for viewing and cancelling open Zulip invitations.
Lets administrators view a list of open(unconfirmed) invitations and
resend or revoke a chosen invitation.

There are a few changes that we can expect for the future:

  * It is currently possible to invite an email that you have already
    invited, it might make sense to change this behavior.

  * Resend currently sends an invite reminder instead of resending the
    original invite, this is because 'custom_body' was not stored when
    the first invite was sent.

Tweaked in various minor ways, primarily in the backend, by tabbott,
mostly for style consistency with the rest of the codebase.

Fixes: #1180.
2017-10-27 13:07:43 -07:00
Tim Abbott
b936e8c24b lint: Fix lines in Python codebase longer than 125 characters. 2017-10-26 17:36:54 -07:00
Abhijeet Kaur
a88178afaf Embedded bots: Add support for creating embedded bots via the API.
Adds support to add "Embedded bot" Service objects. This service
handles every embedded bot.

Extracted from "Embedded bots: Add support to add embedded bots from
UI" by Robert Honig.

Tweaked by tabbott to be disabled by default.
2017-10-25 16:13:29 -07:00
Vishnu Ks
9314a7ac8b backends: Move EmailLogBackEnd to email_backends. 2017-10-25 14:35:12 -07:00
Tim Abbott
d91e49b681 settings: Fix whitespace in new LDAP settings code. 2017-10-24 11:47:01 -07:00
Tim Abbott
8a588ba23d ldap: Default REGISTER_LINK_DISABLED=False when using LDAP backend.
This should help prevent confusion where new users find themselves on
the LDAP login form and click "register" because they know they don't
have an account.  Whereas in fact, their account will be auto-created
if they just login, so there's no need for them to access it.
2017-10-24 11:21:49 -07:00
Tim Abbott
ad165a6f8f settings: Remove remaining DEPLOYMENT_ROLE_* code remnants.
These should have been removed when we removed Zilencer.
2017-10-23 21:15:03 -07:00
darshanime
6078f648cb settings: use None for memcached cache timeout.
This is a cleaner approach than the old "really big number" strategy.
2017-10-23 16:34:43 -07:00
Maarten Rijke
841f02934d push_notifications: Add setting to redact content.
This commit adds a setting that toggles redacting content of push
notifications and replacing it with "***REDACTED***".
2017-10-19 18:16:06 -07:00
Rishi Gupta
a08543f1ce server settings: Add setting for MAILING_ADDRESS.
The rules here are fuzzy, and it's quite possible none of Zulip's emails
need an address at all. Every country has its own rules though, which makes
it hard to tell. In general, transactional emails do not need an address,
and marketing emails do.
2017-10-19 00:04:07 -07:00
Aastha Gupta
0fae83b301 notifications: Prompt user to enable desktop notifications.
This is a two-step notifications process that will ask a user
to enable notifications and if they click exit give them three
options:

1. Enable notifications.
2. Ask later.
3. Never ask on this computer again.

The first two are self-explanatory (ask later = next session it
asks again). The third is captured and stored in localStorage and
a check is done on page load to see whether or not notifications
should be displayed.

Commit modified heavily by Brock Whittaker <brock@zulipchat.com>.

Fixes #1189.
2017-10-18 21:55:43 -07:00
Greg Price
b095463f9b logging: Clear out the Django default config on logger django.
By default, Django sets up two handlers on this logger, one of them
its AdminEmailHandler.  We have our own handler for sending email on
error, and we want to stick to that -- we like the format somewhat
better, and crucially we've given it some rate-limiting through
ZulipLimiter.

Since we cleaned out our logging config in e0a5e6fad, though, we've
been sending error emails through both paths.  The config we'd had
before that for `django` was redundant with the config on the root --
but having *a* config there was essential for causing
`logging.config.dictConfig`, when Django passes it our LOGGING dict,
to clear out that logger's previous config.  So, give it an empty
config.

Django by default configures two loggers: `django` and
`django.server`.  We have our own settings for `django.server`
anyway, so this is the only one we need to add.

The stdlib `logging` and `logging.config` docs aren't 100% clear, and
while the source of `logging` is admirably straightforward the source
of `logging.config` is a little twisty, so it's not easy to become
totally confident that this has the right effect just by reading.
Fortunately we can put some of that source-diving to work in writing
a test for it.
2017-10-12 22:45:14 -07:00
Tim Abbott
9426ee024b logging: Filter tornado.general logging for being spammy.
Apparently, this sockjs.tornado logging code resulted in a lot of
buggy error emails whenever a Zulip browser tried to reconnect on a
new IP.  I don't see an obvious way to suppress them from within
sockjs, but that might be a good follow-up issue.

Fixes #6959.
2017-10-11 16:53:52 -07:00
Brock Whittaker
df9092cf42 components: Split out "app_components.css" for shared components.
This new “app_components.css” is for shared components between the
internal app and the portico pages.
2017-10-11 12:24:44 -07:00
Tim Abbott
88bb6c6cad uploads: Set a BOTO_CONFIG path to override broken GCE configuration.
The comment is pretty self-explanatory.  The fact that Google Compute
Engine has this problem does not impress confidence about their
product, but hopefully this is the only really dumb thing they do.

Fixes #4839.
2017-10-08 19:46:57 -07:00
Greg Price
0691724836 passwords: Set default zxcvbn threshold to 10k guesses.
See the discussion in the revised docs for background and motivation,
and an explanation of why this value.
2017-10-08 15:48:44 -07:00
Greg Price
a116303604 passwords: Express the quality threshold as guesses required.
The original "quality score" was invented purely for populating
our password-strength progress bar, and isn't expressed in terms
that are particularly meaningful.  For configuration and the core
accept/reject logic, it's better to use units that are readily
understood.  Switch to those.

I considered using "bits of entropy", defined loosely as the log
of this number, but both the zxcvbn paper and the linked CACM
article (which I recommend!) are written in terms of the number
of guesses.  And reading (most of) those two papers made me
less happy about referring to "entropy" in our terminology.
I already knew that notion was a little fuzzy if looked at
too closely, and I gained a better appreciation of how it's
contributed to confusion in discussing password policies and
to adoption of perverse policies that favor "Password1!" over
"derived unusual ravioli raft".  So, "guesses" it is.

And although the log is handy for some analysis purposes
(certainly for a graph like those in the zxcvbn paper), it adds
a layer of abstraction, and I think makes it harder to think
clearly about attacks, especially in the online setting.  So
just use the actual number, and if someone wants to set a
gigantic value, they will have the pleasure of seeing just
how many digits are involved.

(Thanks to @YJDave for a prototype that the code changes in this
 commit are based on.)
2017-10-08 15:48:44 -07:00
Greg Price
aa4104a5af logging: Add option to show the PID in each log message. 2017-10-06 19:21:40 -07:00
Vishnu Ks
eef72a98e4 backends: Create custom email backend EmailLogBackEnd.
Create a new custom email backend which would automatically
logs the emails that are send in the dev environment as
well as print a friendly message in console to visit /emails
for accessing all the emails that are sent in dev environment.
Since django.core.mail.backends.console.EmailBackend is no longer
userd emails would not be printed to the console anymore.
2017-10-04 08:20:29 -07:00
Tim Abbott
7581e0e1c5 settings: Remove remaining references to REALMS_HAVE_SUBDOMAINS. 2017-10-02 16:43:54 -07:00
Tim Abbott
1d72629dc4 subdomains: Hardcode REALMS_HAVE_SUBDOMAINS=True. 2017-10-02 16:42:43 -07:00
Tim Abbott
63bbbba5aa backend: Add support for mobile_flow_otp in social auth.
It turns out that very little code change is required to support
GitHub auth on mobile.  Ideally, this would come with tests, though
the complicated part of the code path is covered by the Google auth
version.  But writing a test for this would take a long time, and I
think it's worth having the feature now, so I'll be doing tests as a
follow-up project.
2017-09-30 09:02:46 -07:00
Tim Abbott
16c4ec3a2e settings: Cleanup AUTH_LDAP_BIND_PASSWORD use of get_secret.
At this point, it's pretty reasonable to remove the old way of storing
this in settings.py directly.
2017-09-29 22:47:09 -07:00
Tim Abbott
3e9da25571 settings: Organizing the first section of DEFAULT_SETTINGS. 2017-09-29 22:44:46 -07:00
Tim Abbott
a7e81baed8 settings: Document SEND_LOGIN_EMAILS setting. 2017-09-29 22:39:58 -07:00
Tim Abbott
c73c12f0e7 settings: Document privacy policy setting.
Also clean up the text around the TERMS_OF_SERVICE setting a bit.
2017-09-29 22:37:56 -07:00
Tim Abbott
f0c8e75f57 settings: Remove obsolete VERBOSE_SUPPORT_OFFERS.
We removed the actual functionality of emails differing depending on
this setting several months ago.
2017-09-29 22:35:12 -07:00
Tim Abbott
0374664fb5 settings: Clean up ZULIP_PATHS conditionals.
This replaces the ugly fixed list of things that don't go into var/log
with a reasonable test based on the actual path.
2017-09-29 22:33:32 -07:00
Tim Abbott
fb846c567d settings: Remove STATS_DIR.
The use case was removed in 2013 in
81bb2603eb.
2017-09-29 22:32:20 -07:00
Tim Abbott
db97c6f77e settings: Remove obsolete PERSISTENT_QUEUE_FILENAME.
We removed the corresponding pickle-based storage code a long time
ago.
2017-09-29 22:30:30 -07:00
Tim Abbott
bf2ee2e686 settings: Cleanup logic for ANALYTICS_LOCK_PATH.
We can use the standard ZULIP_PATHS system for this rather code in
dev_settings.py.
2017-09-29 22:29:59 -07:00
Tim Abbott
ef5a4000a9 settings: Clean up social auth settings.
The main change here is moving SOCIAL_AUTH_FIELDS_STORED_IN_SESSION to
be with the other hardcoded settings, since it's not something that
makes sense for a sysadmin to change.  But while we're at it, we also
group the overall social auth settings separately from the
GitHub-specific settings.
2017-09-29 22:23:29 -07:00
Tim Abbott
6f343e6bb0 settings: Document remaining undocumented settings with comments.
There's a good number of TODO comments; we should plan to remove most
of these in follow-up cleanup work.
2017-09-29 22:21:43 -07:00
Tim Abbott
698b2bf545 settings: Document the dev/test settings section. 2017-09-29 21:57:29 -07:00
Tim Abbott
f920115f18 settings: Remove configuration for EMAIL_GATEWAY_BOT.
This bot user is handled by INTERNAL_BOTS automatically.
2017-09-29 21:44:20 -07:00
Tim Abbott
2a81060acd settings: Organization optional system bots. 2017-09-29 21:44:20 -07:00
Tim Abbott
2f7f8f1cf0 settings: Clean up EMAIL_GATEWAY_PASSWORD.
We haven't ever recommended that people put EMAIL_GATEWAY_PASSWORD in
their /etc/zulip/settings.py, and so this is probably just
ultra-legacy code.
2017-09-29 21:35:55 -07:00
Tim Abbott
94ba678f5b email_mirror: Clean up EMAIL_GATEWAY_EXAMPLE setting.
This isn't something that a user can ever modify, so it doesn't belong
in DEFAULT_SETTINGS.  While we're at it, we align the appearance of
the email gateway in the docs with whether this setting in the docs
will be valid.
2017-09-29 21:32:26 -07:00
Greg Price
e90b37d9a6 settings: Remove GCM key from DEFAULT_SETTINGS.
We handle this with `get_secret`, below.
2017-09-29 21:26:36 -07:00
Greg Price
6718b199e0 settings: Classify DEFAULT_SETTINGS by audience.
This will help identify the settings that need attention: either
to remove, or to document for server admins, or to just add a
comment to explain.

Identified with the following shell "one-liner" (one 313-char line
as I originally ran it; indentation added here for clarity):

perl -lne 'next unless (/^DEFAULT_SETTINGS/../\}\)?$/);
           next unless (/'\''(.*?)'\''/);
           print $1' \
    zproject/settings.py \
 | while read var; do \
     echo -n "$var: "; \
     (grep -lw "$var" zproject/{prod_settings_template,{dev,test}_settings}.py \
        || echo none) \
      | sed s,zproject/,,g \
      | fmt -w1000; \
   done
2017-09-29 21:26:36 -07:00
Greg Price
b728a2c2d5 settings: Remove S3 key settings from DEFAULT_SETTINGS.
These are handled below with `get_secret`.
2017-09-29 21:26:36 -07:00
Tim Abbott
d3bfc132fb settings: Remove TWITTER_ settings from DEFAULT_SETTINGS.
These are handled via `get_secret` instead.
2017-09-29 10:19:43 -07:00
Tim Abbott
0572d5d4a7 settings: Remove SHOW_OSS_ANNOUNCEMENT.
This was used for the old zulip.com to announce that Zulip was now
open source.
2017-09-29 10:19:43 -07:00
Tim Abbott
43f967bccd settings: Remove obsolete VOYAGER setting. 2017-09-29 10:19:43 -07:00
Tim Abbott
b38cc4ecf7 settings: Remove obsolete ZULIP_COM setting.
We'll probably soon way to remove VOYAGER as well.
2017-09-29 10:19:43 -07:00
Tim Abbott
6e3f4446a0 settings: Remove DBX_APNS_* settings.
These were only ever used by Dropbox.
2017-09-29 10:19:43 -07:00
Tim Abbott
640d4c9eea settings: Improve DEFAULT_SETTINGS indentation. 2017-09-29 10:19:43 -07:00
Greg Price
412f6e70d8 logging: Add option to show the module a log message came from. 2017-09-28 18:26:39 -07:00
Greg Price
0163920577 logging: Make our own Formatter class.
This doesn't yet do much, but it gives us a suitable place to
add code to customize how log messages are displayed, beyond what
a format string passed to the default formatter can do.
2017-09-28 18:26:39 -07:00
Greg Price
e0a5e6fade logging: Explain quirks of Python logging config, and apply a style.
This should make it a little easier to understand our logging config
and make changes to it with confidence.

Many of these items that are now redundant used to be required when we
were setting disable_existing_loggers to True (before 500d81bf2), in
order to exempt those loggers from being cleared out.  Now they're not.

One bit of test code needed a tweak to how it got its hands on the
AdminZulipHandler instance; it can do it from the list on the root
logger just as well as on the `django` logger.
2017-09-28 18:26:39 -07:00
rht
f01b629bf9 zproject: Remove absolute_import. 2017-09-27 20:20:07 -07:00
rht
b937e1fede Remove the rest of import six outside zerver.lib.
We no longer need the six module, now that we're Python 3 only.
2017-09-27 17:09:15 -07:00
Greg Price
315f5b393b logging: Quiet pika.adapters.
This was giving a couple of lines of logs on every normal,
successful connection -- clearly a job for DEBUG, but emitted
on INFO.  Quiet it down.

Fixes #6674.
2017-09-26 21:00:52 -07:00
Greg Price
9e2b9893f9 logging: Organize logger configs for easier reference.
This is a pure reordering.
2017-09-26 21:00:41 -07:00
Vishnu Ks
f58c87917e emails: Log emails that are queued or sent in dev environment.
Tweaked by tabbott to add some comments and clarify the code.
2017-09-26 15:02:47 -07:00
hollywoodno
ee7d4808fe emails: Add log for outgoing emails.
Fixes #5900.
2017-09-25 15:37:09 -07:00
Greg Price
500d81bf2c logging: Stop forcing pre-Python 2.5 legacy config behavior.
The `disable_existing_loggers` option to the `logging.config` module
turns on a rather complicated behavior of disabling some, but not all,
loggers that might have been already configured when the call to
`logging.config.dictConfig` or `logging.config.fileConfig` is made:

> This behaviour is to disable any existing loggers unless they or
> their ancestors are explicitly named in the logging configuration.
  (https://docs.python.org/3/library/logging.config)

Turns out the only reason this is there is as a compatibility hack to
match the behavior of Python 2.4 and below.  See the thread where the
new behavior was introduced: https://bugs.python.org/issue3136

Just as the author of the new behavior explains in that thread from
2008, the legacy behavior forces all logging configuration to be
awkwardly centralized in one place.  That makes the code harder to
read, and it perennially causes confusion when a perfectly
normal-looking `logging.getLogger` call at the top level of one module
mysteriously has no effect, while that in another module works fine,
under the influence of the details of what gets imported when.

So, switch to the shiny new behavior of Python 2.5.  Here LOGGING is a
Django setting which just becomes an argument to logging.config.dictConfig.

This may cause a few of the logfiles in ZULIP_PATHS to become active
that have been dormant for a long time.
2017-09-25 15:06:30 -07:00
Tim Abbott
a2243378ea settings: Set REALMS_HAVE_SUBDOMAINS by default.
This change means that almost every Zulip server out there will now be
using subdomains for every realm.  There are a few complications noted
in the release notes.
2017-09-25 06:52:32 -07:00
Eeshan Garg
390a1fec92 zulip_bots: Generate static files during provisioning.
This commit implements support for copying over static files
for all bots in the zulip_bots package to
static/generated/bots/ during provisioning. This directory
isn't tracked by Git. This allows us to have access to files
stored in an arbitrary zulip_bots package directory somewhere
on the system. For now, logo.* and doc.md files are copied over.

This commit should act as a starting point for extending our
macro-based Markdown framework to our bots/API packages'
documentation and eventually rendering these static files
alongside our webhooks' documentation.
2017-09-22 15:14:42 -07:00
Tim Abbott
c5cfcd7844 settings: Move the templating configuration further down.
The motivation for this is that we'll want to use the STATIC_ROOT
variable in this code in the upcoming commits.

While we're at it, we give it a proper section in the file.
2017-09-22 15:12:13 -07:00
Tim Abbott
b4aa1336e7 logging: Avoid logging certain common 404 errors to error logs.
This should help keep /var/log/zulip/errors.log relatively clean on
production Zulip servers.
2017-09-16 08:07:06 -07:00
Supermanu
28beddfd76 backends.py: Enable auth with any ldap attributes as username.
This commit enables user to authenticate with any attribute set in
AUTH_LDAP_USER_SEARCH given that LDAP_EMAIL_ATTR is set to an email
attributes in the ldap server. Thus email and username can be
completely unrelated.

With some tweaks by tabbott to squash in the documentation and make it
work on older servers.
2017-09-15 10:28:41 -07:00
Tim Abbott
a0a1fe1512 settings: Rename SERVER_URI to ROOT_DOMAIN_URI.
This should be a lot less confusing.

See #6013 for discussion.
2017-08-28 14:09:28 -07:00
Tim Abbott
e092f1afff logging: Fix soft_deactivation log declaration.
Apparently, the soft deactivation log was incorrectly grabbing the
root logger, and thus screwing up where everything got logged.
2017-08-27 18:30:52 -07:00
Tim Abbott
3d4893d906 settings: Rename SUBDOMAINS_HOMEPAGE to ROOT_DOMAIN_LANDING_PAGE.
This new setting name is a lot more readable.
2017-08-24 19:32:16 -07:00
Tim Abbott
be0b523b72 settings: Set STAGING to False in DEFAULT_SETTINGS.
This makes it possible to use this for features like managing the
soft-deactivation cron job running in only one place.
2017-08-23 00:36:06 -07:00
Brock Whittaker
d07281fa66 rename: Change list_rendering.js => list_render.js.
The method itself is called `list_render`, so the file should
reflect that name.
2017-08-22 17:37:36 -07:00
Aditya Bansal
34d30706da soft-deactivation: Log users which were soft deactivate/reactivated. 2017-08-15 22:05:19 -07:00
Tim Abbott
4725afe99a settings: Move NOREPLY_EMAIL_ADDRESS to DEFAULT_SETTINGS.
Since we're auto-detecting the value anyway, there's no reason it
can't be moved to DEFAULT_SETTINGS.

This lets us remove some clutter from the installation documentation.
2017-08-15 17:21:40 -07:00
Greg Price
12fa1d406f Update iOS app id to reflect the current app.
With this change, we get as far as printing the message
"APNS: Sending apple push notification to devices" to the
log when a recent TestFlight build of the app is due for
a notification, and then don't hit an exception.  But
on the other hand I still don't get an actual notification
on my phone, so there's still some debugging to do.
2017-08-14 17:41:57 -07:00
Steve Howell
57f17ae543 Extract top_left_corner.js.
Here are the functions in top_left_corner:

    get_global_filter_li: pure code move
    update_count_in_dom: simplifed copy of similar function in stream_list.js
    update_dom_with_unread_counts: pure code move, split out from function
       of same name in stream_list.js
    delselect_top_left_corner_items: pure code move
    handle_narrow_activated: pure code move + rename
    handle_narrow_deactivated: pure code move, split out from from function
       of smae name in stream_list.js
2017-08-14 13:03:57 -07:00
rht
ea5fef45b5 mypy: Annotate settings.py. 2017-08-14 09:47:28 -07:00
Vaida Plankyte
cd560a80f2 settings.py: Add GOOGLE_OAUTH2_CLIENT_ID to DEFAULT_SETTINGS.
Allows the command-line help text to appear for Google
authentication, rather than a traceback.
2017-08-07 18:38:06 -07:00
James Rowan
db4a8227ee realm creation: Move DEFAULT_NEW_REALM_STREAMS out of settings. 2017-08-01 22:38:22 -07:00
Steve Howell
3f06f28ad7 sending messages: Extract sent_messages.js.
This commit extract send_messages.js to clean up code related
to the following things:

    * sending data to /json/report_send_time
    * restarting the event loop if events don't arrive on time

The code related to /json/report changes the following ways:

    * We track the state almost completely in the new
      send_messages.js module, with other modules just
      making one-line calls.

    * We no longer send "displayed" times to the servers, since
      we were kind of lying about them anyway.

    * We now explicitly track the state of each single sent
      message in its own object.

    * We now look up data related to the messages by local_id,
      instead of message_id.  The problem with message_id was
      that is was mutable.  Now we use local_id, and we extend
      the local_id concept to messages that don't get rendered
      client side.  We no longer need to react to the
      'message_id_changed' event to change our hash key.

    * The code used to live in many places:
        * various big chunks were scattered among compose.js,
          and those were all moved or reduced to one-line
          calls into the new module
        * echo.js continues to make basically one-line calls,
          but it no longer calls compose.report_as_received(),
          nor does it set the "start" time.
        * message_util.js used to report received events, but
          only when they finally got drawn in the home view;
          this code is gone now

The code related to restarting the event loop if events don't arrive
changes as follows:

    * The timer now gets set up from within
      send_messages.message_state.report_server_ack,
      where we can easily inspect the current state of the
      possibly-still-in-flight message.

    * The code to confirm that an event was received happens now
      in server_events.js, rather than later, so that we don't
      falsely blame the event loop  for a downstream bug.  (Plus
      it's easier to just do it one place.)

This change removes a fair amount of code from our node tests.  Some
of the removal is good stuff related to us completing killing off
unnecessary code.  Other removals are more expediency-driven, and
we should make another sweep at ramping up our coverage on compose.js,
with possibly a little more mocking of the new `send_messages` code
layer, since it's now abstracted better.

There is also some minor cleanup to echo.resend_message() in this
commit.

See #5968 for a detailed breakdown of the changes.
2017-08-01 08:58:56 -07:00
Brock Whittaker
8445f886d7 Add ability to pan and zoom lightbox images.
This adds the ability to pan and zoom lightbox images because they
are now converted to <canvas> elements.
2017-07-31 17:03:31 -07:00
Jack Zhang
c242df042f hotspots: Style hotspots. 2017-07-28 16:34:15 -07:00
Steve Howell
bc0761b22b Extract topic_data.js.
This new module tracks the recent topic names for any given
stream.

The code was pulled over almost verbatim from stream_data.js,
with minor renames to the function names.

We introduced a minor one-line function called stream_has_topics.
2017-07-27 14:26:22 -07:00
Pweaver (Paul Weaver)
01b0ec69d6 Move the webpack stats files to var/ and deploy root.
static/ serves static files which get copied around per deploy. Since
the webpack stats files need a consistent name and change per deploy,
they can't live in static/.

This fixes a bug that preventing downgrading a Zulip server to an old
version.
2017-07-18 15:02:58 -07:00
Rishi Gupta
a249822ff5 confirmation: Add validity_in_days to _properties.
Also renames settings.EMAIL_CONFIRMATION_DAYS to
CONFIRMATION_LINK_DEFAULT_VALIDITY_DAYS, and adds a new setting for
invitation links.
2017-07-17 23:18:47 -07:00
Steve Howell
475eb21a5e Revert commits related to client_message_id.
I pushed a bunch of commits that attempted to introduce
the concept of `client_message_id` into our server, as
part of cleaning up our codepaths related to messages you
sent (both for the locally echoed case and for the host
case).

When we deployed this, we had some strange failures involving
double-echoed messages and issues advancing the pointer that appeared
related to #5779.  We didn't get to the bottom of exactly why the PR
caused havoc, but I decided there was a cleaner approach, anyway.
2017-07-14 12:13:35 -07:00
Steve Howell
f6d670ae3d Extract sent_messages.js.
This is mostly straightforward moving of code out of compose.js.

The code that was moved currently supports sending time
reports for sent messages, but we intend to grow out the new
module to track more state about sent messages.

The following function names in this commit are new, but their
code was basically pulled over verbatim:

    process_success (was process_send_time)
    set_timer_for_restarting_event_loop
    clear
    initialize

All the code in the new module is covered by previous tests that
had been written for compose.js.  This commit only modifies
a few things to keep those tests.

The new module has 100% node coverage, so we updated `enforce_fully_covered`.
2017-07-13 23:42:27 -04:00
Brock Whittaker
1cf7ee966b help: Redesign /help/ pages to be a single page app.
This redesigns the /help/ page sets to be a single page app that uses
history.pushState to work the same as the old app.

The big new feature is that now we have the index in a nicely designed
left sidebar.
2017-07-13 15:43:44 -07:00
Umair Khan
f94e8530ee Django 1.11: Remove jinja2 backend.
Django 1.11 adds the ability to pass context processors in Jinja2
backend. Django also sends template_rendered signal in tests.
These two issues were the reason why we added Jinja2 backend, but
after upgrading to Django 1.11 we can remove it.

We still need jinja2/__init__.py, which modifies the environment,
and jinja2/compressors.py, which adds minify_js compressor.
2017-07-12 17:53:25 -07:00
Aditya Bansal
5f87a0f8f8 pep8: Add compliance with rule E261 zproject/settings.py. 2017-07-11 11:53:33 -07:00
Vaida
d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Rishi Gupta
07a3bb4d35 confirmation: Remove get_link_validity_in_days.
This commit removes the ability to configure different validity durations
for different types of confirmation links. I don't think the extra
configurability was worth the extra complexity, either for the user trying
to understand the settings, or for the developer trying to understand the
code.

The commit replaces all confirmation validity duration settings with a
single setting, settings.EMAIL_CONFIRMATION_DAYS.

The only setting it removes is settings.EMAIL_CHANGE_CONFIRMATION_DAYS,
which was introduced in 5bf83f9 and never advertised in prod_settings.py.
2017-07-07 07:46:31 -04:00
Cory Lynch
802ea9abf5 Refactor to delete mousewheel.js.
This old third party library added support
for a "mousewheel" event to detect scrolling.
However, it is not compatible with jQuery 3
and is obsolete now that there is a standard
"wheel" event that accomplishes the same thing.
2017-07-03 11:04:20 -04:00
Pweaver (Paul Weaver)
2c10e7c0db Change from and noreply emails to use EXTERNAL_HOST domain.
This eliminates another mandatory setting from what one needs to
configure to setup a Zulip server.
2017-07-01 08:09:25 -07:00
Rishi Gupta
85d38bd17b emails: Remove DEFAULT_FROM_EMAIL from prod_settings_template. 2017-06-29 17:54:33 -07:00
Rishi Gupta
577e201df0 emails: Replace DEFAULT_FROM_EMAIL with ZULIP_ADMINISTRATOR.
We're getting rid of DEFAULT_FROM_EMAIL from prod_settings_template, since
we only really need one of {DEFAULT_FROM_EMAIL, ZULIP_ADMINISTRATOR}.
2017-06-29 17:54:33 -07:00
Vishnu Ks
9b7585a436 bots: Make welcome-bot cross realm. 2017-06-25 11:23:45 -04:00
Umair Khan
a66b7b7083 Remove Django Guardian dependency.
Guardian adds functionality on top of Django auth system to set
per object permissions. Its problem is that it is has poor performance.
So we decided to remove it in release 1.4.0, but we still kept the
option to revert back to an older version which used Guardian.
See commit 49799440a4 for more details.

This commit is the final piece in the string of commits which move
us towards completely removing guardian from our codebase. The way
we do it as follows:

If you are upgrading from a version <= 1.3.10, you first need to
upgrade to 1.4.x (we recommend 1.4.2). The reason is that we
deprecated Guardian in this version. Once you have upgraded to
1.4.x we can be sure that your Zulip installation doesn't depend
on Guardian and all the data has been successfully migrated away from
Guardian. The second step is to upgrade to latest release which will
not include any reference to Guardian in the codebase. After this
commit migrating directly to the latest release will not work because
in that case Guardian data will not migrate.

The backward incompatible change that this introduces is that
we have squashed all the migrations till version 1.4.0. This was
necessary to remove Guardian because it was needed by the reverse
migration. These migrations were from 0001 to 0028.

Fixes #5420
2017-06-23 08:40:57 -04:00
Harshit Bansal
c0e4f24a25 webpack: Migrate common.js bundle to webpack. 2017-06-22 19:06:32 -04:00
Vaida Plankyte
7577a80fd9 settings.py: Change jinja2 blocks to be stripped of whitespace. 2017-06-22 14:19:33 -04:00
Harshit Bansal
c50b3672c2 webpack: Migrate signup.js bundle to webpack static asset pipeline. 2017-06-20 09:45:26 -04:00
Harshit Bansal
3953734a1c webpack: Migrate stats.js bundle to webpack static asset pipeline. 2017-06-20 09:45:26 -04:00
Umair Khan
95fc16d90d Django 1.11: MIDDLEWARE_CLASSES setting is deprecated.
Django provides MiddlewareMixin to upgrade old-style middlewares. See
https://docs.djangoproject.com/en/1.11/topics/http/middleware/#upgrading-middleware
2017-06-13 15:04:04 -07:00
Pweaver (Paul Weaver)
28874cf26f webpack: Add --minify option to run-dev.py for to test minification. 2017-06-13 12:45:40 -07:00
Mehanig
1e5ce918e3 webpack: Move app_debug.js to webpack bundle. 2017-06-13 12:45:40 -07:00
Mehanig
04626acd5a webpack: Move zxcvbn.js to webpack bundle. 2017-06-13 12:45:39 -07:00
Mehanig
ae29d4b8d2 webpack: Move landing-page.js to webpack bundle. 2017-06-13 12:36:55 -07:00
Mehanig
2c6f2fb93a webpack: Move activity js to webpack. 2017-06-13 12:36:51 -07:00
Rafid Aslam
4f482834b4 deps: Upgrade and move perfect-scrollbar from static/third to npm.
- Remove `perfect-scrollbar` from `static/third` and fetch it from npm.
- Upgrade `perfect-scrollbar` to 0.7.1.
- Bump up the `PROVISION_VERSION` to 5.6.

Changed `wheelSpeed` in "static/js/scroll_bar.js" to 0.5, because when it
20, the scrollbar scrolls very fast.

Changed 'wheelSpeed' in "static/js/emoji_picker.js" from 25 to 0.68
(based on tabbott's testing of scrolling through the emoji list).

Part of #1709.
2017-06-11 10:15:48 -07:00
Mehanig
8c0e500d6b webpack: Migrate api and katex js files to Webpack bundler.
This also injects the dev webpack sockets to katex, since that's part
of the main web app bundle.
2017-06-09 12:33:52 -07:00
Rishi Gupta
15e5814c27 emails: Prevent subject lines and plaintext emails from being HTML-escaped.
Adds a new template engine with autoescape=False, and uses the new engine to
render subject lines and plaintext emails.
2017-06-06 23:22:22 -07:00
Rishi Gupta
4d67ce8051 settings.py: Make template engine settings easier to duplicate.
No functional changes.
2017-06-06 23:22:22 -07:00
Cory Lynch
5d7828096e Split out server_events_dispatch.js from server_events.js. 2017-06-02 16:49:18 -07:00
Akhil
64f2b51496 typeahead: Add pm_conversations module.
In pm_conversations.js, added function to make a user a PM partner and
another function to check if a user is a PM partner. A PM partner is
someone with whom the user has been in a PM with.
2017-06-01 08:05:37 +00:00
Akhil
f04da3d52e typeahead: Add recent_senders module.
In recent_senders module, added a data structure to hold timestamps of
users' latest message in a topic. Also added a function to compare 2
users based on above timestamp. Added a function to process messages for
the data structure and a call in add_message_metadata. Also added node
tests for insertion of data into recent_senders.senders.
2017-06-01 08:05:37 +00:00
Rohitt Vashishtha
2d73e03e37 ui-refactor: Rename modals.js to overlays.js.
Fixed #4702.
2017-05-29 11:24:46 -07:00
Mehanig
3f5d0e69fb Build and handle jsfiles using webpack instead of django-pipeline.
Also renames bundle.js to translations.js.
2017-05-24 18:38:03 -07:00
Umair Khan
603b60940d github: Access private emails.
Previously our scope setting only allowed us access to
publicly listed email addresss. This commit changes that
to get access to private email addresses as well.

Fixes: #4937.
2017-05-24 08:24:26 -07:00
Eeshan Garg
baff121115 app_filters: Render HTML to render Jinja2 syntax within Markdown macros.
If a Markdown macro contains Jinja2 template code, it isn't rendered
because render_markdown_path calls template.render on the including
.md file before the macro has been included. And then the including
.md file is converted to HTML. Therefore, the template code within
a Markdown macro (if any) never gets rendered and is returned as it is.

Now, after the source .md file is converted to HTML,
render_markdown_path renders the resulting HTML so that any template
code within included macros (if any) is finally rendered.
2017-05-22 18:19:14 -07:00
Tim Abbott
9d63a5ab3a push_notification: Delete obsolete DBX_IOS_APP_ID code.
I'm not sure that this was ever actually used, but it's definitely
just clutter for Zulip today.
2017-05-16 12:26:55 -07:00
Eeshan Garg
10dcc99983 zerver/decorators: Log all exceptions raised in api_key_only_webhook_view.
Fixes #4742.
2017-05-13 14:53:47 -07:00
Brock Whittaker
6e7305f784 js: Implement DynamicText class.
This implements the DynamicText class for resizing of text to fit the
parent node.
2017-05-11 17:23:53 -07:00
Tim Abbott
5019b53492 auth: Pass is_signup option around. 2017-05-10 17:20:34 -07:00
Tim Abbott
c21d336bfe css: Use the Zulip static asset pipeline for emoji sprite sheet CSS.
This should be a nonfunctional change in most cases; the main benefit
should be preventing caching issues if/when our sprite sheets change.
2017-05-09 19:45:57 -07:00
Steve Howell
0a0f567aeb Split out markdown.js from echo.js.
The new module handles markdown rendering.

The code left behind in echo.js does local-echo kind of things
like reifying message ids.
2017-05-09 11:06:10 -07:00
Steve Howell
70d4ac93ce Add modals.info_overlay_open().
This also removes ui_state.js, since its last function
is now replaced by modals.info_overlay_open().
2017-05-09 09:19:27 -07:00
Mahim Goyal
6464514ca9 Break compose.js and drafts.js dependency. 2017-05-08 14:43:49 -07:00
Rishi Gupta
962b56efbd settings: Change example NOREPLY_EMAIL_ADDRESS to have a display name.
We used to use constructions like
from_email = "Zulip <%s>" % (settings.NOREPLY_EMAIL_ADDRESS,)
but no longer do. All references to settings.NOREPLY_EMAIL_ADDRESS in the
codebase now do not append a display name.
2017-05-05 14:38:25 -07:00
Rishi Gupta
e46cbaffa2 email: Remove Mandrill pathways and dependency.
Everything it was doing (send_future_email) can now be done using
ScheduledJob.
2017-05-05 14:38:23 -07:00
Tim Abbott
84008b9c37 context_processors: Rename confusingly named add_settings context.
Also document this.
2017-05-01 15:47:58 -07:00
K.Kanakhin
f13d6a18eb realm-emoji: Add realm emoji uploading instead url providing.
- Add file_name field to `RealmEmoji` model and migration.
- Add emoji upload supporting to Upload backends.
- Add uploaded file processing to emoji views.
- Use emoji source url as based for display url.
- Change emoji form for image uploading.
- Fix back-end tests.
- Fix front-end tests.
- Add tests for emoji uploading.

Fixes #1134
2017-05-01 14:50:20 -07:00
Jack Zhang
f8b3ce7d15 emoji: Move all emoji picker logic/events into emoji_picker module.
Added emoji_picker.js to static asset pipeline.
2017-04-30 14:13:36 -07:00
Cory Lynch
0965c43238 Add typeahead for syntax highlighting languages.
Modified composebox_typeahead.js to recognize the triple backtick
and tilde for code blocks, and added appropriate typeahead functions
in that file and in typeahead_helper.js.

Additionally, a new file pygments_data.js contains a dictionary of
the supported languages, mapping to relative popularity
rankings. These rankings determine the order of sort of the
languages in the typeahead.

This JavaScript file is actually in static/generated/pygments_data.js, as it
is generated by a Python script, tools/build_pymgents_data.py. This is
so that if Pygments adds support for new languages, the JavaScript file
will be updated appropriately. This python script uses a set of popularity
rankings defined in lang.json.

Corresponding unit tests were also added.

Fixes #4111.
2017-04-28 17:22:59 -07:00
Brock Whittaker
7afbc9ddd6 Redesign login and registration pages.
This completes a major redesign of the Zulip login and registration
pages, making them look much more slick and modern.

Major features include:
* Display of the realm name, description and icon on the login page
  and registration pages in the subdomains case.
* Much slicker looking buttons and input fields.
* A new overall style for the exterior of these portico pages.
2017-04-26 18:04:05 -07:00
Steve Howell
8eb86335b9 Extract narrow_state.js.
Despite the length of this commit, it is a very straightforward
moving of code from narrow.js -> narrow_state.js, and then
everything else is just s/narrow.foo()/narrow_state.foo()/
(with a few tiny cleanups to remove some code duplication
in certain callers).

The only new functions are simple setter/getters that
encapsulate the current_filter variable:

    narrow_state.reset_current_filter()
    narrow_state.set_current_filter()
    narrow_state.get_current_filter()

We removed narrow.predicate() as part of this, since it was dead
code.

Also, we removed the shim for narrow_state.set_compose_defaults(),
and since that was the last shim, we removed shim.js from the app.
2017-04-25 09:57:32 -07:00
Steve Howell
7326971380 Extract stream_edit.js.
This code makes the right pane work in "Manage Streams" when
you are editing a stream subscription.  It handles basic
functionality (submitting forms, etc.), live updates, and
showing the pane as needed.

Most of the code here was simply moved from subs.js, but some
functions were pulled out of larger functions:

    live update:
        add_me_to_member_list
        update_stream_name
        update_stream_description

    collapse/show:
        collapse
        show_sub

We also now export subs.show_subs_pane.

We eventually want stream_edit not to call into subs.js, and
this should be fairly easy--we just need to move some shared
methods to a new module.
2017-04-25 09:57:32 -07:00
Steve Howell
ca2aea8d01 Extract stream_create.js.
This new modules handles the UI to create streams.  It mostly moves
code from subs.js.

It introduces an API around what used to be called meta.stream_created:

    reset_created_stream()
    set_name()
    get_name()

It only partially moves new_stream_clicked().
2017-04-25 09:57:32 -07:00
Steve Howell
5ae284dcb1 Add topic_generator.js to the app. 2017-04-22 11:46:47 -07:00
Shayan Toqraee
534c951ec4 Add rtl.js library for detecting direction of text.
This comes complete with some documentation and node tests, and is a
key step towards implementing RTL support in Zulip.
2017-04-22 11:25:54 -07:00
Tim Abbott
55bea73035 Revert "github: Call the appropriate authenticate."
This reverts commit ab260731a9.

The overridden authenticate method was buggy.
2017-04-19 10:06:00 -07:00
Steve Howell
9591b3a95b Extract stream_sort.js. 2017-04-19 09:16:37 -07:00
Tim Abbott
cddee49e75 Add support infrastructure for push notification bouncer service.
This is an incomplete cleaned-up continuation of Lisa Neigut's push
notification bouncer work.  It supports registration and
deregistration of individual push tokens with a central push
notification bouncer server.

It still is missing a few things before we can complete this effort:
* A registration form for server admins to configure their server for
  this service, with tests.
* Code (and tests) for actually bouncing the notifications.
2017-04-18 23:03:06 -07:00
Tim Abbott
55a9101573 settings: Add support for ZULIP_ORG secrets.
These can be used to authenticate the current Zulip server to
zulip.org.
2017-04-18 23:00:10 -07:00
Steve Howell
70b7d4c00b Extract compose_state.js.
This is mostly just moving methods out of compose.js.

The variable `is_composing_message`, which isn't a boolean, has
been renamed to `message_type`, and there are new functions
set_message_type() and get_message_type() that wrap it.

This commit removes some shims related to the global variable
`compose_state`; now, `compose_state` is a typical global
variable with a 1:1 relationship with the module by the same
name.

The new module has 100% line coverage, most of it coming
via the tests on compose_actions.js.  (The methods here are
super simple, so it's a good thing that the tests are somewhat
integrated with a higher layer.)
2017-04-18 12:26:58 -07:00
Umair Khan
ab260731a9 github: Call the appropriate authenticate.
This commit makes sure that GitHubAuthBackend will only authenticate
using its own authenticate method. This is done by adding a new
Python Social Auth strategy which instead of calling authenticate
method of Django, calls the authenticate of the backend directly.

The problem this commit solves is that while authenticating through
GitHub backend, we were ending up getting authenticated through
ZulipDummyBackend. This might happen because the default strategy used
by Python Social Auth calls the authenticate method of Django which
iterates over all the backends and tries the authenticate methods
which match with the function arguments. The new strategy this commit
adds calls the authenticate method of GitHub backend directly which
makes sense because we already know that we want to authenticate with
GithHub.

The actual problem of why we are ending up on ZulipDummyBackend is
still a mystery because the function arguments passed to its
authenticate method are different. It shouldn't be called.
2017-04-17 21:03:08 -07:00
Steve Howell
22e21cddcb admin/settings: Lazy-load Organization sections.
We now wait to load Organization sections until you
click on the section (or virtually click by using arrow
keys).

Some of the sections are coupled in terms of their setup,
so some sections will already be loaded if you had clicked
on a related section.
2017-04-17 20:55:42 -07:00
Brock Whittaker
9576d5caef frontend: Implement list_render class.
This implements a list_render closure class that allows for
progressive, responsive rendering of long, scrollable lists, with
filtering support.

It isn't used, at present.
2017-04-14 14:52:50 -07:00
Steve Howell
dd0c50f0df Extract compose_actions.js.
This module extracts these two functions that get called by
several other modules:

    start()
    cancel()

It is a little bit arbitrary which functions got pulled over
with them, but it's generally functions that would have only
been called via start/cancel.

There are two goals for splitting out this code.  The first
goal is simply to make `compose.js` have fewer responsibilities.
The second goal is to help break up circular dependencies.
The extraction of this module does more to clarify
dependencies than actually break them.  The methods start()
and cancel() had actually been shimmed in an earlier commit,
and now they no longer have a shim.

Besides start/cancel, most of the functions here are only
exported to facilitate test stubbing.  An exception is
decorate_stream_bar(), which is currently called from
ui_init.js.  We probably should move the "blur" handler out
of there, but cleaning up ui_init.js is a project for another
day.

It may seem slightly odd that this commit doesn't pull over
finish() into this module, but finish() would bring in the
whole send-message codepath.  You can think of it like this:

* compose_actions basically just populates the compose box
* compose.finish() makes the compose box do its real job,
  which is to send a message
2017-04-14 13:09:19 -07:00
hackerkid
2a0a84b229 Add moment.js third-party module via npm.
This is needed for timezone manipulations.  We may be able to replace
XDate with it in the future as well.
2017-04-14 10:22:06 -07:00
Steve Howell
6b549248e8 Extract settings_filters.js. 2017-04-13 10:39:39 -07:00
Steve Howell
3e37f64f71 Extract settings_streams.js. 2017-04-13 10:39:39 -07:00
Steve Howell
70afb59cff Extract settings_users.js.
This affects three admin sections:

    * Users
    * Deactivated users
    * Bots
2017-04-13 10:39:39 -07:00
Steve Howell
0fc7b9907f Extract settings_org.js (and fix live updates).
This is mostly moving code, but we do add short-circuit logic
for some live-updating methods here.

Note that this affects two different sections of the admin app:

    * Organization settings
    * Authentication methods

We really want to move to one module per section, but there is some
legacy coupling that makes this difficult for now.
2017-04-13 10:39:39 -07:00
Steve Howell
a3b44148af Extract settings_emoji.js. 2017-04-13 10:39:39 -07:00
K.Kanakhin
4891a8d850 Add default response for terms and privacy endpoints.
- Add setting for `privacy policy` template defining.
- Configure default templates for `privacy policy` and
  `terms of service` pages.
- Add route for privacy page.
- Remove condition for showing `privacy` and `terms` pages.
- Add `privacy_policy` setting to context processor.
- Add documentaion part for `privacy` and `terms` templates
  configuration.
- Add tests.

Fixes #3686.
2017-04-12 14:09:14 -07:00
Tim Abbott
79ad174ad3 settings: logging configuration to consistently email errors.
This fixes 2 issues:
* Some exceptions were not being properly emailed to admins.

* A bug in the parens placement in the default Zulip handlers list
  resulted in the console/file handlers being accidentally excluded if
  !ERROR_REPORTING.

Fixes #4127.
2017-04-12 11:17:57 -07:00
Steve Howell
06f9c28fd2 settings: Lazy-load settings sections.
For the settings UI, we now wait until a user goes to a particular
settings section before calling the appropriate function to set
up the section (which usually involves setting up click handlers
and populating initial data).
2017-04-07 15:20:12 -07:00
Steve Howell
7f17fc020f Extract settings_bots.js. 2017-04-07 15:20:12 -07:00
Steve Howell
f37ce1eeb1 Extract settings_lab.js. 2017-04-06 11:28:36 -07:00
Steve Howell
89128a2272 Extract settings_muting.js. 2017-04-06 11:28:36 -07:00
Steve Howell
1f38884b27 Extract settings_notifications.js. 2017-04-06 11:28:36 -07:00
Steve Howell
fa143d4582 Extract settings_display.js 2017-04-06 11:28:36 -07:00
Steve Howell
47bdecdc4f Extract settings_account.js.
This code handles the settings pane for "Your account," which
has email/name/password/avatar/etc.
2017-04-06 11:28:36 -07:00
Steve Howell
4bbd73a9a2 Extract list_util.js for navigating lists.
The code here used to live in hotkey.js.  Its complicated calling
protocol made it difficult to unit test.  We are also trying to
slim down hotkey.js.

Our arrow navigation for things like `#stream_filters` has always
been kind of awkward, since it's difficult to get the focus to
their list items.  This commit does nothing to fix that yet.
2017-04-05 11:53:52 -07:00
Steve Howell
2718bd0b5d Extract presence.js to track presence info.
Most of this code was simply moved from activity.js with some
minor renaming of functions like set_presence_info -> set_info.

Some functions were slightly nontrivial extractions:

    is_not_offline:
        came from activity.huddle_fraction_present

    get_status/get_mobile:
        simple getters

    set_user_status:
        partial extraction from activity.set_user_status

    last_active_date:
        pulled out of admin.js code

We also fixed activity.filter_and_sort to take user_ids.
2017-04-04 15:57:10 -07:00
Brock Whittaker
34f9ccb87c alerts: Change sidebar alerts to be at top of the screen.
This changes the alerts to be individual boxes that slide down from
the top of the screen for a better UI experience.
2017-04-03 16:22:05 -07:00
Steve Howell
2d52463b61 Extract ui_report.js.
This moves the implementations of error/report/message from
ui.js to ui_report.js.  They had been shimmed before, so calling
modules still use the same names to call the functions, but we
no longer need the shims.
2017-04-03 07:13:25 -07:00
Rohitt Vashishtha
c7b40e76ac css: Make thirdparty-fonts.css load before Zulip stylesheets. 2017-03-29 21:18:43 -07:00
Amy Liu
6f061beb46 hotspots: Add backend support for tutorial hotspots.
This commit adds the backend support for a new style of tutorial which
allows for highlighting of multiple areas of the page with hotspots that
disappear when clicked by the user.
2017-03-29 11:34:32 -07:00
Tim Abbott
1a17b11788 logging_handlers: Add tests for main error reporting code path. 2017-03-26 13:10:43 -07:00
Tim Abbott
5672618b82 css: Extract popovers.css. 2017-03-25 20:14:17 -07:00
hollywoodno
75d9630258 Add notifications on new logins to Zulip.
This adds helpful email notifications for users who just logged into a
Zulip server, as a security protection against accounts being hacked.

Text tweaked by tabbott.

Fixes #2182.
2017-03-25 16:50:52 -07:00
sinwar
397e9c109e auth: Separate development login from main login page.
This allows us to enable EmailAuthBackend by default in development
without cluttering the development login experience.

Fixes #3652.
2017-03-22 12:54:30 -07:00
Raghav Jajodia
9707c74f33 message_edit: Added copy to clipboard button.
A copy-to-clipboard button is added over message-edit textarea.
Closes #3239.
2017-03-22 11:00:18 -07:00
Steve Howell
4fb8339954 Extract typing_events.js. 2017-03-22 07:29:42 -07:00
Steve Howell
642be6ad18 Revamp state tracking for outbound typing indicators.
This change moves most of the logic related to starting and
stopping outbound typing indicators to a new module called
typing_status.js that is heavily unit tested.

While this was in some sense a rewrite, the logic was mostly
inspired by the existing code.

This change does fix one known bug, which is that when we
were changing recipients before (while typing was active), we
were not stopping and starting typing indicators.  This was
a fairly minor bug, since usually users leave the compose
box to change recipients, and we would do stop/start under
that scenario.  Now we also handle the case where the user
does not leave the compose box to change recipients.
2017-03-22 07:01:20 -07:00
Steve Howell
67a2094ed1 Add typing_data.js, which can track users who are typing.
(A subsequent commit will actually integrate this into the app.)
2017-03-21 17:24:40 -07:00
Yago González
34a9e1ae11 markdown: Add TeX typesetting support.
Co-authored-by: Reid Barton <rwbarton@gmail.com>

Fixes #2056.
2017-03-21 16:40:00 -07:00
adnrs96
a91012bd70 Move thirdparty-fonts.css from static/styles to static/third. 2017-03-21 13:40:05 -07:00
Steve Howell
fd856d728c Extract message_util.js 2017-03-19 21:03:45 -07:00
Steve Howell
8d3d70984d Extract message_events.js. 2017-03-19 21:03:45 -07:00
Steve Howell
1114c8bf62 Extract message_fetch.js. 2017-03-19 21:03:45 -07:00
wangjames
12eeb27442 js dependencies: Extract ui_state.js.
The function home_tab_obscured used to be in the ui_state
namespace via a shim, but now we have an actual module for
it.
2017-03-19 14:11:50 -07:00
Steve Howell
d026344b37 Extract js/ui_util.js. 2017-03-19 11:05:45 -07:00
Steve Howell
1d7d6869c9 Extract stream_events.js 2017-03-19 10:56:09 -07:00
Steve Howell
cfd1e8cbc3 Extract stream_muting.js. 2017-03-19 10:56:09 -07:00
Steve Howell
ab34b5ee9f Extract ui_init.js. 2017-03-19 10:56:09 -07:00
Rishi Gupta
19d8d16126 js dependencies: Split hash_util.js from hashchange.js. 2017-03-18 20:40:34 -07:00
Steve Howell
35d38d62f3 Add shim.js w/narrow_state global. 2017-03-18 15:52:50 -07:00
Tim Abbott
f67e9a7e5e CSS: Rename #overlay to #lightbox_overlay. 2017-03-18 13:54:11 -07:00
Tim Abbott
ed90879602 js: Extract lightbox.js from ui.js. 2017-03-18 13:54:11 -07:00