Commit Graph

113 Commits

Author SHA1 Message Date
Zev Benjamin
0073014ea9 Remove redundant EXTERNAL_HOST settings
(imported from commit dcf9709bc5c2a2c3f00410fd1b2849d3ab879fb9)
2013-11-05 14:14:19 -05:00
Zev Benjamin
787215d743 [manual] Switch over to new /etc/zulip/zulip.conf config file
Run the following commands as root before deploying this branch:
 # /root/zulip/tools/migrate-server-config
 # rm /etc/zulip/machinetype /etc/zulip/server /etc/zulip/local /etc/humbug-machinetype /etc/humbug-server /etc/humbug-local

(imported from commit aa7dcc50d2f4792ce33834f14761e76512fca252)
2013-11-05 14:14:19 -05:00
Tim Abbott
792b283dcd Report end-to-end sending times as experienced by clients.
This should help with determining the prevalence of slow sends as
experienced by users.

(imported from commit f00797679315c928af3c87ad8fdf0112f1dfa900)
2013-11-04 18:32:34 -05:00
Leo Franchi
6765ec0795 Use a site-specific Zulip Admin email in django error pages
(imported from commit 2d5415d7cd81befc3051b5de3835c0cd258b6375)
2013-11-04 16:35:50 -05:00
Kevin Mehall
78b45b61fb Minify JS in the localserver upgrade tarball.
This moves the list of removed files from .gitattributes to
tools/build-local-server-tarball because static/ and tools/ are
necessary for update-prod-static, and it seemed best to keep the
entire list in one place.

(imported from commit 2a447cbde29e90d776da43bb333650a40d4d363c)
2013-11-04 13:40:43 -05:00
Tim Abbott
68dcc760c3 Clean up some unused imports.
(imported from commit 0c5d8e2a55ba1b8909ba807fee3afe863dcdc226)
2013-11-04 11:51:17 -05:00
Tim Abbott
2844a8fb97 Enable postgres database-level autocommit.
This is something we'll want to do before we switch to Django 1.6,
since it's the default there, and I'm not aware of any significant
problems this switch will cause.

(imported from commit fadea41f09e2179735328f99a3ec55b46c92041e)
2013-11-04 07:31:11 -05:00
Leo Franchi
ca8187b8cf Email gateway bot variable name changed to EMAIL_GATEWAY_BOT
(imported from commit 01c430bd9b916c9bb152ee821214495333f1d37e)
2013-11-01 16:46:17 -04:00
Leo Franchi
9b4491db6a Don't use hardcoded bots in nagios' check_send_receive
(imported from commit 82add135bf5b819bcc992af8420eec14cf829ccc)
2013-11-01 14:13:05 -04:00
Leo Franchi
db6550e99a Make built-in bots come from settings.py, and allow localserver-specific ones
(imported from commit e21933e37487314ac986147562817a19227e8960)
2013-11-01 14:13:05 -04:00
Steve Howell
95dc0894d8 Add /realm_activity report.
This report will eventually replace the per-realm report that is
now accessible through /activity.  In order not to disrupt Waseem,
I'm leaving the old reports around until we've polished the new
ones.

The old report does 24 different queries to get per-realm user data.
The new approach gets all the data at once, and it slices and dices
the data in Python to accomodate our slightly quirky data model.

On localhost, this is a typical query:

LOG:  duration: 5.668 ms  statement: SELECT "zerver_useractivity"."id", "zerver_useractivity"."user_profile_id", "zerver_useractivity"."client_id", "zerver_useractivity"."query", "zerver_useractivity"."count", "zerver_useractivity"."last_visit", "zerver_userprofile"."id", "zerver_userprofile"."email", "zerver_client"."id", "zerver_client"."name" FROM "zerver_useractivity" INNER JOIN "zerver_userprofile" ON ("zerver_useractivity"."user_profile_id" = "zerver_userprofile"."id") INNER JOIN "zerver_realm" ON ("zerver_userprofile"."realm_id" = "zerver_realm"."id") INNER JOIN "zerver_client" ON ("zerver_useractivity"."client_id" = "zerver_client"."id") WHERE "zerver_realm"."domain" = 'zulip.com'  ORDER BY "zerver_userprofile"."email" ASC, "zerver_useractivity"."last_visit" DESC

(imported from commit 0c71f4e32fe5a40f4496749dc29ad3463868d55e)
2013-11-01 11:46:26 -04:00
Steve Howell
3577ba7585 Add /user_activity page.
This page shows aggregate activity for a user on various
clients.  This allows Waseem to troubleshoot things like users
switching between website and desktop, etc.

This particular page probably won't be used too much, but some of the
logic is gonna be reused in the per-realm activity pages.

(imported from commit b8c1fad5bfa45daab40954f92319f6f89a3fa433)
2013-11-01 11:46:26 -04:00
Tim Abbott
21663601b4 Remove obsolete zulip.com/what-is-humbug page.
(imported from commit 0e2f317cf0bf262eb113caa7daab7d71ce564e9e)
2013-10-31 15:29:49 -04:00
Tim Abbott
0e35424a7c Rename humbug => zulip in some log file names.
(imported from commit 0bb5284566861e5a2f260cc66dcb53ebac0d7b97)
2013-10-31 15:29:49 -04:00
Jessica McKellar
f1e90086f5 Remove the onboarding checklist.
Looking at the historical data, fewer than 50% of active users have
completed the checklist, which means that it is just persistent
clutter. We also have other better ways of encouraging people to send
traffic and get the apps now.

This commit removes both the frontend UI and backend work but leaves
the db row for now for the historical data.

(imported from commit e8f5780be37bbc75f794fb118e4dd41d8811f2bf)
2013-10-31 11:34:15 -04:00
Steve Howell
d7d1888ce0 Add TestWorker queue processor.
TestWorker is for troubleshooting, and it simply appends lines to a file
in /tmp.

(imported from commit 96b646a193a5474f3222b41725742e359b5301ec)
2013-10-30 16:47:16 -04:00
Zev Benjamin
57bef07832 [manual] Move /etc/humbug-* files into /etc/zulip
The moved files are:
humbug-server
humbug-local
humbug-machinetype

Their new names are their old names with 'humbug-' removed.

zulip-puppet-apply must be run before this commit is deployed

(imported from commit f4eb523244d3409b5809c279301225d3fdf0c230)
2013-10-30 15:42:25 -04:00
Jessica McKellar
8e838f4db3 Add a first pass at Stash integration via their webhook mechanism.
(imported from commit 6173379f72aeef318550b35e51d4aa1fd69997ed)
2013-10-29 22:40:42 -04:00
Steve Howell
884e602185 [puppet] Handle exceptions from queue workers.
Subclasses of QueueProcessingWorker that don't override start() will
have their consume() functions wrapped by consume_wrapper(), which
will catch exceptions and log data from troublesome events to a log
file.

We need to do a puppet apply to create /var/log/zulip/queue_error.

(imported from commit 3bd7751da5fdef449eeec3f7dd29977df11e2b9c)
2013-10-29 17:45:31 -04:00
Leo Franchi
8e05f76511 Only enable analytics and dropbox on non-LOCAL_SERVER
(imported from commit 1ba877550b3afde51bec6f344762ea998800c5b6)
2013-10-29 17:33:36 -04:00
Kevin Mehall
06590f4332 Upload to local filesystem on local server.
(imported from commit 6c38a8cae721b2e7a0863470692cc56425006ecb)
2013-10-29 16:19:04 -04:00
Kevin Mehall
209ff75af8 Remove duplicate MANDRILL_API_KEY setting
(imported from commit c71928b5bfd71e9fcda1f5d57e9c7b99361ce070)
2013-10-29 15:47:13 -04:00
Steve Howell
7cf66b5bb8 Allow users to upload avatars on the Settings page.
We were using Gravatar for user avatars, but now users can
upload their avatars directly to Zulip, and we will store
their avatar for them.  This removes the old Gravatar-related
interface and polling code.

This commit does not attempt to update the avatars in
messages that have already been loaded, either for the user
making the change or other users.

(imported from commit 301dc48f96f83de0136c93de57055638c79e0961)
2013-10-29 10:59:41 -04:00
Steve Howell
68942a8f3a Properly split up "Your Account" and "Notifications".
The "Your Account" and "Notifications" boxes on the Settings
page each had their own border and their own "Save changes"
button, but they were within the same form and sending to the
same back end point.

This commit creates a separate form and endpoint for each
of the two boxes.

(imported from commit 04d4d16938f20749a18d2c6887da3ed3cf21ef74)
2013-10-29 10:59:41 -04:00
Tim Abbott
56e9ad230e [manual] Move our deployment scripts to scripts/.
This will require updating the post-receive code on git.zulip.net to
work.

(imported from commit 2e51fa2d7b891c1138d3f22ae534cfb8a6cf174c)
2013-10-28 10:54:48 -04:00
Tim Abbott
0a44ba2a28 Move servers/configure-rabbitmq to scripts/setup/.
(imported from commit 2b4d5ccb88675447ae744fb985246211deef7486)
2013-10-28 10:54:48 -04:00
Steve Howell
09979252dd Add Steve's laptop to USING_RABBITMQ list.
(imported from commit 3527edd27e3f386eb8b668fe8ee7e3875282ead8)
2013-10-27 11:27:27 -04:00
Zev Benjamin
cdcbe61cb0 [manual] Change references to the humbug_test user, schema, and database to zulip_test
This requires no changes in production, but is tagged as manual to
remind developers that they need to edit and run the tools/migrate-db
script to fix up their local database instances.

(imported from commit fbf764fb61592ef994d6d2ad56edad65ff01f14b)
2013-10-26 04:16:28 -04:00
Zev Benjamin
7b6f6823a5 [manual] Change references to the humbug user, schema, and database to zulip
This commit must be simultaneously deployed on both staging and
prod0.  It also requires completely taking down the app.

To deploy these changes, do:
* check out this commit at /root/zulip on postgres0, postgres1, staging, and prod0

* stop the process_fts_updates job on postgres0 and postgres1
* stop the app on staging and prod0

* do a puppet apply on postgres0, postgres1, staging, and prod0
* move the new client certificates into place on staging and app
* move the new server certificates into place on postgres0 and postgres1
* reload the database config on postgres0 and postgres1 (this might
  actually require a restart)
* run tools/migrate-db on postgres0 as root

* do a deploy through this commit on staging and prod0
* start the process_fts_updates job on postgres0 and postgres1

* do a puppet apply on nagios

(imported from commit 819bdd14326c1425e2d3041a491a8ca3b9716506)
2013-10-26 04:16:27 -04:00
Waseem Daher
66f48288b4 Rename LOCALSERVER -> LOCAL_SERVER.
(imported from commit b3abdd10d54d2ad7a9c463af9a291d2e2127707f)
2013-10-25 17:37:06 -04:00
Leo Franchi
8e21d70afe Disable statsd on local server
(imported from commit cac2c95c2de18b613a0900fe1c04fd97963b9350)
2013-10-25 16:37:29 -04:00
Leo Franchi
c12a97577e Rework local_settings_template.py to be clear about what the user needs to do
(imported from commit f69ea0d57e885509abc70dc4f2bee6e36169691f)
2013-10-25 16:35:27 -04:00
Luke Faraone
238d19e1f1 Include a rest_dispatch in tornadoviews and call it from urls.py
(imported from commit aefe66b44be60b39d0f6bb3076502e964f1ed6bd)
2013-10-25 14:13:31 -04:00
Luke Faraone
e552307511 Send feedback to a queue to be forwarded to staging.
(imported from commit 4a9a1bfc6c95763a816263a726cc61b3ca90bf15)
2013-10-25 14:13:30 -04:00
Luke Faraone
81d7dd1fda [schema] Support for authenticating Deployments via the API.
Here we introduce a new Django app, zilencer. The intent is to not have
this app enabled on LOCALSERVER instances, and for it to grow to include
all the functionality we want to have in our central server that isn't
relevant for local deployments.

Currently we have to modify functions in zerver/* to match; in the
future, it would be cool to have the relevant shared code broken out
into a separate library.

This commit inclues both the migration to create the models as well as a
data migration that (for non-LOCALSERVER) creates a single default
Deployment for zulip.com.

To apply this migration to your system, run:
   ./manage.py migrate zilencer

(imported from commit 86d5497ac120e03fa7f298a9cc08b192d5939b43)
2013-10-25 14:13:30 -04:00
Leo Franchi
0bc5b13648 Fix Zulip username of email gateway bot
(imported from commit f24c5fd3fc5888bf7dc4d9501723777ad8447704)
2013-10-25 11:51:13 -04:00
Leo Franchi
a9b9c0ec7b Add Email Mirror default settings and error out with readable warning if not configured
(imported from commit adac321d09561b51277ec1e314a1d480dc01e0c8)
2013-10-25 11:51:13 -04:00
Leo Franchi
2d276179d0 Refactor email-mirror to handle running on any machine
(imported from commit 2971449ceaacb564770e66874fc095f77e68d445)
2013-10-25 11:37:34 -04:00
Kevin Mehall
3bad5acc71 Fix upload auth redirect for user_activity.
Use rest_dispatch for upload auth redirect so it doesn't send the
long URL to user_activity.

(imported from commit ab327bbd529412e43eee6d109f8550180544dbbb)
2013-10-24 17:33:12 -04:00
Kevin Mehall
0a3a22cb3d Support authenticated upload URLs.
Trac #1734

This is implemented by bouncing uploaded file links through a view
that checks authentication and redirects to an expiring S3 URL.

This makes file uploads return a domain-relative URI. The client converts
this to an absolute URI when it's in the composebox, then back to relative
when it's submitted to the server.

We need the relative URI because the same message may be viewed across
{staging,www,zephyr}.zulip.com, which have different cookies.

(imported from commit 33acb2abaa3002325f389d5198fb20ee1b30f5fa)
2013-10-24 17:01:06 -04:00
Zev Benjamin
c13114c7e3 [manual] Change rabbitmq user to 'zulip'
Before deploying this commit, the following commands must be run:
  # rabbitmqctl add_user zulip $(tools/get-django-setting RABBITMQ_PASSWORD)
  # rabbitmqctl set_user_tags zulip administrator
  # rabbitmqctl set_permissions -p / zulip '.*' '.*' '.*'

(imported from commit 76d66fa6ac69aa39c91f55b9b5d5a954f9e94d73)
2013-10-24 16:54:27 -04:00
Zev Benjamin
ec21275b19 Move rabbitmq username into settings.py
(imported from commit f9a8f80f53b8774781b8558444e601567cb59835)
2013-10-24 16:40:38 -04:00
Waseem Daher
94c0b2fe70 Comparison page.
(imported from commit 4e2ce843bd066fd20e5840b2fd5cd2d991be02e7)
2013-10-24 16:22:00 -04:00
Leo Franchi
410ee44eb6 Send users push notifications when they miss messages
(imported from commit 6c54fe44a82c5796268e56d3f5577bf4cfc8163a)
2013-10-24 14:54:31 -04:00
Leo Franchi
a9bd2743f0 Initialize default settings if they are missing in the setting module
(imported from commit 97eb19c603436d460ad1832827dc3ec34c36bf31)
2013-10-24 10:19:24 -04:00
Tim Abbott
391de2d592 Add local settings template for local server instances.
(imported from commit 96f59e6a041992f5b2c467558a2fcc14a951a8a9)
2013-10-23 17:36:34 -04:00
Zev Benjamin
ca16644152 Add client-side SockJS wrapper
The wrapper handles our RPC protocol, authentication scheme, and
reconnections.

(imported from commit 1fed2d160582c235a32de80a80b3e451c13a7b1c)
2013-10-22 18:45:11 -04:00
Zev Benjamin
5979af3a45 [manual] Add asynchronous message sender via sockjs-tornado
New dependency: sockjs-tornado

One known limitation is that we don't clean up sessions for
non-websockets transports.  This is a bug in Tornado so I'm going to
look at upgrading us to the latest version:
https://github.com/mrjoes/sockjs-tornado/issues/47

(imported from commit 31cdb7596dd5ee094ab006c31757db17dca8899b)
2013-10-22 18:45:11 -04:00
Steve Howell
0662300e9e Remove /json/subscriptions/list endpoint.
(imported from commit db2c38f6b12ebee5250737a6cdd193863a4605af)
2013-10-22 15:51:19 -04:00
Zev Benjamin
06390de20a Load underscore before blueslip
blueslip depends on underscore.

(imported from commit 13a379d62bdb9824c052d8695ec030c69f41a5db)
2013-10-22 15:39:17 -04:00