All Systems operational.

Systems

Response Times

Incidents

2017-03-13 17:42:34 UTC resolved Elasticsearch
Elasticsearch cluster availability has been affected

One of our Elasticsearch cluster's availability had been affected for a few hours on March 13, 2017. On investigating, the cause has been a memory leak in environments that use Docker v1.9 and Linux Kernel 4.3 environments. https://github.com/docker/docker/issues/18295 is an open issue.

Update 2017-03-13 17:42:42 UTC

A good follow-up step should be to upgrade the host and docker environments to the most stable recent release.

2017-03-08 12:02:00 UTC resolved Users Service
Increased errors: Accounts API

Investigating increased number of errors in the accounts API. Service is unable to connect to the metrics backend.

Update 2017-03-08 12:30:59 UTC

Had an issue with the underlying server nodes. Migrated the Users Service to a new node cluster.

2017-02-17 08:17:17 UTC resolved Users Service
User login issue

A section of users logging in with Github are affected by a 500 error when they log in, the issue was reported on Feb 13th.

Update 2017-02-17 08:24:42 UTC

A recent change in the GET /user endpoint introduced a reliance on refresh tokens, something Google supports in their OAuth but Github doesn't.

We have reverted to not rely on refresh tokens within the GET /user endpoint and the user login behavior should be back to normal.