CentralAuth uses a central wiki for logins the wiki where the login is started and the central login wiki communicate via the DB and other stores. Some of the hardest-to-catch errors happen when (in Wikimedia heterogeneous deployment) a change is deployed to some wikis but not to the central login wiki, or is deployed to the central login wiki but not to some other wikis, so that communication breaks. The effect of a partially-deployed patch is hard to reason about, and hard to test locally. Some sort of CI automation for it would be great.

The shape it would have to take would be something like this:

• have two MediaWiki installs, one for a normal wiki, one for the central login wiki
• check out CentralAuth master to the normal wiki and master + the patch to be tested to the central login wiki
• run some basic Selenium tests (login, signup)
• check out CentralAuth master + the patch to the normal wiki and master to the central login wiki
• run the tests again