Page MenuHomePhabricator
Create Task
Maniphest T249126

WikibaseQualityConstraints fails to connect to db for federated properties on production
Closed, ResolvedPublic
Actions

Description

If WiibaseQualityConstraints is enabled on production commons, a wbcheckconstraints call results in the following error

2020-04-01 11:50:51 [XoSAGwpAEJgAAbBdfm0AAADT] mw1394 commonswiki 1.35.0-wmf.25 exception ERROR: [XoSAGwpAEJgAAbBdfm0AAADT] /w/api.php?action=wbcheckconstraints&format=json&formatversion=2&uselang=en&id=M75908279&status=violation%7Cwarning%7Csuggestion%7Cbad-parameters   Wikimedia\Rdbms\DBConnectionError from line 1418 of /srv/mediawiki/php-1.35.0-wmf.25/includes/libs/rdbms/loadbalancer/LoadBalancer.php: Cannot access the database: Unknown error (10.64.48.150) {"exception_id":"XoSAGwpAEJgAAbBdfm0AAADT","exception_url":"/w/api.php?action=wbcheckconstraints&format=json&formatversion=2&uselang=en&id=M75908279&status=violation%7Cwarning%7Csuggestion%7Cbad-parameters","caught_by":"mwe_handler"}
[Exception Wikimedia\Rdbms\DBConnectionError] (/srv/mediawiki/php-1.35.0-wmf.25/includes/libs/rdbms/loadbalancer/LoadBalancer.php:1418) Cannot access the database: Unknown error (10.64.48.150)
  #0 /srv/mediawiki/php-1.35.0-wmf.25/includes/libs/rdbms/loadbalancer/LoadBalancer.php(934): Wikimedia\Rdbms\LoadBalancer->reportConnectionError()
  #1 /srv/mediawiki/php-1.35.0-wmf.25/includes/libs/rdbms/loadbalancer/LoadBalancer.php(901): Wikimedia\Rdbms\LoadBalancer->getServerConnection(integer, string, integer)
  #2 /srv/mediawiki/php-1.35.0-wmf.25/extensions/WikibaseQualityConstraints/src/ConstraintRepository.php(35): Wikimedia\Rdbms\LoadBalancer->getConnection(integer, array, string)
  #3 /srv/mediawiki/php-1.35.0-wmf.25/extensions/WikibaseQualityConstraints/src/CachingConstraintLookup.php(41): WikibaseQuality\ConstraintReport\ConstraintRepository->queryConstraintsForProperty(Wikibase\DataModel\Entity\PropertyId)
  #4 /srv/mediawiki/php-1.35.0-wmf.25/extensions/WikibaseQualityConstraints/src/ConstraintCheck/DelegatingConstraintChecker.php(410): WikibaseQuality\ConstraintReport\CachingConstraintLookup->queryConstraintsForProperty(Wikibase\DataModel\Entity\PropertyId)
  #5 /srv/mediawiki/php-1.35.0-wmf.25/extensions/WikibaseQualityConstraints/src/ConstraintCheck/DelegatingConstraintChecker.php(441): WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker->getConstraintsToUse(Wikibase\DataModel\Entity\PropertyId, NULL)
  #6 /srv/mediawiki/php-1.35.0-wmf.25/extensions/WikibaseQualityConstraints/src/ConstraintCheck/DelegatingConstraintChecker.php(374): WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker->checkConstraintsForMainSnak(Wikibase\MediaInfo\DataModel\MediaInfo, Wikibase\DataModel\Statement\Statement, NULL, array)
  #7 /srv/mediawiki/php-1.35.0-wmf.25/extensions/WikibaseQualityConstraints/src/ConstraintCheck/DelegatingConstraintChecker.php(346): WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker->checkStatement(Wikibase\MediaInfo\DataModel\MediaInfo, Wikibase\DataModel\Statement\Statement, NULL, array)
  #8 /srv/mediawiki/php-1.35.0-wmf.25/extensions/WikibaseQualityConstraints/src/ConstraintCheck/DelegatingConstraintChecker.php(154): WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker->checkEveryStatement(Wikibase\MediaInfo\DataModel\MediaInfo, NULL, array)
  #9 /srv/mediawiki/php-1.35.0-wmf.25/extensions/WikibaseQualityConstraints/src/Api/CheckingResultsSource.php(53): WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker->checkAgainstConstraintsOnEntityId(Wikibase\MediaInfo\DataModel\MediaInfoId, NULL, array, array)
  #10 /srv/mediawiki/php-1.35.0-wmf.25/extensions/WikibaseQualityConstraints/src/Api/CheckConstraints.php(172): WikibaseQuality\ConstraintReport\Api\CheckingResultsSource->getResults(array, array, NULL, array)
  #11 /srv/mediawiki/php-1.35.0-wmf.25/includes/api/ApiMain.php(1591): WikibaseQuality\ConstraintReport\Api\CheckConstraints->execute()
  #12 /srv/mediawiki/php-1.35.0-wmf.25/includes/api/ApiMain.php(523): ApiMain->executeAction()
  #13 /srv/mediawiki/php-1.35.0-wmf.25/includes/api/ApiMain.php(494): ApiMain->executeActionWithErrorHandling()
  #14 /srv/mediawiki/php-1.35.0-wmf.25/api.php(84): ApiMain->execute()
  #15 /srv/mediawiki/w/api.php(3): require(string)
  #16 {main}

Details

SubjectRepoBranchLines +/-
Inject correct load balancer into ConstraintRepositorymediawiki/extensions/WikibaseQualityConstraintsmaster+11 -1
Inject load balancer factory into ConstraintRepositorymediawiki/extensions/WikibaseQualityConstraintsmaster+57 -20
Customize query in gerrit

Event Timeline

Cparle created this task.Apr 1 2020, 3:39 PM
Restricted Application added a project: Wikidata. · View Herald TranscriptApr 1 2020, 3:39 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Lucas_Werkmeister_WMDE claimed this task.Apr 1 2020, 3:40 PM
gerritbot added a comment.Apr 1 2020, 3:44 PM

Change 585260 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/WikibaseQualityConstraints@master] Inject load balancer factory into ConstraintRepository

https://gerrit.wikimedia.org/r/585260

gerritbot added a project: Patch-For-Review.Apr 1 2020, 3:44 PM
Lucas_Werkmeister_WMDE added a comment.Apr 1 2020, 3:47 PM

I think I know what caused this, but I don’t really know how to test it, given that apparently not even the beta cluster has a DB setup complicated enough to trigger this problem. I guess if nobody finds anything wrong with the above change, we merge, backport, and hope for the best?

We should at least be able to try “can I load constraints” in shell.php before enabling constraint checks in the config, though.

gerritbot added a comment.Apr 2 2020, 8:47 AM

Change 585440 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/WikibaseQualityConstraints@master] Inject correct load balancer into ConstraintRepository

https://gerrit.wikimedia.org/r/585440

gerritbot added a comment.Apr 2 2020, 3:25 PM

Change 585260 abandoned by Lucas Werkmeister (WMDE):
Inject load balancer factory into ConstraintRepository

Reason:
Ie547a623d9 got 2ed instead

https://gerrit.wikimedia.org/r/585260

gerritbot added a comment.Apr 2 2020, 5:23 PM

Change 585440 merged by jenkins-bot:
[mediawiki/extensions/WikibaseQualityConstraints@master] Inject correct load balancer into ConstraintRepository

https://gerrit.wikimedia.org/r/585440

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.27; 2020-04-07).Apr 2 2020, 6:00 PM
Maintenance_bot removed a project: Patch-For-Review.Apr 2 2020, 6:10 PM
Lucas_Werkmeister_WMDE added a comment.Apr 3 2020, 10:00 AM

Should we backport the fix or just wait for the next train?

Cparle added a comment.Apr 3 2020, 11:13 AM

It's fine to wait - I enabled constraints on commons, but then removed it when we came across this error, so it's not causing any problems. Let it ride the train, then I'll swat the config on Thursday next week and hopefully it'll have worked

Lucas_Werkmeister_WMDE added a comment.Apr 3 2020, 1:33 PM

I was hoping we could test the change in shell.php first, but it doesn’t look like that works, because the extension isn’t loaded on Commons at the moment and late-loading extensions isn’t supported.

It looks like testcommonswiki and testwikidatawiki are in different db shards, so we might be able to test this on testcommonswiki, but apparently the extension isn’t loaded there either?

Cparle added a comment.Apr 3 2020, 1:45 PM

Nope, sorry. Maybe could SWAT that on Monday and test the change then?

Lucas_Werkmeister_WMDE added a comment.Apr 3 2020, 1:54 PM

Well, we could only test on Tuesday (train reaches group0), but I’d feel better with that option, if it’s okay for you.

Lucas_Werkmeister_WMDE closed this task as Resolved.Sep 21 2020, 11:55 AM

I’m pretty sure this is resolved by now.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits