Page MenuHomePhabricator

Message blob for {module} should have been preloaded
Open, Needs TriagePublicPRODUCTION ERROR

Description

Error
normalized_message
Message blob for {module} should have been preloaded
Impact

Seeing over 1,000 warnings (very noisy) to the resourceloader channel on logstash and possibly related events in the events list.

Notes

Details

Request URL
https://www.wikidata.org/w/load.php?lang=*&modules=*&only=*&raw=*&skin=*

Event Timeline

Doesn’t seem to be specific to Wikidata – https://logstash.wikimedia.org/goto/2e673c3e690194a2f98280393e7975c4 shows >100k occurrences of the error, but mainly on Wikipedias. (It’s also very “spike-y”.)

image.png (266×665 px, 16 KB)

(Note: The above logstash link is based on the mediawiki-errors backlog, but these messages don’t actually end up on an error channel – I removed that filter.)

Questions to think about:

  • When looking for the normalized message across mediawiki-errors Logstash dashboard, does it affect specific domains, hosts, data centers, or {module} values more than others?
  • How long do the peaks last?
  • Does it have literally zero ocurrences outside the peaks, or is there a continuous trickle at a lower level as well?
  • Do the peaks happen at a regular interval?
  • Do the recurring peaks align with any recurring SAL entries?