Hitting the first "Unable to start, out of quota for memory, memory" errors for jobs.

@Raymond_Ndibe what is the status? I see you merged something. I just tested and the problem still exists.

In T174633#9828797, @JeanFred wrote:

Oh, geez, we never followed-up on that >_>. Adding at least MIT sounds good to me.

No activity for 5 years. I'm just going to go ahead and mark this as declined. If people want to use old symlinks, that's fine. No added value in breaking those.

In T192733#9774988, @Xqt wrote:

Not related to Pywikibot:
https://codesearch.wmcloud.org/pywikibot/?q=trunk%7Crewrite%7Ccompat%7Cpywikipedia&files=&excludeFiles=&repos=

Repo is still very active, just moved to github long time ago, see https://github.com/multichill/toollabs .

For the deletions: https://commons.wikimedia.org/w/index.php?title=Special:Log&page=&tagfilter=&type=&user=Multichill&wpFormIdentifier=logeventslist&wpdate=&wpfilters%5B0%5D=newusers&dir=prev&offset=20230817183454%7C342854881

Deleted:
https://commons.wikimedia.org/w/index.php?title=MediaWiki:Wm-license-cc-by-4.0-au-text&action=edit&redlink=1
https://commons.wikimedia.org/w/index.php?title=MediaWiki:Wm-license-cc-by-4.0-nz-text&action=edit&redlink=1
https://commons.wikimedia.org/w/index.php?title=MediaWiki:Wm-license-cc-by-sa-3.0-IGO-text&action=edit&redlink=1

Report is up. Also had the bot do the easy fixes.

My guess would be that the code at https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/blob/main/tjf_cli/loader.py?ref_type=heads#L23 is tripping up (or not used).

Working again. Just have to check in a couple of hours if I don't have a ton of queued jobs.

Works again, see https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.noclaims/SAL and https://www.wikidata.org/wiki/Special:Contributions/NoclaimsBot

webservice --backend=kubernetes php7.4 start

In T343131#9618275, @Jarekt wrote:

After some discussion we retired Template:License_template_tag which was used to track files without licenses and was redundant to Extension:CommonsMetadata which was introduced latter. That should eventually remove 100M template links.

In T244663#9428323, @Frostly wrote:

I think including the year would be still beneficial for clarity. See T271968 above for the task to auto update the year.

Don't sabotage the grid jobs. That wouldn't be appreciated.

Don't sabotage the grid jobs. That wouldn't be appreciated.

Don't sabotage my grid jobs. That wouldn't be appreciated.

Don't sabotage my grid jobs. That wouldn't be appreciated.

Don't sabotage my grid jobs. That wouldn't be appreciated.

Asked Daniel to comment.

In T161934#9104305, @Yann wrote:

@Aklapper: I mean what are we waiting for?

I'm being bold here: I don't think we have community consensus for this and I see no effort to get it so I'm closing this as declined. Feel free to change if you think this is wrong.

In T344207#9093963, @Aklapper wrote:

The code is custom on-wiki content, so it's up to the Commons community to look into this...

In T343936#9083076, @Lucas_Werkmeister_WMDE wrote:

We should probably handle the 500 error as a recoverable error (just like the socket time out) and retry.

Maybe the WDQS maintainers will disagree, but to me, automatically retrying a query that timed out and already consumed a significant amount of resources in doing so doesn’t sound like a good idea.

With the increased time out I was able to update https://www.wikidata.org/wiki/Wikidata:WikiProject_sum_of_all_paintings/Property_statistics ( https://github.com/multichill/toollabs/blob/master/bot/wikidata/painting_property_statistics.py )

In T343131#9076182, @Jarekt wrote:

About links from commons to commons: I think a lot of them come from c:Category:Language_link_templates (aka /lang templates) which are used by some old templates. For example Template:PD-US-no_notice/lang is transcluded by Template:PD-US-no_notice which is then transcluded to all the files using that template. /Lang templates are automatically generated or updated by template:LLE which uses external links to subpages instead of internal links, in order to help with some database table exploding in 2009. See discussion. A lot of transclusions of /lang templates in license templates was removed in 2013, because they were causing other issues, but not all of them.

Maybe we should retire those templates (I never found them useful) or change template:LLE back to create templates using internal links at the danger of blowing up some other table (template link table?).

Also in the 2009 discussion on the subject @Multichill mentioned that LLE uses external links because "Externallinks was hacked to not be included in a table if it points to the same site". If external links to Commons are stored now, than maybe we should re implement this old hack.

In T343131#9062058, @Ladsgroup wrote:

In T343131#9060994, @LucasWerkmeister wrote:

I was thinking of something that a template/module can add to a page automatically, but SDC probably also works for some use cases, that’s true.

Is WCQS operational? Can you do the queries like this now if people migrate data to SDC? If not, do you have an ETA for when that'd go online?

This is a bug. This causes bots to butcher up dates. Say for example a painting is sourced to be made between 1190 and 1210. The year will be set to 1200 with a precision of a millennium with earliest date 1190 and latest date 1210. The 1200 is used for sorting and other cases where things are just grouped together. This would be changed to 2000 with the current logic. This is not in line with https://www.wikidata.org/wiki/Help:Dates .

https://www.wikidata.org/w/index.php?title=Q118631862&diff=prev&oldid=1924443167 is what is supposed to happen

This task looks stale to me. Can it be closed?

In T336687#8851614, @dcaro wrote:

Yes it is, it should be fixed already, can you retry?

@dcaro is this related to T316544 ?

Example user who seems to be quite successful with this approach: https://commons.wikimedia.org/wiki/User_talk:Mutasim_Elmahadi#Your_Tell_Us_About_Her_on_ISA_2023_prize

Thanks for the update, see also https://www.wikidata.org/w/index.php?title=Wikidata%3AWikiProject_sum_of_all_paintings%2FMost_used_painting_properties&diff=1821925473&oldid=1658545667

Works for me again too. I noticed a spike on https://www.wikimediastatus.net/ for the "Wiki error responses" to 940 errors/second and a drop for "Successful edits".

@TheDJ why did you include @Kolossos and @Zache ? I think both are still active.
Note that @faidon left WMF this month, see https://meta.wikimedia.org/wiki/Special:Contributions/Faidon_Liambotis_(WMF) :-(

@Xqt : Still broken:

Added some of the missing subscribers.

In T319981#8320032, @Xqt wrote:

@Dvorapa: Is this assignment intentionally?

Not that tools-login.wmflabs.org in https://github.com/JeanFred/mirador-wrapper/blob/master/deploy/hosts is no longer functional. You can use login.toolforge.org or dev.toolforge.org

We discussed this a bit on Telegram/IRC (wikimedia-cloud) triggered by the grid to Kubernetes migration (as pasted above). I'll try to describe my use case a bit.

Thanks for your response. Let's close this one and move over to T249787

In T320822#8317452, @komla wrote:

We have toolforge-jobs for submitting jobs to Kubernetes. See: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework
Takea look at this use case continuity table: https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation#Use_case_continuity

I object to some of these subtasks being assigned to me as this is against https://www.mediawiki.org/wiki/Bug_management/Phabricator_etiquette : "Only manually assign a task to someone if they have given their prior agreement. It is up to developers (or their product managers) what they plan to work on." I asked in the Cloud Telegram channel @komla to unassign all these tasks.

70+ jobs, waiting for T320822

Or if you don't want to use templatelinks, you can also switch to categorylinks:

In T243602#8134984, @Umherirrender wrote:

The deletion of templatelinks is done via the job queue, because reparsing many pages takes some time. The counts are at 0 now.

Just wait until the jobqueue is processed, doing null edits is not a good idea on that high number - https://en.wikipedia.org/wiki/Wikipedia:Purge#Null_edit

@OlafJanssen is the latest victim at https://www.wikidata.org/wiki/Wikidata:SPARQL_federation_input