Yesterday and today I've revisited several of the cases mentioned in this ticket (Z13522, Z16137, Z16688). I forced the currently enabled tests to rerun. Orderings got updated when appropriate (and only when appropriate), and the current orderings are consistent with the use of elapsed-time instead of CPU time. (That patch got landed a couple weeks ago.) Hopefully the use of elapsed-time will make the orderings more intuitive and useful from a practical point of view. I also think it will be helpful to give more attention to T330956 and T366660, which I'm planning to do when time permits.

As noted above – Cannot reproduce after forcing the tests to run again, and then force-refreshing the page.

Thanks, @Lucas_Werkmeister_WMDE ! Extremely helpful. Work is proceeding on some orchestrator code to retrieve lexemes via the Linked data interface. It would be useful to know if there are any guarantees about the structure of a Lexeme returned as JSON. For example, is it certain there will always be an id, language, lexicalCategory, and at least one lemma? Can there be a lexeme with no forms? If there are no forms, will there still be a forms property with an empty list? If there are zero claims, will there still be a claims property with an empty object?

The new DB table has been completed, including QA, and the Wikifunctions UI metrics table has been in use for some time already, so the needed info is now available. Closing.

That small discrepancy is gone today, and when it did occur there are several possible explanations, so closing this. Will continue to keep an eye on it.

Thanks, @cmassaro ! Just confirming, this would be a Promise.all statement in the orchestrator, right?

The table contents look good and the table grows a reasonable amount every 24 hours as expected. The number of functions recorded in the table is consistently very close to the number of functions listed at https://www.wikifunctions.org/wiki/Special:ListObjectsByType/Z8 - but there usually seems to be few more functions in the table than on the wiki listing, which is a bit puzzling.

Created by mistake; not completed; not needed; invalid

I agree. It seems to me it should always just be the orchestrator time.

@GrounderUK - Thanks for noticing the double counting of the evaluation duration! yes, this appears to be a mistake in the logic of our front-end code, and I created T369788 for this.

Hi @GrounderUK and @99of9 - Thanks for weighing in. I've made a ticket (T369587) and implemented a patch for the change we discussed above: basing the ordering on runtime (i.e., duration, elapsed time) instead of CPU times. If everything goes smoothly it might get deployed this week; we'll see.

Hi @99of9 - Yes, after studying the Z13522 example, and also having some discussion with other team members, it's clear the issue is real. In fact, it looks like there are 2 questions that need attention: (1) Is the ordering strategy (as described at the doc page) what's needed and (2) Is the implementation of the ordering strategy correct? Until today I thought the answer to (2) was probably "yes", but there is indeed something out of joint with the Z13522 example (from the data displayed in the metadata dialogs, I expect the ordering to be different). I will continue to investigate question (2).

Hi @GrounderUK - Yes, that refers to connected tests. The algorithm considers only the currently connected implementations and tests.

Sounds like IPv6 addresses?

Oh, I see. Yes, that's right. Thanks!

@phuedx , @Ottomata - In our Data Lake table, I now see a number of different IP addresses, which is exciting; thanks! Note however, I see 3 values (out of only about 15 distinct values so far) that don't look like IP addresses. Each of them contains groups of hexadecimal characters separated by colons (6-8 groups with 3-4 characters in each group). Any idea why we would get this sort of value sometimes? Is this something that needs further attention?

Actually, I raise this question in conjunction with considering the possibility of retrieving wikidata content directly from WikiLambda. It's a question that has been raised in the caching design document.

Here's another important question from the Wikifunctions team perspective: Given that we are running a MediaWiki extension (WikiLambda), is there a way for us to arrange for WikiLambda to retrieve Wikidata content (lexemes in particular) without crossing a network boundary?

Hi @GrounderUK - Thanks for making this ticket and providing all the details, and apologies for not commenting sooner here. I did spend some time looking at the initial example above (Z13522), and I formed a hypothesis about the behavior in that case, but I didn't have certainty so I didn't provide info at that time. (It's hard to have complete certainty about these orderings at present, without turning on more detailed logging.) Also, I wrote a doc page outlining how the implementation list ordering happens.