Wikidata:SPARQL query service/WDQS backend update/June 2024 scaling update

Hello all!

The feedback period for our WDQS Graph Split proposal has come to an end. Many thanks to all people who sent comments, your contribution is invaluable!

We’ve incorporated most comments and proposals into our final set of rules for the graph split. The main proposals (including some that were rejected) were:

• Duplicate properties in both graph (wd:P*) does not seem necessary and won't be done
• The list of types of publications that identify what is a scholarly article have been improved, see the final list of items here
• It was discussed whether sitelinks should inform the nature of the split or not; this idea was not incorporated because it might make it harder to understand what is where
• Discussions and investigations regarding items that define multiple instance of (P31) which might be ambiguous, it appears that it might not affect a lot of items and that the solution might be to disambiguate these instances by creating separate entities (see the Clinical Trials section of the Talk Page).
• Re-thinking how scholarly articles are modelled was raised, especially by identifying the nature of the publication using a separate property rather than using instance of (P31). This idea should probably be explored and discussed by the wikicite community, it does not affect the nature of the split but could be a nice criteria to take into consideration in the future.

We are now working on implementing the appropriate tooling to manage this split, including a new way of processing the Wikidata dumps for an initial load, modification to the update pipeline to support the graph split, and additional automation. We hope to have new SPARQL endpoints that are live updated with the graph split by the end of June. This timeline is probably slightly optimistic, we’ll let you know when those are ready.

Once the new SPARQL endpoints that are live updated with the graph split are available, we will provide a 6 months transition period, during which the current endpoint (query.wikidata.org/sparql) will keep serving the full graph. Once that transition is over, query.wikidata.org will only serve the main graph. Queries that need federation will need to be rewritten. You can ask for help in Wikidata:Request_a_query to ask for query rewrites.

Thank you all for your help and support!

Guillaume