Talk:Wikimedia Foundation Annual Plan/2024-2025/Product & Technology OKRs

Questions

1. Volunteering on the Wikimedia projects should feel rewarding. We also think that the experience of online collaboration should be a major part of what keeps volunteers coming back. What does it take for volunteers to find editing rewarding, and to work better together to build trustworthy content?

2. The trustworthiness of our content is part of Wikimedia’s unique contribution to the world, and what keeps people coming to our platform and using our content. What can we build that will help grow trustworthy content more quickly, but still within the quality guardrails set by communities on each project?

3. To stay relevant and compete with other large online platforms, Wikimedia needs a new generation of consumers to feel connected to our content. How can we make our content easier to discover and interact with for readers and donors?

Just make it work. Interactive content, discoverability of the expensive tools we build and forget, sister projects integration (instead of burying them) and unbreaking the things that are broken would be a good start. We defined this 8 years ago, and is still pending. -Theklan (talk) 18:06, 29 February 2024 (UTC)

4. In an age where online abuse thrives, we need to make sure our communities, platform, and serving system are protected. We also face evolving compliance obligations, where global policymakers look to shape privacy, identity, and information sharing online. What improvements to our abuse fighting capabilities will help us address these challenges?

5. MediaWiki, the software platform and interfaces that allow Wikipedia to function, needs ongoing support for the next decade in order to provide creation, moderation, storage, discovery, and consumption of open, multilingual content at scale. What decisions and platform improvements can we make this year to ensure that MediaWiki is sustainable?

WE1 Contributor experience

Both experienced and new contributors rally together online to build a trustworthy encyclopedia, with more ease and less frustration.

The paragraph is so vague, that it could mean one thing and the opposite. When you say "we will make improvements to critical workflows for experienced contributors, we will lower barriers to constructive contributions for newcomers, and we will invest in ways that volunteers can find and communicate with each other around common interests."... what do you mean? Are we going to build something modern or just make some patches about whatever the sentence means? -Theklan (talk) 17:59, 29 February 2024 (UTC)

WE2 Encyclopedic content

Increased growth in encyclopedic content is achieved through tools and resources that are easier to access, reuse, improve, and can reliably ensure trustworthiness of the content as per policies and guardrails used on Wikimedia projects.

When you say "Tools and resources (both technical and non-technical) that are available for contributors to use for their needs can be made more discoverable, and reliable." are we saying that the tools are going to be part of our infrastructure, and directly accessible from the editing or reading section, or does it mean a completely different thing? -Theklan (talk) 18:01, 29 February 2024 (UTC)

WE3 Consumer experience

A new generation of consumers arrives at Wikipedia to discover a preferred destination for discovering, engaging, and building a lasting connection with encyclopedic content.

1. @OVasileva (WMF): I object to talking about "consumers" as if Wikipedia was another commercial project pushing out "content". I object to talking about encyclopedia articles as "content" as if they were a commercial project. In addition, this all seems very vague, a sort of bland corporate statement. What are you actually proposing to do? If you don't propose to do anything new, that's good too, but say that instead of talking cagily about "work[ing] across platforms to adapt our experiences and existing content, so that encyclopedic content can be explored and curated by and to a new generation of consumers and donors. " Because that's just buzzwords and corporatese.

I don't want to sound brusque, but I'm finding it difficult to understand what you're actually proposing here. 🌺 Cremastra (talk) 00:52, 21 February 2024 (UTC)

Hi @Cremastra - thank you for writing this out! We are definitely still working on the language here and appreciate your feedback. I think the reason that we went with "consumers" is because we wanted to widen the audience from our usual "readers" to include people that might learn or use Wikipedia in different ways - for example, people who are more visual learners or people who use assistive technologies and might not necessarily be reading, but consuming the content in a different way. Readers didn't seem to fit this wider group. I agree that the con to "consumers" or "knowledge consumers" is that is that it sounds like a much more generic term. We're definitely welcome to suggestions on what other terms might fit better (for example, for a while we considered "learners" but that also felt a bit too vague). Do you have any ideas around this?

In terms of the proposal itself - what we want to do is make it easier to learn on Wikipedia by making it easier to discover content (here meaning articles, but also other encyclopedic content such a lists, images, etc). This would most likely mean thinking of different ways people can find articles or topics of interest, working with communities to curate content, and to present existing curated content in engaging and easy to find ways to readers and other consumers. Hope that makes more sense - I realize this is a pretty wide umbrella as well, so can definitely provide more examples too if that would be helpful! OVasileva (WMF) (talk) 13:47, 21 February 2024 (UTC)

@OVasileva (WMF): Thanks for the reply, this is very helpful.

• More support for accessibility projects like Spoken Wikipedia would be good. Because articles are always getting changed and updated, there's a lot of work to be done there. This is far-fetched, but maybe a tool that allows contributors to that project to easily record their work? Specifically, I was thinking a tool that can be activated on any article, (probably from a drop-down menu), that allows editors to record and upload their reading of the article there, via the tool's interface.
  • I've created a visual mock-up of what this could in theory look like at User:Cremastra/sandbox (all the buttons and links are dummies).

Thanks, 🌺 Cremastra (talk) 21:10, 21 February 2024 (UTC)

Thanks for the idea and mock @Cremastra! It aligns with the idea of the objective of making it easier for the community to curate content and make it easier to discover for others. I'll share it with some of the other folks that will be working on this objective to get their thoughts as well. OVasileva (WMF) (talk) 09:39, 22 February 2024 (UTC)

The wording is really worrying, not only from the "consumers" side, also from the "donors" word. Your team was the one that hidded on purpose our sister projects, making them virtually impossible to be found. What are the plans to "making our content more easy to discover and interact with" if the sister projects are now impossible to be found? Is there any plan to make them more visible and evident? Theklan (talk) 18:03, 29 February 2024 (UTC)

WE4 Trust & Safety

Improve our infrastructure, tools, and processes so that we are well-equipped to protect the communities, the platform, and our serving systems from different kinds of scaled and directed abuse while maintaining compliance with an evolving regulatory environment.

WE5 Platform evolution

Evolve the MediaWiki platform and its interfaces to better meet Wikipedia's core needs. What does "This includes continuing work to define our knowledge production platform, strengthening the sustainability of the platform, a focus on the extensions/hooks system to clarify and streamline feature development, and continuing to invest in knowledge sharing and enabling people to contribute to MediaWiki." mean? This can be one thing, the other, or the opposite. Which are the exact plans in the next years to make our platform less obsolete? -Theklan (talk) 18:05, 29 February 2024 (UTC)

WE6 Developer Services

Technical staff and volunteer developers have the tools they need to effectively support the Wikimedia projects.

SDS1 Essential metrics

Our decisions about how to support the Wikimedia mission and movement are informed by high-level metrics and insights.

SDS2 Experimentation platform

Product managers can quickly, easily, and confidently evaluate the impacts of product features.

• User:TTaylor (WMF), I think that evaluating the effect shouldn't stop at "launch", which this wording unfortunately (but I think unintentionally?) implies. I think we need phab:T89970 microsurveys as a low-level, ongoing survey, so that every launch has not only an easy opportunity to provide simple feedback, but also a baseline to compare it against. WhatamIdoing (talk) 23:32, 20 February 2024 (UTC)

You are correct, WhatamIdoing, it is not meant to imply that impact evaluation stops at launch. The generalized capability we would like to have is commonly described as A/B testing, where a feature under development is shown to a small % of users and compared against a control or alternative group. Some product teams have done experiments like this before, with significant effort required to set up, run, and evaluate them. We want to streamline the effort involved so that more product teams can run experiments like this. TTaylor (WMF) (talk) 14:33, 21 February 2024 (UTC)

Thanks for the quick reply, @TTaylor (WMF). I love a good mw:A/B test, but I'm not sure that it's enough.

For one thing, they don't usually capture sentiment changes. The A/B test can see that I'm still making 20 edits/day, but it can't see that I'm mad.

For another, they can produce spurious results. At least among core community members, I can be in the "control" group and still have my day disrupted by a product launch. When we launched the visual editor in 2013, people not using the visual editor were having problems, because they had to clean up a lot of unwanted whitespace changes and other problems that the visual editor produced at that time. If you start the A/B test at launch, you might see some of the "A" group stop editing because the product is bad, and some of the "B" group down tools in protest over the mess that the remaining "A" group is making (or spending their day arguing outside the mainspace, or writing AbuseFilters, or whatever it is that isn't what they would normally do). In such a scenario, an A/B test could show the two groups as "equal", so supposedly no harm's being done, when what's really equal is the number of people who quit editing because of the problematic change.

What I do like about the idea of improving the A/B infrastructure is that it should make it easier to do partial rollouts. Particularly for appearance-related changes at the larger wikis, deploying a new thing to 10% of registered editors, and then waiting to see what problems are revealed before expanding to a larger group, is gentler than deploying the new thing to 100% of registered editors at the same time. WhatamIdoing (talk) 06:51, 22 February 2024 (UTC)

WhatamIdoing, your last paragraph nailed exactly the outcome we'd eventually like to achieve. I agree that experiments can produce unwanted outcomes or side effects, or be designed in a way that produces invalid results. We are planning for one of the key results for this objective to be focused on experimentation guidelines, to try an avoid the kind of outcome you described. I don't want to pretend this is easy or we'll always get it exactly right, but we do want to get better and more efficient at running experiments so that we can learn much faster what works and what doesn't. TTaylor (WMF) (talk) 13:07, 22 February 2024 (UTC)

Imagine, two years ago the design team made an A/B test for the Zebra design, and the deployment is still pending. We can't wait for years for every change, it doesn't make sense. Theklan (talk) 18:34, 29 February 2024 (UTC)

FA1 Test hypotheses

Test hypotheses. Sorry, I'm reading "Bring Wikimedia's knowledge and pathways to contribution to popular third-party platforms?" and I really don't understand what this means. Why aren't we improving OUR OWN PLATFORM, instead of thinking on third-party platforms? Why aren't we investing on disruptive knowledge content creation and everything read so vague? Can the meaning of this sentence be explained, please? -Theklan (talk) 17:56, 29 February 2024 (UTC)

PES1 Efficiency of operations

Make the foundation's work faster, cheaper, and more impactful.