Page MenuHomePhabricator

Add an Image: filtering by suggestion "kind" or "confidence"
Open, MediumPublic

Description

User Story:

As a newcomer editing Wikipedia, I want to receive task suggestions that are fairly accurate and structured, so that I can get started and successfully edit on a mobile device.

Documentation:

https://www.mediawiki.org/wiki/Platform_Engineering_Team/Data_Value_Stream/Data_Gateway#Image_Suggestions
https://www.mediawiki.org/wiki/Help:Growth/Tools/Add_an_image
https://www.mediawiki.org/wiki/Growth/Personalized_first_day/Structured_tasks/Add_an_image

Background:

Structured tasks provide new editors with machine generated suggestions and structured the edit in a way that helps more new account holders edit constructively (1).

We tested initial suggestions with our pilot wikis to ensure the tasks provide good suggestions the majority of the time. Task onboarding indicates that suggestions aren't always correct, and that's why editors are asked to review the suggestions. In other words, we don't expect the suggestions to be 100% accurate, but they should be good suggestions ~70% of the time.

However, as we looked into scaling "add an image" to more wikis, we completed an additional round of evaluation:
T366925: Evaluate image suggestions for a new set of Wikipedias.

When reviewing these suggestions it seemed that certain suggestion "kinds" were less likely to be good suggestions. The istype-depicts and istype-commons-category suggestions were often lower quality. After removing these task kinds from the suggestion pool, we notice suggestions improve.

The downside of removing these suggestions is that the task pool is reduced dramatically.

Open questions:
  • Should we remove istype-depicts and istype-commons-category task kinds from certain wikis or all wikis?
  • Should these suggestions be removed from both the article-level and section-level tasks?
  • Should we allow communities to adjust this within Community Configuration?
  • Should we allow communities to adjust the confidence level for "add an image" tasks?
Acceptance Criteria:

TBD