Page MenuHomePhabricator

Scale: default to mentorship questions from help panel
Closed, ResolvedPublic

Description

In T250235: Scale: pilot help panel with mentorship, we developed the ability for questions coming from the help panel to go to mentors instead of central help desks. In that task and in T272753: Scale: pilot help panel with mentorship in frwiki, bnwiki, arwiki, viwiki, we validated in our pilot wikis that the results from this change are acceptable. Therefore, we want this configuration to be the default for all wikis with the Growth features. We would also like to retain the ability for this to be configurable, so that in the future, some wikis could elect to send questions to help desks instead.

This may involves these lines of work:

  • Make this an option that wikis can configure via T274031 (not a blocker for this task)
  • Convert all Growth wikis to the new configuration.
  • Use this new configuration as the default for new wikis going forward.

This should not be done until it is announced to communities by the newsletter.

Event Timeline

@Urbanecm_WMF -- could you look this over and make sure I've written it correctly? Please make any needed changes, and then it is Ready for Development.

@Trizek-WMF -- does this all sound good to you? Please note that I think we should announce this change in an upcoming newsletter. If it's too late for the one we're translating now, I think that's okay, and we can save it for the next one.

@Urbanecm_WMF -- could you look this over and make sure I've written it correctly? Please make any needed changes, and then it is Ready for Development.

Thanks for the ping. This is not blocked on T274031: Provide custom editing interface for GrowthExperiments on-wiki configuration from the technical perspective; we can do this change now (but the wikis won't be able to turn it back off, unless they come to us and ask for the change). I'm not sure whether it's a product requirement to have wikis able to control this. If it is, T274031 should be a subtask of this task. If it is not, description should be updated.

We are also able to change the default for new wikis now, and change the current wikis later, after T274031 is completed -- if this is what we want, we'll probably want to split this to two tasks, one for changing the default, and second for changing all current wikis, since those two things can happen at different times.

@MMiller_WMF Could you confirm what do we want to do here? Thanks!

@Urbanecm_WMF -- yes, we should do the change now for all wikis (it is not blocked on T274031), and it should be the default for future wikis. Then, once we allow wikis to make configuration changes, this is one of the aspects they should be able to configure. Could you please update the task description here, create other needed tasks, and update the spreadsheet on T275086 to reflect all this?

Change 667529 had a related patch set uploaded (by Urbanecm; owner: Urbanecm):
[operations/mediawiki-config@master] Set wgGEHelpPanelAskMentor to true by default

https://gerrit.wikimedia.org/r/667529

Change 667531 had a related patch set uploaded (by Urbanecm; owner: Urbanecm):
[mediawiki/extensions/GrowthExperiments@master] Set GEHelpPanelAskMentor to true by default

https://gerrit.wikimedia.org/r/667531

@MMiller_WMF @Trizek-WMF I assume this would need some community communication. Should we wait for a next newsletter having this change? Or can I go forward with this one at any time?

Change 667531 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Set GEHelpPanelAskMentor to true by default

https://gerrit.wikimedia.org/r/667531

I can't remember.... What if there is no mentor list? Do we still need a fallback on the help desk?

If so, scaling would be a bit pointless, since we still need to figure out what is the help desk, and configure it.

Or do we simply remove the possibility from newcomers to post a question?

@Trizek-WMF @Urbanecm_WMF -- we should wait for the newsletter to be sent before making this change.

I can't remember.... What if there is no mentor list? Do we still need a fallback on the help desk?

If so, scaling would be a bit pointless, since we still need to figure out what is the help desk, and configure it.

Or do we simply remove the possibility from newcomers to post a question?

Here's how I think we should handle this:

  • Communities will be able to configure both a mentor list and a help desk.
  • If there is no mentor list, but there is a help desk, then the help panel falls back to the help desk.
  • If there is no mentor list and no help desk, then the help panel does not have the option for asking a question.

How does that sound?

Also, @Urbanecm_WMF, I want to make sure you make the additional tasks you mentioned above.

Communities will be able to configure both a mentor list and a help desk.

  • If there is no mentor list, but there is a help desk, then the help panel falls back to the help desk.
  • If there is no mentor list and no help desk, then the help panel does not have the option for asking a question.

How does that sound?

If the help desk part is entirely on their hands, should I still keep help desk configuration in the scaling tasks? TBH, it is often the most difficult part, and it may be complicated to work on the maintenance for this, since communities use multiple help desks configurations.

I think we should go for an option that only includes questions to mentor. I would push communities to have a mentor list, since we think a privileged relationship seems to be better than posting on the help desk. If we can remove the help desk from the configuration, it will really help on scaling.

Just in case, I want to clarify, the mechanics of working with questions in the ruwiki will not change after the implementation of this task? If changes are planned, then I need to consult with the community.

I think we should go for an option that only includes questions to mentor. I would push communities to have a mentor list, since we think a privileged relationship seems to be better than posting on the help desk. If we can remove the help desk from the configuration, it will really help on scaling.

I think we shouldn't forget about very small projects where you can't get enough mentors. And in large projects, the forum is preferable at first, since this allows you to track the emotional coloring of answers to questions on a general scale and correct it if it does not look very good. If we initially send all questions only to mentors, then this can prevent us from promptly responding to "bad answers".

[...]
If the help desk part is entirely on their hands, should I still keep help desk configuration in the scaling tasks? TBH, it is often the most difficult part, and it may be complicated to work on the maintenance for this, since communities use multiple help desks configurations. ¨

I'd prefer keeping it in the "new wiki template", until T273118: Help panel: Remove dependency on Help Desk title existing is resolved. Right now, in a no-mentor situation (also quite common for new wikis), help panel will fallback to help panel. If there's no help panel, it'll error out, and not show up at all.

  • Communities will be able to configure both a mentor list and a help desk.
  • If there is no mentor list, but there is a help desk, then the help panel falls back to the help desk.

✅ That's how it currently works.

  • If there is no mentor list and no help desk, then the help panel does not have the option for asking a question.

⚠ This needs T273118: Help panel: Remove dependency on Help Desk title existing to be solved first. Right now, help panel cannot work in a no-help desk scenario. Should we make T273118 a subtask of this one, or redefine?

If changes are planned, then I need to consult with the community.

This change is based on an experiment we did on multiple wikis. We found that it has no impact on newcomers to post a message to their mentor, and it really eases our effort to scale to more wikis. So the help panel will post messages to mentors' talk pages where a mentor list exists.

I agree that informing them about the change is required, but w do you think a community consultation would be needed there?

I think we shouldn't forget about very small projects where you can't get enough mentors.

To be honest, based on my experience across wikis, wikis without enough to be mentors won't have enough people to reply to questions to the help desk.

Wikis without enough mentors don't get the mentorship module. As a consequence, newcomers won't have an option to ask their questions., which is unfortunate, but much better than having questions stacking at a place where no one cares about.

And in large projects, the forum is preferable at first, since this allows you to track the emotional coloring of answers to questions on a general scale and correct it if it does not look very good. If we initially send all questions only to mentors, then this can prevent us from promptly responding to "bad answers".

another way to address this is to watch other mentors' talk pages. This way, you can cover people who take too much time to reply, or the ones who are away.

I'd prefer keeping it in the "new wiki template", until T273118: Help panel: Remove dependency on Help Desk title existing is resolved. Right now, in a no-mentor situation (also quite common for new wikis), help panel will fallback to help panel. If there's no help panel, it'll error out, and not show up at all.

Our goal is to remove our dependency to the help panel, in order to have better ways to go to more wikis. More wikis than I thought don't have an help desk (Esperanto is an example I came across recently).

  • If there is no mentor list and no help desk, then the help panel does not have the option for asking a question.

⚠ This needs T273118: Help panel: Remove dependency on Help Desk title existing to be solved first. Right now, help panel cannot work in a no-help desk scenario. Should we make T273118 a subtask of this one, or redefine?

Yes, it should be a subtask.

@Trizek-WMF I moved the new subtask to current sprint board, and uploaded a patch for it (as part of making myself familiar with help panel code, as I never worked with it before). Provided it is merged soon, we will be able to do the switch when newsletter is sent.

The newsletter has been sent. However, it is not a big deal if the change hasn't been done yet.

Has this been recently pushed to ptwiki as well? A user noticed the lack of significant traffic there in the past 3 days. I did manage to post a test question there though.

Has this been recently pushed to ptwiki as well? A user noticed the lack of significant traffic there in the past 3 days. I did manage to post a test question there though.

It works if you managed to write a message to your mentor using the help panel. Was it the case?

I don't have a lot of context on help panel flow. But my point there is with a recently created test account the help panel question went to the Help Desk. No other questions have arrived at the help desk in the past 4 days, though.

Messages written using the help panel are supposed to go to the mentor, not the help desk. @Urbanecm_WMF, has the change been done?

Regarding no questions coming to the help desk, we have noticed that the number of posts was fluctuating. Newcomers tend to post their messages to their mentor, hence this help panel change.

Messages written using the help panel are supposed to go to the mentor, not the help desk. @Urbanecm_WMF, has the change been done?

Not yet. I was waiting for an all-green from you. If this is ready, I can do it next week.

Change 667529 merged by jenkins-bot:
[operations/mediawiki-config@master] Set wgGEHelpPanelAskMentor to true by default

https://gerrit.wikimedia.org/r/667529

Mentioned in SAL (#wikimedia-operations) [2021-03-08T20:38:14Z] <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 5ce7b4602d2b109adfb86bef6795a4d07a1208b9: Set wgGEHelpPanelAskMentor to true by default (T275908) (duration: 01m 07s)

Checked on GrowthExperiments wikis.

For PM review
DONE (1) Danish wiki, Thai wiki and Croatian wiki do not have Mentorship module enabled and do not have Help panel displayed. Does it mean that they do not have Help desk?

(2) Also, Preferences still have the option "Enable the editor help panel" - should it work to disable "Ask your mentor" panel in namespaces where it's present.

[...]
(1) Danish wiki, Thai wiki and Croatian wiki do not have Mentorship module enabled and do not have Help panel displayed. Does it mean that they do not have Help desk?

That's my mistake... I'm pretty sure help panel did fallback to help desk when no mentor is configured, but apparently that broke (or my memory is bad). Let's fix this. Thanks for noticing!

Change 670021 had a related patch set uploaded (by Urbanecm; owner: Urbanecm):
[mediawiki/extensions/GrowthExperiments@master] Make help panel fallback to help desk if no mentor is available

https://gerrit.wikimedia.org/r/670021

[...]
(1) Danish wiki, Thai wiki and Croatian wiki do not have Mentorship module enabled and do not have Help panel displayed. Does it mean that they do not have Help desk?

That's my mistake... I'm pretty sure help panel did fallback to help desk when no mentor is configured, but apparently that broke (or my memory is bad). Let's fix this. Thanks for noticing!

I think that's the same as T276759: TypeError: title is undefined, FWIW.

Change 670021 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Make help panel fallback to help desk if no mentor is available

https://gerrit.wikimedia.org/r/670021

Change 670108 had a related patch set uploaded (by Urbanecm; owner: Urbanecm):
[mediawiki/extensions/GrowthExperiments@wmf/1.36.0-wmf.34] Make help panel fallback to help desk if no mentor is available

https://gerrit.wikimedia.org/r/670108

Change 670109 had a related patch set uploaded (by Urbanecm; owner: Urbanecm):
[mediawiki/extensions/GrowthExperiments@wmf/1.36.0-wmf.33] Make help panel fallback to help desk if no mentor is available

https://gerrit.wikimedia.org/r/670109

Change 670108 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@wmf/1.36.0-wmf.34] Make help panel fallback to help desk if no mentor is available

https://gerrit.wikimedia.org/r/670108

Change 670109 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@wmf/1.36.0-wmf.33] Make help panel fallback to help desk if no mentor is available

https://gerrit.wikimedia.org/r/670109

Mentioned in SAL (#wikimedia-operations) [2021-03-09T12:16:16Z] <urbanecm@deploy1002> Synchronized php-1.36.0-wmf.33/extensions/GrowthExperiments/: dbd6f0cb299bcfb6648b351e1476100fe669cc58: Make help panel fallback to help desk if no mentor is available (T275908; T273782) (duration: 01m 01s)

[...]
(1) Danish wiki, Thai wiki and Croatian wiki do not have Mentorship module enabled and do not have Help panel displayed. Does it mean that they do not have Help desk?

That's my mistake... I'm pretty sure help panel did fallback to help desk when no mentor is configured, but apparently that broke (or my memory is bad). Let's fix this. Thanks for noticing!

@Etonkovidova Hi, can you check again, please? For the record, help panel should always appear, even for wikis have no help desk and no mentorskip. The ask a question part will just not appear.

Thanks, @Urbanecm_WMF - re-checked dawiki, thwiki, and hrwiki - the Help panel is present. Also did some quick regression check - all seems to be fine.

I apologize for such a long answer :)

If changes are planned, then I need to consult with the community.

This change is based on an experiment we did on multiple wikis. We found that it has no impact on newcomers to post a message to their mentor, and it really eases our effort to scale to more wikis. So the help panel will post messages to mentors' talk pages where a mentor list exists.

I agree that informing them about the change is required, but w do you think a community consultation would be needed there?

Well, in my opinion, this is a rule of good form :) Which allows you to smooth out corners when communicating with the community. For example, now, since you changed the news at the last moment, I did not have time to translate it, and the old version of the message left the community. And even now I don't quite understand whether the mechanics for ruwiki have changed? During the consultation, we would definitely make sure that everyone is aware of the changes, and they know that the load will increase on them.

I think we shouldn't forget about very small projects where you can't get enough mentors.

To be honest, based on my experience across wikis, wikis without enough to be mentors won't have enough people to reply to questions to the help desk. When I was the admin of the Russian Wikibooks, I ran into the problem that the newbies did not understand that it is advisable to ask a question on the forum, that this forum exists and that you can work with it. With such a solution, we can lose some of the feedback for small communities.

Wikis without enough mentors don't get the mentorship module. As a consequence, newcomers won't have an option to ask their questions., which is unfortunate, but much better than having questions stacking at a place where no one cares about.

In fact, on our help forum, one of the most active participants does not want to sign up for the mentoring module. Perhaps a small wiki will be composed of such contributors.
Also keep in mind that some wikis have one forum for everyone, and the person following small projects will be watching the forum, not the mentor's talk page.

And in large projects, the forum is preferable at first, since this allows you to track the emotional coloring of answers to questions on a general scale and correct it if it does not look very good. If we initially send all questions only to mentors, then this can prevent us from promptly responding to "bad answers".

another way to address this is to watch other mentors' talk pages. This way, you can cover people who take too much time to reply, or the ones who are away.

In fact, it can be difficult, considering that we have about 30 mentors and their number will increase in the future.

I'm sorry that the time and constraints on our side didn't match your wiki's usual workflow. This is my fault: rushing on scaling to more wikis, I forced this simplification change for future configurations, without enough time to really consider the impact on all wikis.

We are working on easing the configuration of the Growth features for each wiki. In a meeting today, we have discussed about the opportunity to let communities decide of the use of the help desk. I see that you already commented on T274031: Provide custom editing interface for GrowthExperiments on-wiki configuration, which will certainly cover this.