Page MenuHomePhabricator

Record baseline statistics about syntax highlighting (CodeMirror) use
Closed, ResolvedPublic1 Estimated Story Points

Description

We want to track how much the syntax highlighting function is used. To understand this we propose tracking, if possible:

  • how often the syntax highlighting 'highlighter' button is clicked to turn it on and off VE 2017 wikitext editor and 2010 source code editor
  • number of people have it activated (as saved in settings) in the VE 2017 wikitext editor and 2010 source code editor
    • Join user preferences: usecodemirror x { visualeditor-newwikitext, usebetatoolbar } (Logic table is slightly more complicated because other editors take precedence.)
  • number of edit sessions that have it activated in the VE 2017 wikitext editor and 2010 source code editor
  • if possible, number of people with related gadgets and scripts enabled:
    • wikEd
      • User preference: gadget-wikEd
    • Extension:CodeEditor
      • User preference: usecodeeditor
    • User:Remember_the_dot/Syntax_highlighter, Benutzer:Schnark/js/syntaxhighlight
      • Not easy to check script usage.

Similar to T258920: Collect on-going numbers related to VE template dialog and TemplateWizard use, collect on-going numbers to create a baseline and also see if this changes in the future with our work. Segment by:

  • VE 2017 wikitext editor and 2010 source code editor
  • Wiki, use the same four wikis to compare patterns of use: en, de, fa, tr

Open questions

  • Is this already being tracked? If not, is it possible to track now or would we need to create the infrastructure for it?

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Lena_WMDE set the point value for this task to 5.Aug 12 2020, 3:07 PM

We can capture the preference toggling by adding the preference name to an inclusion list in WikimediaEvents PrefUpdateInstrumentation::PROPERTY_TRACKLIST.

In the original ticket they also mention the related gadgets and some usage numbers T101246: Highlighted wikisyntax while editing articles [AOI] - @Lena_WMDE could be useful to gather the current gadget usage numbers and compare with the CodeMirror hidden preference?

In the original ticket they also mention the related gadgets and some usage numbers T101246: Highlighted wikisyntax while editing articles [AOI] - @Lena_WMDE could be useful to gather the current gadget usage numbers and compare with the CodeMirror hidden preference?

Thanks for the pointer, these are also user preferences so it's easy to collect them along with the CodeMirror usage counts.

Change 628317 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/WikimediaEvents@master] Collect data about CodeMirror preference usage

https://gerrit.wikimedia.org/r/628317

Preliminary counts,

image.png (423×500 px, 37 KB)

It looks like wikEd is twice as popular as the CodeMirror highlighter. To be continued...

Note to self, both the VE 2017 wikitext editor and the 2010 source code editor share the usecodemirror preference.

Moving to review so that we can start collecting data.

Lena_WMDE changed the point value for this task from 5 to 2.

Change 631229 had a related patch set uploaded (by Krinkle; owner: Awight):
[mediawiki/extensions/WikimediaEvents@wmf/1.36.0-wmf.11] Collect data about CodeMirror preference usage

https://gerrit.wikimedia.org/r/631229

Change 628317 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Collect data about CodeMirror preference usage

https://gerrit.wikimedia.org/r/628317

Done once we verify that stats are collected.

Lena_WMDE changed the point value for this task from 2 to 0.

The patch isn't merged yet, it's waiting on the MW-1.36-notes (1.36.0-wmf.12; 2020-10-05; NEVER DEPLOYED) branch which has been blocked.

We'll probably just wait another week for the train to continue normally.

Here's a query to verify data collection:

select * from event_sanitized.prefupdate where year=2020 and month=10 and event.property = 'usecodemirror' limit 1

Change 631229 abandoned by DannyS712:
[mediawiki/extensions/WikimediaEvents@wmf/1.36.0-wmf.11] Collect data about CodeMirror preference usage

Reason:
I guess this wasn't deployed? No longer needed since all sites are now at 1.36.0-wmf.14

https://gerrit.wikimedia.org/r/631229

Data is present!

awight changed the point value for this task from 0 to 1.

We need to also keep the mentioned gadget preferences in the sanitized preference update event stream.

awight set Final Story Points to 3.

Unchecking the "usecodemirror" history, it was difficult to join this data with choice of editor so we're reimplementing using a different strategy.

Change 641983 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/WikimediaEvents@master] Track changes to the wikiEd syntax highlighter preference

https://gerrit.wikimedia.org/r/641983

awight changed Final Story Points from 3 to 5.

Change 642024 had a related patch set uploaded (by Andrew-WMDE; owner: Andrew-WMDE):
[mediawiki/extensions/CodeMirror@master] [WIP] Record usage statistics for syntax highlighting

https://gerrit.wikimedia.org/r/642024

thiemowmde moved this task from Doing to Sprint Backlog on the WMDE-QWERTY-Sprint-2020-11-18 board.
thiemowmde added a subscriber: Andrew-WMDE.

Change 642024 merged by jenkins-bot:
[mediawiki/extensions/CodeMirror@master] Record usage statistics for syntax highlighting

https://gerrit.wikimedia.org/r/642024

WMDE-Fisch claimed this task.

Change 645345 had a related patch set uploaded (by Andrew-WMDE; owner: Andrew-WMDE):
[analytics/reportupdater-queries@master] [WIP] Process EventLogging events for CodeMirror

https://gerrit.wikimedia.org/r/645345

We need to make a minor decision for this metric,

number of edit sessions that have it activated in the VE 2017 wikitext editor and 2010 source code editor

If the setting is toggled during the session, then how do we count which sessions are enabled and which are not?
a) by the first setting of the session?
b) by the last setting of the session?
c) any session where codemirror was enabled at some point?

Just noting that we will *not* deduplicate global users for this requirement,

number of people have it activated (as saved in settings) in the VE 2017 wikitext editor and 2010 source code editor

In the past we've deduplicated because we wanted the overall number of users with a preference, across all wikis. In this case, we're getting a count per wiki, and it would make sense that users have different preferences on each wiki due to the local context.

After some conversation:

We'll do this,

c) any session where codemirror was enabled at some point

And this is fine,

Just noting that we will *not* deduplicate global users for this requirement

Change 645345 merged by Mforns:
[analytics/reportupdater-queries@master] Process EventLogging events and tally preferences for CodeMirror

https://gerrit.wikimedia.org/r/645345

Change 641983 abandoned by Thiemo Kreuz (WMDE):

[mediawiki/extensions/WikimediaEvents@master] Track changes to alternative editor preferences

Reason:

Ticket is closed as resolved for a long time.

https://gerrit.wikimedia.org/r/641983