Wikipedia:Bots/Requests for approval/Demibot

Demibot

Operator: Demize (talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)

Time filed: 20:33, Friday November 21, 2014 (UTC)

Automatic, Supervised, or Manual: Automatic

Programming language(s): Python

Source code available: On Github

Function overview: Generate indexes of talk page archives

Links to relevant discussions (where appropriate):

Edit period(s): Daily (more often if discussion believes it's warranted)

Estimated number of pages affected: 3207 (based on current configuration and logging)

Exclusion compliant (Yes/No): No

Already has a bot flag (Yes/No): No

Function details: Builds a list of all pages transcluding {{User:HBC Archive Indexerbot/Optin}} (currently only in the Talk, User Talk, Wikipedia Talk, File Talk, and WikiMedia Talk namespaces, although I will extend that to other namespaces when the bot is operational) and generates archive indexes according to the specification of HBC Archive Indexerbot. Specifically:

Finds and parses the Optin template on each page
Builds a list of all pages that fit the mask specified on the page
Iterates through that list and finds different thread with a regex, then does the following for each thread:
- Stores the title of the topic
- Estimates the number of replies
- Finds the earliest and latest times in the thread, stores those as well as the difference between them
- Generates a link to the thread
Grabs either the default or a specified template and generates the talk page archive index from this
Writes the talk page archive index to the page specified in the Optin template (this is the only edit the bot will make aside from to its own log page)
Logs its actions to User:Demibot/log

Discussion

I've been writing the bot already, and it's mostly functional for my own talk page, but before I write anything that makes edits (even to my own talk page index) I wanted to bring this to BRFA. The code is a bit of a mess right now, partly because it's my first time writing Python, but it's functional. It doesn't generate the index from a template yet, but the functionality is there; I just need to write a parser for the template. This bot replaces the now-inactive HBC Archive Indexerbot and the inactive Task 15 on Legobot, neither of which appear to be coming back any time soon. If anybody has any questions, I'll be glad to answer them. If anyone has any concerns or suggestions, I'll be glad to hear them. demize (t · c) 20:33, 21 November 2014 (UTC)[reply]

This is much needed functionality to the extent that I was considering coding it and requesting permission to run from Their Majesties. I'm sure it will have wide ranging support.

Note: You can of course run it on your own talk pages, even before trial approval.

All the best: Rich Farmbrough, 15:35, 22 November 2014 (UTC).

Note: This bot appears to have edited since this BRFA was filed. Bots may not edit outside their own or their operator's userspace unless approved or approved for trial. AnomieBOT ⚡ 18:12, 24 November 2014 (UTC)[reply]
It did, and I tagged the page it created for deletion (it's been deleted) and fixed the problem that made it create a page that it shouldn't have. I've also run it on my own talk page, and it generates the archive index excellently, in my opinion, with the one flaw that sections without any timestamps have a first comment date of December 31, 9999 and a last comment date of January 1st, 1900. I'm thinking that I'll just make it output dashes instead of dates and durations when there are no timestamps, that'll be a bit better. demize (t · c) 19:00, 24 November 2014 (UTC)[reply]
Just an update on the current status of the bot: As per above, I have tested this on my own talk page. I have also made a version of the bot that should work on all talk pages. That's been pushed to GitHub, but the changes haven't yet been pulled to the Tools Labs server. What happens next with the bot is up to the BAG. demize (t · c) 22:56, 24 November 2014 (UTC)[reply]
((BAGAssistanceNeeded)) As it's been a week since I filed the request with no comment from a BAG member, I'm requesting BAG assistance. I've been making some changes to the code on Github over the past week, but there's very little I can do now without approval for a trial and a trial scope. demize (t · c) 18:20, 28 November 2014 (UTC)[reply]
(Non-BAG member comment) Demize, any specific reason why the bot isn't marked as exclusion compliant? APerson (talk!) 00:19, 30 November 2014 (UTC)[reply]
Strictly speaking, it isn't. It's opt-in rather than opt-out; it doesn't look for the {{bots}} template at all, since if someone wanted to remove it from their page they could remove the opt-in template. It's still just as easy to stop from editing your pages, just not technically exclusion compliant. demize (t · c) 04:50, 30 November 2014 (UTC)[reply]

Approved for trial. Please provide a link to the relevant contributions and/or diffs when the trial is complete. Definitely useful. Please do not do a full run but instead trial against one or two users talk page archives, such as mine! (which should already have the HBC template, if not then just add it :) ). Post links once complete! ·addshore· ^{talk to me!} 15:15, 4 December 2014 (UTC)[reply]

Thanks! I'll get this done tomorrow. I'll run it against mine and yours, and if another user or two volunteers their page I can run it against theirs as well. demize (t · c) 15:23, 4 December 2014 (UTC)[reply]

@Demize: Feel free to index my talk page ;) (iirc, I don't have HBC template, so please add it!) — Revi 15:41, 4 December 2014 (UTC)[reply]

Trial complete. Fixed a few issues that popped up (turns out my attempts to catch exceptions threw a few, and I added some other things without testing them... they're fixed now). It took longer than I expected due to real life and my need to figure out what on earth the masks for -revi's talk page archives could be. The mask option really wasn't designed for monthly archives, it turns out. That aside, the bot worked as expected once I ironed out the bugs. demize (t · c) 21:03, 9 December 2014 (UTC)[reply]

(edit conflict) DePiep brought up quite a valid point on my talk page regarding the Optin template just now (User talk:Demize#Duplicate_parameters). The duplicate parameters are an issue, and an easily correctable one if I want to make this bot not entirely compatible with the current operating instructions. I could make the masks all be specified as one parameter, with multiple masks separated by semicolons (and this could be done while preserving the ability to specify them as multiple parameters). If there's any way to exclude the template this bot uses from CAT:DUPARG, then that could be done as well. Certainly an issue to consider. demize (t · c) 01:24, 10 December 2014 (UTC)[reply]

In this edit, the bot/botowner added a template with multiple duplicate parameters |mask=. I commented that out. This is about CAT:DUPARG. User talk:Demize described that this is part of the bot, but I think we should not expect multiple parameters (that being undocumented). DePiep (talk) 01:17, 10 December 2014 (UTC)[reply]