Page MenuHomePhabricator

LinkSearch results should use as much of the path as is provided, not simply search by domain
Closed, ResolvedPublic

Description

Author: mike.lifeguard+bugs

Description:
So you have a wildcard on the front, which is great.

But on the other end, if you have some path after the domain name, it is ignored: [[Special:Linksearch/*.linkedin.com/in/nguyenta]] is equivalent to [[Special:Linksearch/*.linkedin.com]], but shouldn't be. It should list only the links which follow the /in/nguyenta path (like Special:Prefixindex).


Version: unspecified
Severity: normal

Details

Reference
bz15218

Related Objects

StatusSubtypeAssignedTask
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
DuplicateLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedAntoine_Quhen
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedLadsgroup
ResolvedBUG REPORTLadsgroup

Event Timeline

bzimport raised the priority of this task from to Low.Nov 21 2014, 10:17 PM
bzimport set Reference to bz15218.
bzimport added a subscriber: Unknown Object (MLST).

Currently we can't cleanly do both a subdomain wildcard *and* a path prefix, because of the way the index is done.

A URL like this:

http://sub.example.com/path/file.html

gets transformed to this indexable form:

http://com.example.sub./path/file.html

This lets us do a subdomain-wildcard search:

LIKE 'http://com.example.%'

or a path prefix search:

LIKE 'http://com.example.sub./path/%'

Both of these are a straight prefix match, which is very efficiently indexed.

But if we wanted to search both a subdomain wildcard *and* a path, our query would look like this:

LIKE 'http://com.example.%/path/%'

This can get us some efficient lookups for the first wildcard, but then the second wildcard has to be matched within the results, potentially very slow depending on the number of matches. For instance assuming there are lots of links to other Wikipedia pages, a search like this:

LIKE 'http://org.wikipedia.%/w/api.php%'

would end up being very inefficient, since we wouldn't get the indexing boost on the path part.

Now in theory at least a good query optimizer might be able to speed that up a lot, but my impression is that MySQL isn't that smart about it right now and would just not bother touching the indexes for the second wildcard.

mike.lifeguard+bugs wrote:

So if you specify the prefix you can search within the path as well? That I did not know, and will make life much easier. Still, it'd be nice to have both I guess.

Extensions is now part of MediaWiki core (1.14alpha) -> changing product and component

De-assigning this from me. Would be nice, but don't know a good clean way to index it in straight MySQL.

Created attachment 5850
linksearch documentation enhancements

attachment kk ignored as obsolete

Darn, if one makes an attchement, the comments that one types into this box here are blown away.

I was trying to say
Why not just let * by itself work. I want to find all links on my
small wiki, why force me to use one query for each possible TLD?!

Then part of my attachment wouldn't be needed.

*Bulk BZ Change: +Patch to open bugs with patches attached that are missing the keyword*

sumanah wrote:

improving MessagesEn.php documentation for linksearch

This is just jidanni's patch, but rebased against current trunk.

-'linksearch-text' => 'Wildcards such as "*.wikipedia.org" may be used.<br />
-Supported protocols: <tt>$1</tt>',
+'linksearch-text' => 'Wildcards such as "*.wikipedia.org" may be used. Need at least a TLD, e.g., *.org<br />
+Supported protocols: <tt>$1</tt> (but don\'t enter them below!)',

Attached:

  • Bug 32671 has been marked as a duplicate of this bug. ***

Actually this will be automatically done in any wiki that goes to READ_NEW mode on externallinks migration. Nothing more needed.

That's already the case in testwiki now. Shall I just close this ticket or wait for all wikis to be on read new?

Re: Tech News - What wording would you suggest as the content?
And does anything related need to be updated within https://www.mediawiki.org/wiki/Help:Linksearch ?
Thanks!

Sorry, I missed this ping:

LinkSearch and its API counterparts now search for all of the URL provided in the query. It used to be up to the first 60 characters. This feature was requested fifteen years ago.

Edit mercilessly