Minor improvements to regex.list

[niklasea]

I would also have suggested blocking subdomains of IDNs (i.e. subdomains of xn--), but I see you had issues with Google Play. What kind of issues did you have and with what domains? The regex would be much better if it at least blocked the www subdomain as well to catch domains like www.ɡooɡle.com (in punycode: www.xn--oole-z7bc.com).

[mmotti]

Hi,

What are the reasons for removing underscores and lazy evaluation?

I believe, although I could be wrong, underscores are valid characters especially for subdomains? And the lazy evaluation can save rather a lot of steps in some cases which may save precious processing time.

I didn't personally have issues but I had issues reported presumably from international users where they were experiencing problems with Google play; I can't recall whether they specified if it was general access or downloading apps. I can certainly add the optional www subdomain to these though :-)

[niklasea]

As I wrote in the commit message, lazy evaluation is not possible with POSIX extended regex. I don't know how it gets parsed exactly, but it certainly does not do what you think it does.

With regards to underscores, I may be wrong on that front. I did some more reading and although underscore is valid in DNS records, AFAIU it is not valid in hostnames. I can't think of a situation where these regexes would be used to block DNS records that do not point at a web host. However, in case we do want to include underscores, I wonder why they were not included in every regex?

EDIT: So I actually did the legwork and I can see that, indeed, I do have names with underscores on my blocklist and some of them do resolve to web servers. So I'll concede that they should remain, but I wonder if they should be applied to every regex or if there was a reason why you used them only where you did?

[mmotti]

Ah, I see! Apologies I'm on my phone so I didn't go as far as looking at the commit comments. I will have to take a look into this as I tested them outside of grep originally. If this is the case then I shall remove them :-) I may remove the underscore matching too, as suggested, as tbh now that you mention it I haven't yet personally come across an example where they're used. I applied them originally only to those regexps that I thought they'd be most likely to appear. I will take a look at this a little later today.

[niklasea]

I see you replied via email. I edited my comment after I tested and confirmed that underscores can in fact be used and should be left in.

For reference, this seems to be the line that parses regexes in Pi-hole.

[mmotti]

@niklasea take a look through 32d97e8. I think it should cover everything in this pull request, in addition to moving the - within the character classes to the end to avoid syntax errors relating to selecting an invalid range.

[niklasea]

I think it looks good.

[mmotti]

Resolved in 32d97e8

Thanks @niklasea