Skip to content
This repository has been archived by the owner on May 17, 2024. It is now read-only.

[False Positive] Xbox Live Achievement Domains blocked #161

Closed
Bryantdl7 opened this issue Apr 16, 2020 · 14 comments
Closed

[False Positive] Xbox Live Achievement Domains blocked #161

Bryantdl7 opened this issue Apr 16, 2020 · 14 comments
Labels
broken-site Valid block breaks a legit site wontfix This will not be worked on

Comments

@Bryantdl7
Copy link

Bryantdl7 commented Apr 16, 2020 •
edited

Thank you for using my hosts lists. I appreciate feedback on this project, but I do expect you to put effort into your request. At the end of the day, this is my list and I maintain it for my usage. I'll address reasonable requests, but I cannot make everyone happy. This means you might have to whitelist a few things on your own. A list that breaks nothing is a list that blocks nothing.

  • False positive?
    Yes - a few of the domains on your list are stopping Xbox One's from uploading achievement properly.

The second I unblocked these domains, the xboxes began notifying me of achievements from months ago (I have been using your list for a few months now)

Here's the domains I have it narrowed down to, I didn't check the list to see which ones arent present on your https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt list.

Here are the domains, this has been crazy to track down!:

v20.events.data.microsoft.com - on your list
watson.telemetry.microsoft.com - on your list
web.vortex.data.microsoft.com - on your list
v10.events.data.microsoft.com - on your list

=========================================
Domains not on your list, but may help some sorry soul having this problem in the future DuckDuckGo'ing the solution. Make sure all these domains are allowed if you use multiple lists:

attestation.xboxlive.comcert.mgt.xboxlive.com
ctldl.windowsupdate.comdef-vef.xboxlive.com
device.auth.xboxlive.comeds.xboxlive.com
help.ui.xboxlive.comlicensing.xboxlive.commicrosoft.com
notify.xboxlive.comsettings-win.data.microsoft.com
title.auth.xboxlive.comtitle.mgt.xboxlive.com
v10.vortex-win.data.microsoft.com
www.msftncsi.com
xbox.ipv6.microsoft.com
xboxexperiencesprod.experimentation.xboxlive.com
xflight.xboxlive.comxkms.xbolive.com
xsts.auth.xboxlive.com
v20.events.data.microsoft.com
watson.telemetry.microsoft.com
web.vortex.data.microsoft.com
v10.events.data.microsoft.com
@lightswitch05
Copy link
Owner

I didn't check the list to see which ones arent present

So... I’m expected to do that for an issue that your having, which may or may not have anything to do with my list because you won’t look to see if they are in there or not?

As my issue template says: I do expect you to put some effort into your request.

@Bryantdl7
Copy link
Author

Bryantdl7 commented Apr 17, 2020 via email

@Bryantdl7
Copy link
Author

Bryantdl7 commented Apr 17, 2020 •
edited

Here are the domains present on your list, original post updated to note this as well:

v20.events.data.microsoft.com
watson.telemetry.microsoft.com
web.vortex.data.microsoft.com
v10.events.data.microsoft.com

Most of these domains are mentioned in the following forum post: https://answers.microsoft.com/en-us/xbox/forum/all/local-cache-is-full-stats-and-achievements-are-no/222d572a-56d3-45a1-97bc-b907c007e2dd

Additionally, Watson telemetry, in the world of windows is used to upload crash logs for when things like programs (games in this case) crash. That is helpful for developers to fix problems with their games, I am unsure how this relates to the overall problem of achievements other than that, but it was part of the long journey of slowly unblocking sites denied by my Xboxes.

Xbox has an annoying feature where you can not change a DHCPD-set IPv6 DNS server to a different value. I am certain others using your list will encounter this again in the future if they use pihole in DHCP mode, with IPv6 enabled, as the Xbox One prefers IPv6 when available.

Apologies for being vague in my original post.

@lightswitch05
Copy link
Owner

Thanks for the updated report @Bryantdl7 - I'm very busy at the moment, but I will look at this as soon as I have some time.

@lightswitch05 lightswitch05 added the broken-site Valid block breaks a legit site label Apr 17, 2020
@Bryantdl7
Copy link
Author

Bryantdl7 commented Apr 17, 2020 via email

@XhmikosR
Copy link

I'd be very careful with removing the aforementioned domains before extensive research.

Just my 2 cents.

@Bryantdl7
Copy link
Author

Bryantdl7 commented Apr 18, 2020 via email

@lightswitch05
Copy link
Owner

I added both *.telemetry.microsoft.com and *.events.data.microsoft.com on 2019-11-06 in 9dd786a8

Additionally, Watson telemetry, in the world of windows is used to upload crash logs for when things like programs (games in this case) crash. That is helpful for developers to fix problems with their games

Unfortunately diagnostic data like crash logs and tracking/analytics data is generally vacuumed up by the same tools and then separated out server side. Also, crash logs often contain sensitive information like PII. So my general rule of thumb is to block it all and let people whitelist whatever they decide is trustworthy. I prefer opt-in.

Just to see what some other list maintainers are doing, I searched the lists I have on my pihole for the given hosts, and came across these:

* v20.events.data.microsoft.com
    * https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
* watson.telemetry.microsoft.com
    * https://raw.githubusercontent.com/StevenBlack/hosts/master/data/StevenBlack/hosts
    * https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
    * https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt
    * https://raw.githubusercontent.com/michaeltrimm/hosts-blocking/master/_hosts.txt
    * https://raw.githubusercontent.com/vokins/yhosts/master/hosts
    * https://hosts-file.net/ad_servers.txt
    * https://raw.githubusercontent.com/bigdargon/hostsVN/master/hosts
    * https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
    * https://raw.githubusercontent.com/jerryn70/GoodbyeAds/master/Hosts/GoodbyeAds.txt
* web.vortex.data.microsoft.com
    * https://raw.githubusercontent.com/StevenBlack/hosts/master/data/StevenBlack/hosts
    * https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
    * https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt
    * https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
* v10.events.data.microsoft.com
    * https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
    * https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt

The results are really all over the place. Given the auto-expand nature of my list, and that I have telemetry.microsoft.com blocked - its not surprising that I have some of the better sub-domain coverage compared to some of the other lists. But it also could be some other list maintainers have decided to whitelist some subdomains. Its interesting that Steven Black's list has v10c.events.data.microsoft.com in it, but not v10.events.data.microsoft.com.

Alright... so the reason I'm showing that a bunch of other lists block these too: If I chose to move them to the aggressive list, its very likely most people will still have to whitelist them. Particularly since Steven's list is a default PiHole list.

I hate that I'm breaking Xbox for you and that it took a long time for you to figure it out. I really appreciate you sharing your results back with me. I'm always saying that I appreciate feedback and it is true. I make a lot of mistakes. In this case however, we know that Microsoft is using these hosts to do tracking, data mining, segmenting, and then targeting people with that data.

At this time I'm going to leave the list blocked. But lets leave the ticket open to make it easier for people to find. I may even add a section to the README that links here.

@lightswitch05 lightswitch05 added the wontfix This will not be worked on label Apr 18, 2020
@lightswitch05 lightswitch05 pinned this issue Apr 18, 2020
@Bryantdl7
Copy link
Author

Bryantdl7 commented Apr 18, 2020 via email

@liamengland1
Copy link

The domains are already listed on https://discourse.pi-hole.net/t/commonly-whitelisted-domains/212. How did this take you three months to figure out?

@beerisgood
Copy link

Just to see what some other list maintainers are doing, I searched the lists I have on my pihole for the given hosts, and came across these:

Problem is that many people only copy domains from other, without think or read about it.

It doesn't make sense to use a OS which you don't trust. Also Windows provide disable sending personal info. At minimum level, only "needed" data are send.

Of course privacy "experts" think they're better then official documentation from Microsoft and telemetry is always bad, which is totally nonsense.
Sadly this would never change.

I already reporting a lot of these as false positive at different lists but i guess you know what most dev's answer. Only few make the right decision :(

@lightswitch05
Copy link
Owner

lightswitch05 commented Apr 18, 2020 •
edited

Hey @beerisgood, the decision was made and I provided my line of thought. Your welcome to voice your opinion that the blocks are too aggressive, but leave it at that. There is no cause to try and belittle people, no one here is claiming to be an expert. If you do not like my list that I maintain for my own personal use then please do not use it.

@llacb47 some people have a harder time tracking down troublesome blocks then you do 😀

@Obihoernchen Obihoernchen mentioned this issue May 10, 2020
Closed
@lightswitch05 lightswitch05 mentioned this issue Jul 27, 2020
Closed
@lightswitch05
Copy link
Owner

I added a section to the README about this: https://github.com/lightswitch05/hosts#common-issues

@bremich bremich mentioned this issue Aug 6, 2020
Closed
@soredake
Copy link

This also breaks authorization in "your phone" app from microsoft.

@lightswitch05 lightswitch05 mentioned this issue May 21, 2021
Closed
@lightswitch05 lightswitch05 unpinned this issue Aug 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
broken-site Valid block breaks a legit site wontfix This will not be worked on
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@XhmikosR @lightswitch05 @soredake @Bryantdl7 @beerisgood @liamengland1