IPv4/IPv6 Issues

[ScriptTiger]

Does anyone have any issues related to IPv4/IPv6? Some examples might be IPv6 queries not getting blocked, an IPv6 system not successfully querying on an IPv4 network, etc. We have talked about this numerous times on Steven Black's repo, but I just wanted to open an issue here to discuss it as it relates specifically to Windows systems running this script.

If you are encountering an issue related to this, please first try manually installing one of the IPv4_IPv6 hosts files from the below link:
https://scripttiger.github.io/alts/

Whether the above solves your problem or not, please comment as to what your issue was before and what happened after. If this is a relevant issue for us, I will add support for it in the script.

[abhijithvijayan]

I have issues with ipv4
My ip assigning doesn't work correctly

It should assign IP 10.1.1.x instead it's assigning 169.254.164.x and thus internet is down

[abhijithvijayan]

and the hosts are not updated correctly I guess
See the difference at the top itself

[ScriptTiger]

@abhijithvijayan, the difference in your hosts file is intentional and defined by the default ignore.txt. If you would like it to look identical, you can remove all of the lines in the ignore.txt not starting with #. The reason why this is intentional in the defaults is because this script is meant to work with pre-existing hosts file entries and does not overwrite your entire hosts file. This allows users to manage custom entries independently from my script without having to worry about them being lost every time my script updates their black list. If your hosts file does not have any pre-existing entries and is essentially empty, or at least doesn't have any conflicts, as I said before, you can just clear out the uncommented lines in the ignore.txt to update everything without any filters.

As far as your IP assignment issue, are you running your own local DHCP server or do you believe this to be an adapter issue?

[abhijithvijayan]

@ScriptTiger The issue is after I update the hosts with the script/HostMan, and then after a reboot the assigned local IP for the network LAN/Wifi is somehow changing to as I said 169.254.164.x which is the reason I can't connect to the internet. I'm connected to a PPPoE network by the way. And when I reset the hosts file to the Windows default host the issue fades and the IP assigns correctly.
See the screenshot, the local ip is 10.1.1.x

And I tried to exclude the hosts file in Kaspersky Antivirus just to make sure it's not the antivirus that's causing the trouble.

[ScriptTiger]

I am actually very intrigued by this problem. You say it happens after every time you use the script/HostMan, but have you tried installing a hosts file manually?

https://scripttiger.github.io/hosts-packages/
https://scripttiger.github.io/alts/

I am curious if it is the hosts file entries or if it is a problem with the script/HostMan. The script/HostMan include other "helper" instructions to help load and reset your system to work with a newly loaded hosts file, so I am curious if it is really these instructions that may be causing the problem and not the hosts file data itself. If you copy and paste the hosts file data manually and insert it into your hosts file, do you still encounter the same problem?

[abhijithvijayan]

@ScriptTiger and I will try this once again later. Thanks.

[abhijithvijayan]

I uninstalled the antivirus and found it out it was not the issue with it. Its exactly with the hosts file. When I manually remove aa the contents, the internet is back.

See the ip 169.254.164.x

[ScriptTiger]

Have you tried playing with the "compressed" or "mcompressed" formats?
https://scripttiger.github.io/alts/

Normally this format is to resolve DNS cache issues with Windows machines, but it is also conceivable that if this problem is due to something shutting down due to an overload of sorts, this format may also work. Since I don't know what the issue is exactly, it's possible the hosts file is simply too many lines for your system and the networking service is getting caught up somewhere. Compressing it should work if this is the case. Since nobody can really explain why Windows even has a DNS cache issue with hosts files like Steven Black's that are comparably tiny compared to what the system specs should be able to handle, I am very curious to see if this is in fact another such peculiarity.

[ScriptTiger]

Let me know if the compressed formats work or don't for you. I'll be adding this feature into the script soon and would like to have a list of all the use cases.

[abhijithvijayan]

Sure

[ScriptTiger]

@abhijithvijayan, I just implemented a new compression level feature into the script. Try setting yours to 9 and see if it helps.

[abhijithvijayan]

I have a feeling that updating hosts somehow triggers the auto configuration issue
Maybe it's not about the hosts file after all. I'll implement the compression and see if that triggers the same issue again and if so, I'll fix the auto configuration thing.

btw nice work on the compression

I think the issue is fixed after the compression. Internet is fine now. I'll let you know.

[ScriptTiger]

Using a static assignment would definitely work around having DHCP issues, but I am still curious what the problem is.

So you have a modem that has a PPPoE connection to your ISP, who also provides you with DNS and DHCP. Then your Windows machine is directly connected to the modem and pulls DHCP directly from your ISP and not from a Wi-Fi router or something else in the middle, is that correct? Or do you have a Wi-Fi router that has a DNS resolver and DHCP server?

If you have a Wi-Fi router with a DHCP server, I would recommend logging into it, google the default credentials if you haven't before, and make sure the DHCP server is configured correctly.

If your Windows machine is directly connected to a modem with no Wi-Fi router and their DHCP server is assigning you a 10.1.1.X address, 10.0.0.0/8 is a 24-bit block IPv4 private network, which means your ISP is not giving you a public IP address and you are being NATed somehow, either sharing a single IP address or a pool of IP addresses with other people using your ISP. This also means this issue is not related to IPv6 at all.

Can you try commenting out all of the below lines just to test?

Change this:

127.0.0.1 localhost.localdomain
127.0.0.1 local
255.255.255.255 broadcasthost
::1 localhost
::1 ip6-localhost
::1 ip6-loopback
fe80::1%lo0 localhost
ff00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
0.0.0.0 0.0.0.0

To this:

# 127.0.0.1 localhost.localdomain
# 127.0.0.1 local
# 255.255.255.255 broadcasthost
# ::1 localhost
# ::1 ip6-localhost
# ::1 ip6-loopback
# fe80::1%lo0 localhost
# ff00::0 ip6-localnet
# ff00::0 ip6-mcastprefix
# ff02::1 ip6-allnodes
# ff02::2 ip6-allrouters
# ff02::3 ip6-allhosts
# 0.0.0.0 0.0.0.0

If your ISP is not giving you a public IP address, it's not operating within standard best practices for ISPs. ISPs might do this to save costs because they can pay their higher tier telecommunications provider for a small network and share a small pool of public IPs with more customers. In most developed countries this is actually a federal crime because of legislation like the USA PATRIOT Act requires government agencies like the FBI to be able to quickly and easily trace online connections in the event of "terrorist" and/or other criminal activity. If every customer is not given a public IP address, it means government agencies have an extra step of tracing the public IP to the ISP, and then cross-reference the logs of the ISP (NAT, DHCP, etc.) to find out who exactly made a particular connection. This extra step can make tracing less accurate and possibly make someone less legally liable of committing a crime if it can't be accurately proven. If this is the case and they are not following best practices, it's also possible there is another type of communication trying to be established here that the above lines might be interfering with.

[abhijithvijayan]

@ScriptTiger Thanks for the concern. After compressing hosts to 9th level itself solved my issue. Now everything is working perfectly.

and I have a windows machine that is connected to a wireless modem that pulls DHCP server directly from the ISP who provides me with Google DNS and their DHCP server.
Compressing made the host file of 1.5MB which was 2.3 MB without compression. I guess size has probably something to do with the Auto configuration IP allocation issue..

As of now everything works perfectly. Thank you for making it happen. and btw I'm an Indian citizen.

[ScriptTiger]

Nice! Thank you for bringing up another use case for compression, as these problems are apparently not very well documented.