Does anyone know of any off the shelf tool (online or offline) to find duplicates in several DNS blocklists and merge them into one?
Context: I am running AdGuard on one GL.iNet router with ~10 blocklists some of them pretty huge and most of the times the lists are updated the router comes to one halt while doing so, having to often times reboot it through the old power-off-and-on.
I would rather download the lists myself from time to time and merge them into one file but with duplicates extracted somehow.
You must log in or register to comment.
Isnt there a tool developed by the AdGuard team to handle exactly this?
Just looked through my files, look into this tool, it does exactly what you want: https://github.com/AdguardTeam/HostlistCompiler
Thank you this looks promising
I’ve used this for similar tasks before. Might need a few steps if the formats vary to get them all together but it should be possible.
This is very helpful thank you :)
