[Turorial]Internet on steroids(dnsmasq & squid)

[proxx]

Hello EZ,

Its been a while since I wrote something uselful so here goes.


Recently I had some time and decided it to setup a spare machine as a server.
Ill explain to you what I have done thus far in a tutorial like fashion.



[Requirements]

-Computer.
-Spare drive or partition.
-Any linux distro (My case Arch).
-Router or whatever.
-Squid.
-Dnsmasq.
-Basic linux skills.
-Cold beer or coffee


[Background]

First off ill explain to you what dnsmasq and squid are.

-Squid

Squid is a proxy server.
This tool has been around for a long time and is often deployed in scenarios with a slow connection but more frequently
it is used to control web traffic(http(s) in most cases)
The primary usuage for us now is its caching ability.
What this means is that when for example the logo of a website is downloaded the proxy stores this image.
The next time its requested squid will return the picture from the cache so it doesnt have to be downloaded from the original webserver again.
This might not seem like a big deal but I can tell you that the evilzone website loads about twice as fast for me.
Squid has a lot of interesting plugins such as filtering advertisements, blocking websites etc.
Multiple machines on the network that need the same updates only have to be downloaded once.
I could say a lot more about squid and I will state again that this tool is far more powerful than what I have told you thus far.
For more Info: https://en.wikipedia.org/wiki/Squid_%28software%29

-DNSmasq

DNSmasq is a caching DNS server, it does DNS request to a dns server such as opendns and has the ability to cache the results.
There are many tools that do similar things however Ive had good experience with it and many would agree.
The main advantage of caching DNS is that when you often visit the same website you can store the IP addresses and dont have to request themover and over again, providing much better response times.
For example:

The first request.
DNSmasq doesnt have cnn.com in its cache and does a upstream request.
As you can see its 58 msec.

# dig cnn.com server
;; Query time: 58 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Thu May  9 23:18:34 2013
;; MSG SIZE  rcvd: 99
;; Query time: 58 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Thu May  9 23:18:34 2013
;; MSG SIZE  rcvd: 99

The second request.
Now DNSmasq has cached cnn.com and gives the result back to my machine.
Its  0 msec.

# dig cnn.com server
;; Query time: 0 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Thu May  9 23:19:12 2013
;; MSG SIZE  rcvd: 24
;; Query time: 0 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Thu May  9 23:19:12 2013
;; MSG SIZE  rcvd: 24

The digits speak for themself.




[Setup]

In my setup I placed a dedicated drive for the cache and mounted it as such:
/etc/fstab snippet:
(Could be optimized)

Code: [Select]

UUID=1f37353a-13dd-4c51-9b4f-eca47b2eaee6 /cache ext4 defaults 0 1
Installing squid and dnsmasq.

Code: [Select]

pacman -S squid dnsmasq

Now we will configure the config files.

Code: [Select]

nano /etc/squid.conf

At the bottom we will edit this in:

Code: [Select]

shutdown_lifetime 10 seconds
cache_mem 1024 MB
maximum_object_size 10 MB
cache_dir diskd /cache 100000000 16 256
dns_defnames on

shutdown_lifetime - Time to wait before killing the proces not really important
cache_mem - How much ram can be used to temp stuff, depends on machine specs.
maximum_object_size - Limit for individual files(lots of discussion on performance impact, currently testing this)
cache_dir - Basically the size that can be used for cache , in my case 100GB.
dns_defnames - If DNS name doesnt contain a "." its considered local iirc.

Alright thats basically enough for squid.



As for DNSmasq:

Lets edit the config file.

Code: [Select]

nano /etc/dnsmasq.conf
Ill show only the lines ive edited.
A lot more tweaking could be done.

Code: [Select]

resolv-file= /etc/resolv.dnsmasq
cache-size=1000000
interface=eth0

resolv-file= /etc/resolv.dnsmasq - The upstream DNS server when record is not in cache.
cache-size=1000000 - the amount of DNS records allowed to store


Ill explain what ive done.
This is my /etc/resolv.conf

Code: [Select]

nameserver 127.0.0.1
This means locally every DNS name is resolved by the local service running on port 53, which is dnsmasq.
Ive created a second resolv.conf file named : resolv.dnsmasq

Code: [Select]

nameserver 8.8.8.8
nameserver 8.8.4.4

I think its pretty clear what happens here but ill explain it nevertheless.
A program such as a webbrowser wants to resolve a DNS name.
It looks at resolv.conf and is directed to the localhost:53
DNSmasq then looks if it has the entry cached.
If it does it returns the name.
If it does not it goes to /etc/resolv.dnsmasq and sees the DNS server upstream.
It then queries them and stores it in the cache.
This is the reason that the second and requests to come have a much higher response.

Finally its time to start the DNSmasq and the squid server.

Code: [Select]

systemctl start dnsmasq
systemctl start squid

Now on the client you have to change the DNS server to your locally running DNS server.
like this:

Code: [Select]

nano /etc/resolv.conf


Code: [Select]

nameserver 192.168.0.2

If course your webbrowser has to be pointed to the proxy server.
Dont think I have to explain how to do this



[Considerations]
Caching has its disadvantages.
For example an old image with the same name has been replaced.
Old DNS records can get in the way.
One could manually yank them out or write a dirty script to do so.
I might add that later.



Enjoy your browsing on steroids!
I can assure you this shit is fast
Please notify me of any errors in this article.



Have a nice day,
proxx

PS For all the ones that actually read this to the bottom Ill throw in a little bonus:
http://www.k-max.name/wp-content/uploads/2012/02/Saini_K._-_Squid_Proxy_Server_3.1_Beginners_Guide_2011.pdf

[kenjoe41]

I like it. Am going to try it out since my net connection sucs as he'll. Thanks.

[proxx]

Your welcome.
If you run into any issues just let me know.

[proxx]

A Little something I would like to add.

In my case and I believe many others I have multiple linux boxes running.
These are all running archlinux and as you can imagine thats a lot of updates.
So instead of downloading each and every update 3 times I set it up to use the proxy.

/home/user/.bashrc

Code: [Select]

export http_proxy=192.168.0.1
alias sudo='sudo -E'

This goes for all machines.

When I update the first time the proxy downloads the updates for me.
After that any box that runs updates will grab them from the proxy.

vte-common-0.34.5-1-i686                        5.5 KiB  2.71M/s 00:00 [########################################] 100%

This speed is way above my download speed.
Makes updating a whole lot quicker.

[kenjoe41]

Nice! Editting my ../.bashrc file like Now. 

[proxx]

There is one more important thing that needs to be mentioned. By default squid shows the forwarded_for header and reveals your internal IP address. This unwanted behaviour. At the bottom of /etc/squid/squid.conf:

Code: [Select]

forwarded_for off Now restart squid:

Code: [Select]

$ squid -K reconfigure One more thing that I personally like is to hide the version info for squid:

Code: [Select]

httpd_suppress_version_string on And last but not least lets change our hostname into something completely random.

Code: [Select]

visible_hostname 9yf20310p