Varnish - a quick high gloss configuration as a HTTP Director to get you up and running

So you want to get a HTTP accelerator up and running fast! Here's a shake and bake intro configuration.

Varnish like Squid is a media accelerator with one difference being that Varnish is the newest of the bunch. Some say Varnish is faster and its VCL setup is easier to grasp. Some say Squid is more reliable and have had great success with it. Some may even point out that Varnish may not follow the Http specification as completely as Squid does, but in time this should improve. The important poin is Varnish is able to serve all its content from RAM as can Squid. But, the goal of Varnish is that you should leave the kernel and OS processes to the kernel and OS. Varnish doesn't focus on memory management because the kernel should do that, it doesn't focus on raid management because the kernel and OS should do this. This amounts to an easier and more flexible configuration. Which should you use? That's up to you. This article is about getting you up and running with varnish as a HTTP Web Director in as little time as possible. Everything is configured in two files! Just give the underlying system your're running it on lots of RAM.

Depending on your preference, you may want to compile Varnish from scratch or install it as a package. On SuSE you will need to compile it although you can find pre-compiled .rpm packages on the web. Once installed, varnish will create a number of binaries, a start script in /etc/init.d and a configuration file in /etc/varnish. The default configuration is shown below and is basically a Varnish & Apache installation on the same server as you can see the backend is pointing to 'localhost' on port 8080. If you want a web server running locally you will need to configure apache to listen on 8080.


#
# This is a basic VCL configuration file for varnish.  See the vcl(7)
# man page for details on VCL syntax and semantics.
#
# $Id: vcl.conf 1200 2006-10-19 09:21:42Z des $
#

backend default {
        .host = "127.0.0.1";
        .port = "8080";
}

sub vcl_recv {
        # pass mode can't handle POST (yet)
        if (req.request == "POST") {
                return(pipe);
        }

        # don't bother caching large files
        if(req.url ~ "\.(pdf|mp3|flv|mov|mp4|mpg|mpeg|avi|dmg)") {
                return(pipe);
        }

        # force lookup even when cookies are present
        if (req.request == "GET" && req.http.cookie) {
                return(lookup);
        }


}

Varnish is flexible and from here you can use it in a number of ways, and today were going to use it as what I like to term a HTTP Web Director as I diagram in the following image.

Lets say your a small business and your running a couple of web servers and web applications from your premise. One applications web address is 'www.app1.com' and the other is 'www.new_app.com'. Both of these web sites point to a single public IP on port 80, but are housed on different platforms back in the server rooom. Here we will use Varnish to answer all requests and point them to the correct repositories in our private network. We get the added benefit of being able to do some complex caching of web content for these repositories and save the backend servers from getting hammered with requests.

Here's the config file settings for setting this up. (For a complete sample config, see Article Resources below)


#
# Quick Varnish Web Director Config
#

backend app1 {
    .host = "192.168.1.10";
    .port = "8080";
    .connect_timeout = 60s;
    .first_byte_timeout = 60s;
    .between_bytes_timeout = 60s;
    .max_connections = 1000;
}

backend new_app {
    .host = "192.168.2.10";
    .port = "8081";
    .connect_timeout = 60s;
    .first_byte_timeout = 60s;
    .between_bytes_timeout = 60s;
    .max_connections =1000;
}

sub vcl_recv {
	#Route to variuos backends
	if (req.http.host ~ "app1.com$") {
             set req.backend = app1;
        } else if (req.http.host ~ "new_app.com$") {
             set req.backend = new_app;
	}

        # pass mode can't handle POST (yet)
        if (req.request == "POST") {
                return(pipe);
        }

        # don't bother caching large files
        if(req.url ~ "\.(pdf|mp3|flv|mov|mp4|mpg|mpeg|avi|dmg)") {
                return(pipe);
        }

        # force lookup even when cookies are present
        if (req.request == "GET" && req.http.cookie) {
                return(lookup);
        }
}

sub vcl_fetch {

if (req.url ~ "^(www\.)?app1\.com$") {
        unset req.http.Set-Cookie;
        set beresp.ttl   = 60s;
        set beresp.grace = 5m;
        set beresp.http.X-Cacheable = "YES";
    }

    if (req.url ~ "^(www\.)?new_app\.com$") {
        unset req.http.Set-Cookie;
        set beresp.ttl   = 60s;
        set beresp.grace = 5m;
        set beresp.http.X-Cacheable = "YES";
    }
}

There's much more you can do in the VCL.conf, for instance, let's say those above sites have many more names by which they are referred. Since Varnish does not really know that they all point to the same website, and in turn would cache each and every different version, you can easily normalize the names in Varnish. Have a look on the web for more documention. That's it for now! Hope you enjoy using Varnish as I do!


Peace be unto you. Thank you for visiting!