Varnish HTTP Cache
Content production is nothing new. The basic roles involved have not changed.
Typography, Multimedia, CMS and Production
Content Creation needs diverse input methods:
- Text editors, Image scaling/cropping, Fileimport filters, Feeds...
- Flexible Layout/Typography Tools (WYSIWYG, semantic markup, CSS)
- Content cross referencing ("Other articles about paris hilton")
- Composition rules can be complex: ("No airline ads if "crash" present in the headline.)
The fact that you can automate content creation is not new, and the fact that it is complex is also not new. If you really want to get into the content production business ten or twenty years ago, you wanted a Heidelberg. It was produced unchanged for about 75 years. My ambition with Varnish is to do as good a job as that one.
Varnish doesn't produce content, it delivers it. That reduces the load on your CMS. Cheap hardware with Varnish can do 100+ krequests/second. It can assist in content composition and can fix stupid mistakes fast. Finally, it is written in open source. It gives you a delivery mechanism that works on the cheap.
Varnish started out on the VG.no site, which is a classic news site: rapidly changing content in a slow CMS system. They used 12 squid servers, and were unhappy with the performance. We reduced 12 squid servers to 3 varnish servers.
My Goals for Varnish:
- It's only an HTTP accellerator
- Better configuration
- Better management
- Much faster
- CMS focused feature set
RFC2616 only hints at the mention of server side caching. An HTTP accellerator is not a HTTP cache. We're just a web server that picks up HTTP. Think of Varnish as just another web server.
On Configuration Files
Configuration files are very confusing. We just want to find out what is going on and adjust in minute details. You want to lay out the controls over a diagram of the way the system works, so that users can see the system as they adjust it. In Varnish, we have VCL - the Varnish Configuration Language. We don't want a lot of configuration files, we can just use VCL as a configuration tool. We can compile tons of VCL code into a C library and include that at machine speed.
You can have multiple VCL files loaded at the same time, and switch between them on the fly. Or, you can have different VCLs for daytime, weekends, sporting events, etc. You can do anything you want; you can even include inline C code. We're adding module facility in 3.0 so you don't have to use inline C if you don't want to.
- Command line interface for real time control
- Management worker processes split:
- Manager restarts worker. Manager takes care of VCL (trusted), worker manages traffic (not trusted)
- Allows privilege separation
- Contains multithreading to worker process
Performance is not something you add to a program, performance is something you subtract from a program. The best way to make a program run fast is to make it not do things. Programming aganist an old model of a computer doesn't work any more; it's caches all the way down when you program these days. CPUs have caches, memories have caches, your RAID 5 system has a cache in front of it.
The performannce pricelist in programming is much different than what people think.
At the top end, we have the things a CPU can do from a register, like add 5. You can do tons of these. Once you start getting into memory, things get slower. Then, when you have multiple threads, you have to add locking, and that requires locked memory operations, which are even slower. Then you start doing system calls where you have to worry about security and context switches. The real issue is when you get to Disk IO, which at best acts on the order of 1ms. Things like creating a directory; you can do 50 of those per second. That's horribly slow!
Take-away: Do your job starting at the top and work your way down.
Log files can be very expensive. You open the file, write to it, flush it, and close the file. In Varnish, we have a shared memory segment that's locked into the process, and the operating system takes responsibility for writing it to disk. The speed-up is a factor of 100, and that's just the log file.
Using the shared memory segment, we can take the data and present it using varnishtop to see what's happening in real time. For example, to see what your most popular URL is:
varnishtop -i rxurl
To see a histogram of cache service times:
Content management features
- Instant action purges (regexp or exact match)
- TTL/Caching policy control in VCL - for example, shut down Google spiders during peak hours
- Load/Situation mitigation in VCL - increasing the cache time may help your backend survive in a critical situation
- Header Washing - CMSes can get confused by certain headers tricking it into doing a lot of work that's not necessary. You can strip these out.
- Vary - Used if you have the same article in three languages, for example
- Edge-Side-Includes (ESI)
Q: How simple is it to configure Varnish to support VirtualHosts?
A: We do it by default. You can give Varnish a default host header; if the client provides a host header, that will by default be passed through.
Q: What about SSL?
A: SSL is very simple: we don't do that. We do a lot of things very well, and I'm hell-bent on varnish not doing anything halfway. I haven't been able to spot what Varnish can do better with SSL than the existing proxies. Open SSL is 300k lines of code; that's five times bigger than Varnish is. I don't see it happening in Varnish any time soon.
Q: The performance hit of ESI noticeable?
A: You'll see your backend doing a lot less work. It may actually be faster. It transpires that if you use chunk encoding to the client, some of the browsers will start processing as soon as they have a chunk. If you have a large web object, it may make sense to deliver a small chunk early so that the browser can start rendering soon. This can help page render times.
Q: What about having seperate VCL files for VirtualHosts?
A: I wouldn't say it's just because I'm lazy that I haven't done that yet; there are a couple of wrinkles in the back-end configuration. Basically the problem is that in your VCL you configure your backend and set up a poll; you don't want them all to poll, you want them to share the health data. I think we had that model wrong initially; I think the code now in trunk does it right. I'll try to get it into 3.0.
Q: Is there any possible path to have parallel inclusion of ESI? Right now it does it synchronously for each request.
A: Once you start doing things in parallel, you need to park multiple threads on the same requests. That's hard, and it adds a number of race conditions that I haven't been able to find a good solution for. It will probably be sequential for the time being, and your best hope is to make sure that as much as possible is in the cache.
Q: What about compression?
A: It's stuck in the queue along with streaming. We won't have 3.0 until we have compression in some form. ESI was a major piece of surgery on Varnish, and the initial implementation was not great, but it needs to be made more general to be able to work better.
Q: How does Varnish deal with logging?
A: We can spit out NCSA records. Classic logs for any application.
Q: Other than compression, what is your feeling about Varnish 3.0 in terms of time?
A: I have 15 years experience in releasing Open Source projects. I think that's a very interesting question. If you set up a feature list that determines your release, then it may never get released; if you set a calendar date, it will get released on that date, but maybe not with all the features you want. I can see Dries has a learning curve ahead of him! I would expect 3.0 out this winter sometime.
Q: Is there any support for clustering, so multiple Varnishes share the same cache?
A: Yes and no. Squid had a proprietary protocol where caches sent packets to see if other caches has the same object. It's easier to just do a straight HTTP request. I think we can do it in VCL better than we can do it in C code, because we'll have all our knowledge of the topology at hand. Usually your Varnish machine will have really good connectivity to the rest of the world, except maybe your back-end.