Support ETags for immutable files in web front-end

[davidsarah]

Tahoe immutable files are, well, immutable. GETting an immutable file at a given URL should always retrieve the same content. But caching proxies don't know that.

For instance, in ​http://allmydata.org/pipermail/tahoe-dev/2009-November/003221.html , it is pointed out that adding a HTTP caching proxy on the client side of the gateway doesn't work:

My first instinct was to turn to Squid. However, I quickly discovered that because my JPG was being decrypted every time I accessed it, Squid was concluding that it had been changed, and dumping it from the cache.

If the gateway were to use a unique identifier for the file (such as its storage index / verify cap / read cap) as an ETag, and support the associated If-Match, If-None-Match and Vary HTTP headers, then Squid (and any similar caching proxy that supports ETags) would know that the file hasn't changed.

Note that if a caching proxy is used, file integrity will depend on the proxy's correctness and security.

Caching of mutable files/directories is more complicated and potentially error-prone; let's leave that to another ticket.

[davidsarah]

Oops. The current code is supposed to support ETags (search for setETag in code:src/allmydata/web/filenode.py). twisted.web.http also supports If-None-Match. (I couldn't immediately see any support for If-Match or Vary, but I don't know whether they're needed.) So, this ticket is invalid as an enhancement.

This doesn't explain why Squid was concluding that immutable files had been changed, but that should be a different ticket.