Varnish setup undocumented and inadequate


It has become pretty clear to me that the Varnish settings offered by Cloudways are greatly inadequate, I would even say flawed.

First off, there is no proper documentation. Tooltips are not documentation. The knowledgeable base articles lack in details and are not replacements for proper documentation. How do I know what the proper regex syntax is? Are the conditions logically ORed? What applies to client to backend requests versus backend to client responses?

Secondly, Cloudways refuses to publish its VCL code. Therefore there is no way to know how the settings are being used.

Lastly, looking the default varnish 4 VCL code here:
it is pretty obvious that configuring varnish properly requires the use of code that cannot be addressed with the current settings offered by Cloudways.

I mean, sure it works perfectly well on a static website. But I certainly cannot make it work on my custom Woocommerce websites. I see the varnish cache unsetting the cookies woocommerce is trying to set before the backend response reaches the client. Here is an extract of varnishlog:

  • BerespUnset Set-Cookie: wp_woocommerce_session_78923a91d786864d0ee9c9e403f4a1b8=c4c510f31ba9cba95d988224f5f2e847%7C%7C1511341646%7C%7C1511338046%7C%7C8a26fca69f2e9a5e7ff9f6cf2293a40c; expires=Wed, 22-Nov-2017 09:07:26 GMT; Max-Age=172800; path=/
  • BerespUnset Set-Cookie: woocommerce_items_in_cart=1; path=/
  • BerespUnset Set-Cookie: woocommerce_cart_hash=2f2d19b301ba8355710b85c4a158dc63; path=/

Which is Varnish messing with the backend response? By the way, I have excluded all the above cookies in the provided varnish settings.

Anyone has a similar or opposite experience?

1 Like

I think I found the bug in the varnish VCL code. The code looks like this:

if (beresp.http.set-cookie ~ "woocommerce_items_in_cart") { set beresp.uncacheable = true; return (deliver); } #CloudwaysVCL
if (beresp.http.set-cookie ~ "woocommerce_cart_hash") { set beresp.uncacheable = true; return (deliver); } #CloudwaysVCL
if (beresp.http.set-cookie ~ "wordpress_logged_in_") { set beresp.uncacheable = true; return (deliver); } #CloudwaysVCL
if (beresp.http.set-cookie ~ "wordpress_sec_") { set beresp.uncacheable = true; return (deliver); } #CloudwaysVCL
if (beresp.http.set-cookie ~ "wp-resetpass-") { set beresp.uncacheable = true; return (deliver); } #CloudwaysVCL
if (beresp.http.set-cookie ~ "wp_woocommerce_session_") { set beresp.uncacheable = true; return (deliver); } #CloudwaysVCL

The issue is that beresp.http.set-cookie is only the first Set-Cookie entry in the header, but the response header can have multiple entries, as follows:

-   BerespHeader   Set-Cookie: _wp_session=0633833702c09fcc16cd4c5b047fbf6b%7C%7C1511177490%7C%7C1511177130; expires=Mon, 20-Nov-2017 11:31:30 GMT; Max-Age=1800; path=/
-   BerespHeader   Set-Cookie: wp_woocommerce_session_ca4dd0e9884501087c6b8837732219b2=0b7f0fefa0c8e536f4518bc171d29de9%7C%7C1511348490%7C%7C1511344890%7C%7Cd0195378c711bbed3b259a6faa3e1a6a; expires=Wed, 22-Nov-2017 11:01:30 GMT; Max-Age=172800; path=/

There is currently no way I see to match the non-first entries.

The solution is apparently to use or have conf/custom-resp.vcl check bereq.url with the same settings as req.url in conf/custom-recv.vcl.

So I see 3 bugs in the varnish vcl code so far:

  1. beresp.http.set-cookie only corresponds to the first Set-Cookie entry in the http backend response header. The solution is to use

  2. conf/custom-resp.vcl does not check bereq.url the same way conf/custom-recv.vcl checks req.url. The solution is either to add the check, or provide an option in the varnish settings to choose between check for client request only, backend response only, or both.

  3. /etc/varnish/backend_response/woocommerce.vcl has this code:

    if ( (!(bereq.url ~ “(wp-(login|admin)|login)”)) || (bereq.method == “GET”) ) {
    unset beresp.http.set-cookie;
    set beresp.ttl = 4h;

That seems a little ambitious. Removing all cookies without checking which ones they are. Also this is NOT documented anywhere.

For those interested, the varnish cache VCL files are here:

/etc/varnish/cloudways.vcl  # main file
/etc/varnish/additional_vcls/recv.vcl # include varnish app config recv rules
/etc/varnish/additional_vcls/resp.vcl # include varnish app config backend_response rules
/etc/varnish/recv/woocommerce.vcl # woocommerce recv rules
/etc/varnish/backend_response/woocommerce.vcl # woocommerce backend_response rules

/home/<instanceID><user>/conf/custom-recv.vcl # translated varnish app config recv rules
/home/<instanceID><user>/conf/custom-resp.vcl # translated varnish app config backend_response rules

When I turn off varnish, my Top Menu disappears from my Magento website and stays hidden even after purging the cache.

Hi MJ,

Varnish is just a caching layer, therefore it should not add or hide anything on your website. If the menu was there with varnish turned on, it’s possible you’ve been looking at an outdated version of your website, and that something else is messing with your menu bar (css/javascript code, a plugin…). I have never used Magento, so I cannot give you any advice on how to go about it.

If you use a CDN (like Cloudflare), make sure you clear the CDN cache too before going further into your missing menu bar investigation. After that, if you cannot figure it out, I would advise to contact Cloudways support.

Hope this helps a bit.