Problem with Varnish


#1

Hello,

When I keep Varnish activated, my user-agent sniffing rule doesn’t work always.

Usage

# Return 404 for X user-agent

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} .(X).*$ [NC]
RewriteRule .* - [R=404,L,NC]

Any plan to support it?

Thanks & Regards,
Gulshan


#2

Hi,

On behalf of Customer Success team of Cloudways. Hope you are doing great. I have checked at my end using the following code at the top of htaccess

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} firefox [NC]
RewriteRule .* - [R=404,L,NC]

It is working fine with and without varnish. You have to restart the apache after changing .htaccess.
Can you share more details what you are experiencing exactly so that we can check issue at our end.

Kind Regards


#3

Hello,
I followed what you have suggested right now. I think, it’s still not working at my end. Let me go in detail.

Steps to reproduce the issue

  1. Add below .htaccess rule

    RewriteEngine On
    RewriteCond %{HTTP_USER_AGENT} firefox [NC]
    RewriteRule .* - [R=404,L,NC]

  2. Re-start Apache > Purge Varnish cache > Clear your browser cache

  3. First time, access URL in the Chrome instead of Firefox. > Again, please reload the page.

  4. Now, the second time access same URL in Firefox, you will see response 200.
    By any chance, If you see, 404, reload the page. You will get 200 instead of 404.

This is what I am facing.


#4

Hi,

Hope you are doing great. I have added three lines in htaccess file of your cloned application and it is working fine. The URL is not opening in Firefox even I refreshed the page forcefully. Response code is always 404. The URL is working on Chrome because only Firefox agent is blocked. I am also sharing a screen shot for the reference.

http://wordpress-79323-292262.cloudwaysapps.com/

P.S varnish is enabled on the server and application.

Kind Regards


#5

Thanks for the update.
Please allow me some time. I am ready to try again, will send everything in Video. Hope, that can clarify if I am missing any point.


#6


#7

Hi, Any update on this?


#8

Hi,

Hope you are doing great. Sorry for the delay. I have replicated the issue and found the same actually it was MISS at my end that is why I was not getting the response at my end on Firefox. Let me explain how our stack is configured.
Port 80 Nginx (non-SSL)
Port 443 Nginx (SSL)
Port 8080 Varnish
Port 8081 Apache
MySQL 3306 (Localhost)

It will not work in the way you expected because you set rule from Apache. Whenever a request comes it is served by Nginx first which is running as reverse proxy for Apache. In Nginx configuration, varnish and apache are defined as the backend. When Varnish caches the page. It serves the page regardless of which user agent is requesting. In short, cache serve the page before it comes to Apache. In order to do this, we will have to block specified user agent from varnish VCL which will not possible in our current stack.

Kind Regards


#9

Thanks for the response.

If I keep my user-agent rules at NGINX virtual config file, would be possible to make thing working with Varnish?

if ($http_user_agent ~* ".(X).*$"){
	set $rule_0 1$rule_0;
}
if ($rule_0 = "1"){
#ignored: "-" thing used or unknown variable in regex/rew 
}

#10

Hi,

Yes, it can be blocked from Nginx Vhost manually but we do not make core changes in our stack which can cause issues in future. Currently, you can give this on feedback page.
http://feedback.cloudways.com/

Kind Regards


#11

Means, I cannot use Varnish.