I’m dealing with the same issue. Site is still fast, but there has to be a solution for this. It’s been going on for years and I’ve spent days trying to solve it. I’ve followed every topic I can find on this with potential solutions and nothing works. It’s like the people making all the suggestions for how to fix it, don’t actually run woo-commerce sites. Disabling varnish is the only option right now.