Ask HN: Protecting your Sites/Services from Unwanted Traffic?

So... If PoW (proof-of-work) schemes like Anubis ultimately don't' work in practise, because Bots/Crawlers are increasingly using headless browsers, are able to solve captures, proofs, etc;

What options do we have realistically to filter out or block unwanted traffic?

What clever schemes can we come up that don't rely on centralised serices like Cloudflare?

5 points | by prologic 4 hours ago

2 comments

  • wanderingpixel 10 minutes ago
    It depends on what you define as unwanted traffic.

    Personally I do not have an issue with bots as long as they behave and are not straight up malicious, so I rely on a combination of rate limiting, a fine-tuned OWASP CRS ruleset and an aggressive Fail2ban enforcement (hit 2 triggers and you get a 24 hour ban, 2 bans and you get banned for 30 days).

    My sites also make extensive use of static elements and caching.

  • xchili 3 hours ago
    [dead]