Skip to main content


Fighting bots is fighting humans.

One advantage to working on freely-licensed projects for over a decade is that I was forced to grapple with this decision far before mass scraping for AI training.

#AI #ArtificialIntelligence #OpenAccess

in reply to Molly White

In my personal view, option 1 is almost strictly better. Option 2 is never as simple as "only allow actual human beings access" because determining who's a human is hard. In practice, it means putting a barrier in front of the website that makes it harder for EVERYONE to access it: gathering personal data, CAPTCHAs, paywalls, etc.

http://mollywhite.net/micro/entry/fighting-bots-is-fighting-humans

via https://manuelmoreale.com/fighting-bots

#AI #ArtificialIntelligence #OpenAccess

This entry was edited (3 days ago)
in reply to Molly White

For the time being, I’ve set pretty strict throttling limits. If you’re trying to access 60 pages in a minute, you’re a bot. And a badly behaved one.