If you are scraping forbidden data in my robots.txt, I don't give a damn. I am gonna mess with your bots however I like, and I'm willing to go as far as it takes to teach you a lesson about respecting my robots.txt.

Then I will teach you a lesson about trying to make public data private. Residential proxies and headful browsers go brrrr.

Malware installation is something completely different than segfaulting an .exe file that is running the scraper process.

If illegal scraping behavior is expected behavior of the machine, then what the machine is doing is already covered by the Computer Fraud Act.

several points here -

not sure if the same jurisdictions that are under the Computer Fraud Act have determined there is such a thing as "illegal scraping".

Does the Computer Fraud Act cover segfaulting an .exe file? I don't know, I don't live in the country that has it.

If The Computer Fraud act says it is ok to segfault an .exe which I highly doubt, is the organization doing this segfaulting as part of their protection against this supposed "illegal scraping" actually checking that the machines that they are segfaulting are all in jurisdictions that are under the Computer Fraud Act?

What happens if they segfault outside those jurisdictions and there are other laws that pertain there? I'm guessing it might happen they screwed then. Should have thought about that, being so clever.

Hey I get it, I am totally the kind of guy who might decide to segfault someone costing me a lot of money by crawling my site and ignoring my robots.txt. I'm vengeful like that. But I would accept hey what I am doing is probably illegal somewhere, too bad, I definitely wouldn't be going around arguing it was totally legal, and I would also be open to the possibility hey, this fight I'm jumping into might have some collateral damages - sucks to be them.

Everybody else here seems to be all righteous about how they can destroy people's shit in retaliation, and the people whose computers they are destroying might not even know they got a beef with you.

on edit: obviously once it got to courts or the media I would argue it was totally legal, ethical and the right thing to do to prevent these people from being able to attack other sites with their "illegal scraping" behavior. Because I don't win the fight if I get punished for winning. I'm just talking about keeping a clear view of what one is actually doing in the process of winning the fight.

Not my problem. The problem will be the for the malware creator. Twice.

If you are crashing some browser from a disallowed directory in robots.txt, is not your fault.

> If you’re not familiar with this, read up on it, the reasons can be quite thought-provoking

Are the reasons relevant to headless web browsers?

here's a potentially relevant example

https://news.ycombinator.com/item?id=43947910

Some, definitely not. Others, quite possibly.

Because people may be hurt.

Which people may be hurt by crashing the machine where the bot is running?

When said people decide to rob your home, they lose the right to not be hurt, IMO. Of course proportionality and all that.

If that's the case what do we do about websites and apps which do things like disable your back button (mobile phone's direct one) or your right click capabilities (desktop browser) while such functionality disabling is not present in the ToS or even presented to you upon visiting the site or using the app?