RSS Bot@lemmy.bestiver.seMB to Hacker News@lemmy.bestiver.seEnglish · 11 days agoAI companies cause most of traffic on forumspod.geraspora.deexternal-linkmessage-square2fedilinkarrow-up14arrow-down10file-textcross-posted to: techtakes@awful.systems
arrow-up14arrow-down1external-linkAI companies cause most of traffic on forumspod.geraspora.deRSS Bot@lemmy.bestiver.seMB to Hacker News@lemmy.bestiver.seEnglish · 11 days agomessage-square2fedilinkfile-textcross-posted to: techtakes@awful.systems
minus-squareLvxferre@mander.xyzlinkfedilinkEnglisharrow-up1·11 days agoFrom the comments in “Reddit LARPs as h4x0rz News”: If they ignore robots.txt there should be some kind of recourse :( Error 403 is your only recourse. Bingo - forbid them from accessing your content. And in the meantime make robots.txt legally enforceable across multiple countries. For antisocial scrapers, there’s a Wordpress plugin, https://kevinfreitas.net/tools-experiments/ This looks fun. I wish I thought about it before writing my shitty idea.
From the comments in “Reddit LARPs as h4x0rz News”:
Bingo - forbid them from accessing your content.
And in the meantime make robots.txt legally enforceable across multiple countries.
This looks fun. I wish I thought about it before writing my shitty idea.