I know there are tools that supposedly make it harder for AI to be trained on your images. But what about text or code? Is there any way to share things outside of a small group while minimizing the risk that some fucker will scrape them or otherwise use them for building an LLM?

  • merc@sh.itjust.worksEnglish
    1·
    11 days ago

    Even that isn’t enough if they respond to that by just throwing more crawlers at your site.

    • lime!@feddit.nu
      1·
      11 days ago

      yeah it’s not the only thing you want on there of course. i run a tarpit, fail2ban, and anubis.

      • merc@sh.itjust.works
        3·
        11 days ago

        Yeah, at one point he posted that fail2ban was only able to ban about 20 IPs/second, but he was getting 200 requests per second, and the bans weren’t having a meaningful impact on that (i.e. the requests were all coming from different IPs). From what I remember, his system load was over 1000.

          • merc@sh.itjust.works
            1·
            11 days ago

            I think it was running on a VPS, but I’m not sure. As for the traffic, I think it was coming from all over. Amazon-owned netblocks, mysterious ones that were hard to trace, etc.