• EmilieEasie@fedinsfw.app
    link
    fedilink
    English
    arrow-up
    2
    ·
    2 days ago

    Oh really good question. I think something awful is slow but at least it’s not heavily botted already? Maybe try eating them lol. I’m sure it’s already scraping stupid 4chan

    • EliteCloneMike@lemmy.zip
      link
      fedilink
      arrow-up
      2
      ·
      2 days ago

      Definitely is. I think the problem is that any site they scrap has to be able to withstand the amount of scraping they are doing without going under. For example if they scrap a small forum, without providing any funds, the energy consumption of that site would be unsustainable. Their bots are scrapping webpages thousands of times a day, with or without permission from the site administrators. That’s a heavy load. That’s also part of the reason why their switch to force AI into search is so dumb. At least under their original model of search, sites would get traffic from real people. Now it’s just bots scrapping without providing any real traffic. It’s killing off smaller sites, while giving traffic to the likes of Amazon. Google should be broken up and they’re search engine reclassified as a utility.

      • JustTesting@lemmy.hogru.ch
        link
        fedilink
        arrow-up
        2
        ·
        1 day ago

        Except for user numbers, activitypub/fediverse should be their wet dream. They could just make a fake instance and have content delivered to them, instead of having to scrape.