• Ghoelian@piefed.social
    link
    fedilink
    English
    arrow-up
    43
    ·
    8 days ago

    Unfortunately the internet has also changed a lot since then.

    Even on just my personal site, with no real content and which no real person visits (besides people looking at my CV), I still get about 2k hits per day. When I had cloudflare in front they told me it was mostly llm scrapers, not sure how reliable that is though.

    • GottaHaveFaith@fedia.io
      link
      fedilink
      arrow-up
      19
      ·
      8 days ago

      I don’t know how you host your website, but checking my reverse proxy logs I can see the user agents and indeed most of them are from llms