The big AI models are running out of training data (and it turns out most of the training data was produced by fools and the intentionally obtuse), so this might mark the end of rapid model advancement

  • Amerikan Pharaoh@lemmygrad.ml
    link
    fedilink
    English
    arrow-up
    55
    ·
    edit-2
    5 months ago

    While synthetic data is a thing, you’ve really gotta wonder how often you can train a model on basically empty calories before the hallucination rate starts going up.

    I, for one, hope the theftbots die.

    • KnilAdlez [none/use name]@hexbear.net
      link
      fedilink
      English
      arrow-up
      24
      ·
      5 months ago

      I was reading an article about how ChatGPT will sometimes go on existential rants and I figure it’s probably because so much of the training data is now generated by LLMs and posted on the internet. probably a glut of people posting “I asked chatGPT what it was like to be a robot” and things of that nature.