Elon Musk filed a lawsuit in San Francisco’s Superior Court accusing OpenAI and its CEO, Sam Altman, of betraying the startup’s initial commitment to openness, the betterment of society, and lack of profit as a motive. Among other things, Musk’s 35-page complaint argues that OpenAI has violated its original deal to share its GPT large language models with Microsoft, which stated that the software giant would lose access to new LLMs once OpenAI had achieved AGI. According to the complaint, OpenAI reached that epoch-shifting moment a year ago with GPT-4, its most powerful model to date.

Musk—who cofounded OpenAI but left in 2018—is at least as entitled as anyone to come up with his own definition of AGI. His complaint describes it as “a general purpose artificial intelligence system—a machine having intelligence for a wide variety of tasks like a human.” That does sound like GPT-4 as I, a mere layperson, experience it in ChatGPT Plus.

But Musk’s declaration that the AGI era is already upon us is hardly the consensus among AI scientists. Even those who think it’s not far off predict arrival dates that are least a few years away. And GPT-4 falls well short of meeting OpenAI’s own explanation of the term: “A highly autonomous system that outperforms humans at most economically valuable work.”

Consider the evidence:

GPT-4 isn’t remotely autonomous; indeed, it does its best work when humans provide plenty of hand-holding in the form of detailed prompts. The world is still in the process of figuring out what tasks GPT-4 can do, and we frequently overrate its competence. That’s not even getting into the fact that OpenAI’s reference to “most economically valuable work” suggests that true AGI may involve not just software but also sophisticated robotics that don’t exist yet. To guess when OpenAI—or a rival such as Google, Anthropic, Meta, Mistral, or Perplexity—might reach AGI, as OpenAI defines it, is to expect that it’ll be an obvious moment in time. But OpenAI’s definition, like all the others, is squishy and difficult to put to a conclusive test. To riff on Supreme Court Justice Potter Stewart’s famous comment about pornography, maybe we’ll know it when we see it. At the moment, however, I’m convinced that obsessing over AGI’s existence or nonexistence is counterproductive.

The whole notion of AGI is predicated on the assumption that AI started out dumber than a human but could someday match or exceed our level of thinking. Already, though, generative AI is different than human intelligence—far closer to omniscient than any individual flesh-and-blood thinker, yet also preternaturally gullible and prone to blurring fact and fiction in ways that don’t map to common human frailties. That’s because it’s a predictive engine, trained to string together words without truly understanding them. If its present trajectory of simulated brilliance mixed with boneheadedness continues, it might wander off in a direction far afield from most definitions of AGI.

Even if the world lands on a new, more inclusive definition of AGI, it may be hard to prove whether a particular LLM has attained it. Musk’s lawsuit cites proof points of GPT-4’s reasoning power, such as its scoring in the 90th percentile on the Uniform Bar Exam for lawyers and the 99th percentile on the GRE Verbal Assessment. That it can do so is astounding. But acing tests is not synonymous with performing useful work. And even if it were, who gets to decide how many tests an LLM must pass before it’s achieved AGI rather than just bobbled somewhere in its vicinity?

For decades, the Turing Test—which a computer would pass by fooling a human into thinking that it, too, was human—was computer science’s beloved thought experiment for determining when AI had gotten real. Strangely enough, it’s useless as a tool for assessing today’s LLM-based chatbots. But not because they know too little to fake humanity convincingly, or can’t express it glibly enough—but because they betray their artificiality by being so good at churning out endless wordage on more topics than any human knows. AGI could end up in a similar predicament: a benchmark, devised by humans, that’s rendered obsolete by the technology it was meant to measure.

DID YOU HEAR THE ONE ABOUT THE “MAC CAR?” Last week, Apple’s long, expensive quest to build an autonomous EV entered its rearview-mirror phase—a sad fate my colleague Jared Newman blamed on the company’s sometimes counterproductive pursuit of perfection. Wondering what an Apple car would be like has been an obsession for techies since 2012, when news broke that Steve Jobs had toyed with getting into the automobile business even before there was an iPhone. Or maybe it started in 2008, when reports of a meeting between Steve Jobs and Volkswagen’s CEO led to wild speculation about an “iCar.”

Or how about 1998? According to Snopes, that’s when a joke involving cars designed by software companies began spreading like crabgrass across the internet, eventually evolving into an urban legend involving a Bill Gates keynote and a General Motors press release. Along with a Microsoft car that crashed twice a day and occasionally needed its engine replaced for no apparent reason, it mentioned a “Mac car” that “was powered by the sun, was reliable, five times as fast, twice as easy to drive—but would only run on 5% of the roads.”

  • @General_Effort@lemmy.world
    link
    fedilink
    English
    13 months ago

    Isn’t this basically just what my comment about the edge of the knowable was

    No. Not even close.

    We know what obscenity is. A court will tell you if something is obscene. End of story.

    The problem with the SC quote is, that it is at odds with the rule of law. The meaning of the law must be known. It can’t be whatever some judge feels like. US courts use so-called tests to determine - with as much objectivity as possible - if something is meant by a statute or not. Currently, the “Miller test” is used for obscenity.

    No true Scotsman is not an illustration of the edge of the knowable but of irrationality.

    To elaborate on the Othello point:

    I don’t see what point you are trying to make. A bit of googling leads to this: https://thegradient.pub/othello/

    Is that what you are writing about? You are trying to show that the conclusions are unwarranted? What do you think that would imply?

    • @GiveMemes
      link
      English
      1
      edit-2
      3 months ago

      The legal system has nothing to do with understanding and everything to do with arbitrarily assigned human bullshit (just like the turing test). While law tends to be rational, it’s notoriously shit as a way of understanding the universe. (Live in a fascist country? Well, the law’s the law). I really regret trying to use that quote as an example because you’ve ratcheted onto it like a bulldog and simply can’t let go.

      Science is the only way by which we can advance our understanding of the universe. There are cases of unknowable questions in which people use philosophy or religion to try and fill the gap, but they still never actually know, just think.

      That wasn’t the exact study I was referencing, but it is actually better at explaining some of the related concepts both in analogy and in their discussion (a discussion in which, they admit that what they think their findings indicate and what their findings actually indicate could be two different things.)

      But, to conclude that somehow the multidimensional set of vectors is mapping the board out because when you change part of the input data, even counterfactual input data in which the computer hasn’t seen that move before as it’s illegal, the output data changes is another huge leap. Of course the data changes, as the patterns change, and the gpt has internalized the patterns in its training data, just as it internalizes syntax and rules of language.

      I don’t think that it really has any meaningful impact if they were incorrect, but if they are correct it could mean that AI is somehow creating a representation of data within itself, which really also wouldn’t surprise me.

      I guess I was more arguing against the guy trying to quote the study at me in the first place than the study itself, though I do have my issues with their analogy bc it’s simply clownish to compare a crow to a mathematical construct purposely created to internalize the rules and syntax of language.

      Also that journal has a high schooler on the board of editorialists and has no name for itself… not exactly The Journal of Machine Learning Research lol

      • @General_Effort@lemmy.world
        link
        fedilink
        English
        13 months ago

        I really regret trying to use that quote as an example

        As an example of the unknowable? Perhaps you could elaborate what you feel to be unknowable.

        Science is the only way by which we can advance our understanding of the universe.

        I’m actually surprised to read that from you. It doesn’t really square with your fairly dismissive attitude toward empiricism.


        Apparently, you are sure that GPTs can’t reason. However, you don’t know what reason means. So, IDK how you could possibly know whether anything or anyone is capable of reason.

        • @GiveMemes
          link
          English
          13 months ago

          Not an example of the unknowable. An example of knowing that something ‘is not’ without defining what ‘is’.

          I have a dismissive attitude towards things like the Turing test because they’re only empirical insofar as they empirically record a subjective opinion.

          Similarly, with the Othello study, my problem is not with their data, but what they attempt to extrapolate from it.

          In the same way that I can’t define God, I can say with some certainty that you aren’t it. Could I be wrong? Potentially, in an incredibly, incredibly unlikely scenario. Am I willing to take that risk? Yes… and Occams Razor supports such.

          • @General_Effort@lemmy.world
            link
            fedilink
            English
            13 months ago

            An example of knowing that something ‘is not’ without defining what ‘is’.

            So can you define what reason is not?

            I have a dismissive attitude towards things like the Turing test because they’re only empirical insofar as they empirically record a subjective opinion.

            Then you have a dismissive attitude toward much of science.

            Similarly, with the Othello study, my problem is not with their data, but what they attempt to extrapolate from it.

            What do they attempt to extrapolate from it?

            • @GiveMemes
              link
              English
              13 months ago

              Poor word choice on my part. I can know that something is not without defining what is. See God example.

              I definitely have a dismissive attitude towards social sciences. Not actual science tho.

              For my criticisms of the Othello study please see my previous comment elaborating on them.

              • @General_Effort@lemmy.world
                link
                fedilink
                English
                13 months ago

                Poor word choice on my part. I can know that something is not without defining what is. See God example.

                So your opinions on reasoning are part of a religious belief?

                I definitely have a dismissive attitude towards social sciences. Not actual science tho.

                You don’t know what actual science is.

                For my criticisms of the Othello study please see my previous comment elaborating on them.

                I didn’t ask about your criticisms. I’m still trying to piece together where you are actually going with this.

                • @GiveMemes
                  link
                  English
                  -13 months ago

                  If you could read with a 5th grader’s level of comprehension you would know by now. Feel free to go back. It’s all there. Just put it together.

                  Never said anything about a religious belief. People are open to hold them, even if irrational.

                  I’m a literal scientist so I don’t know how that could be possible. It seems like you’re the one that has a problem with extrapolating erroneously from data.

                  I will not be replying again.