And this seems like the biggest limitation for the LLM approach. The model just knows that a certain set of tokens tends to follow another set of tokens.
It has no understanding of what the tokens represent. So it does a great job of producing sentences that look meaningful, but any actual meaning in them is purely incidental.
And this seems like the biggest limitation for the LLM approach. The model just knows that a certain set of tokens tends to follow another set of tokens.
It has no understanding of what the tokens represent. So it does a great job of producing sentences that look meaningful, but any actual meaning in them is purely incidental.