A prevailing sentiment online is that GPT-4 still does not understand what it talks about. We can argue semantics over what “understanding” truly means. I think it’s useful, at least today, to draw the line at whether GPT-4 has succesfully modeled parts of the world. Is it just picking words and connecting them with correct grammar? Or does the token selection actually reflect parts of the physical world? One of the most remarkable things I’ve heard about GPT-4 comes from an episode of This American Life titled “Greetings, People of Earth”.
  • Veraticus@lib.lgbtEnglish
    12·
    11 months ago

    Here, let’s ask GPT4 itself since you’ve decided it’s suddenly an okay source:

    Your statement is correct in asserting that the vector representation in a language model is not an abstract representation. It’s purely a mathematical construct. However, saying it’s “unrelated to anything that actually exists” might be an overstatement. These vectors do capture statistical patterns in human language, which are reflections of human thought and culture. They’re just not capable of the deep, nuanced understanding that comes from human experience.

    I accept it’s an overstatement. But it is neither “incredibly wrong,” nor is it thought. (Or intelligence.)

      • Veraticus@lib.lgbtEnglish
        12·
        11 months ago

        I was in this case – but the overall point I made is still correct. If winning this minor battle is what you were seeking, congratulations. You are no closer to understanding the truth of this or what we were actually talking about. Not that that was either your point or within your capabilities.

    • SirGolan@lemmy.sdf.org
      1·
      11 months ago

      I’d just like to step in here and mention that asking an LLM is probably not a good proof (and this is directed at both of you). Its understanding of AI is from before it was trained, so it is wildly out of date at this point given how much has happened in the space since.

        • SirGolan@lemmy.sdf.org
          1·
          11 months ago

          Care to provide some proof of that? They did update their system prompt to include a few things like it is now GPT4 (it used to always say GPT3). Other than that, I don’t think it knows anything. But in general, I was more talking about developments in AI since it was trained which it certainly does not know.

          Edit: hmm I just reviewed our discussion and I note you only provided one link which was to the psychological definition of intelligence. You otherwise are providing no sources to back up your claims while my responses are full of them. Please start backing up your assertions, or provide some evidence you are an expert in the field.