> Are you saying that GPT-6 would be less intelligent than a human, because we could give it an arbitrary deadline within which to answer the question, while not restricting a human that way? That doesn't sound like a fair test.
The basic design of an ML model is just one really big matrix multiplication that always takes the same time - I'm saying we haven't given GPT the ability to not do that, so it always has that arbitrary default.
It does have a system where it does beam search through the model instead of simply running it once, but it's not fully recurrent.
> That's a reasonable intuition and a fair experiment, but I think it hinges on the question of "What does a human do when they go away and think about a problem?". Could it be that they are just clearing their mind of other distractions, and trying to look at the problem from multiple perspectives, which is something that a machine could do more efficiently?
Well, some problems just take longer to think about or need knowledge that isn't already written down.
> Putting it another way, if you met someone who had never performed an experiment before, would you think they were not intelligent?
I think a prisoner or someone like that could demonstrate they're intelligent. If you're in the room with them, I mean, not like the Chinese room problem. But I think pure "intelligence" is just going to be less interesting that it looks, because it can't accomplish many tasks you'd expect it to if it can't actively interact with anything.
> Are you saying GPT-6 isn't intelligent because it doesn't have access to GPT-6's intelligence, but humans do? Or the fact that humans can go away and use Google means they are more intelligent than GPT-6 which has Google search results built into its "brain" in the first place?
Human + GPT will be more productive than GPT (at writing things that are actually logically correct) is all I mean. This isn't true of all AI - humans + chess programs are slower than chess programs - it's just because of the specific structure of GPT.
> You're just a big blob of inanimate knowledge. Ahem, sorry, what I mean is: Why does knowledge have to be "animate" to be legitimate?
I mean, it's cool that it's a big blob of knowledge, but that isn't the human capacity of "intelligence", it's "memory".
When it combines different inputs from its memory into a single one fluently, that shows something like intelligence, but it was very expensive to make that happen during training time. And GPT has flaws there you can't fix; for instance, it thinks orioles (the bird) and the Baltimore Orioles are the same thing:
I meant “this GPT-2 model has a flaw you as the user can’t fix because you can’t change the model”. Of course you could train it differently from the start and maybe it’d get better.
Btw, GPT-3 has the same issue, just tried it. (This is cherry picked; I used the Q&A example, removed the line about replying Unknown to trick questions, and tried ~5 prompts.)
-
I am a highly intelligent question answering bot. If you ask me a question that is rooted in truth, I will give you the answer.
Q: What species are the Baltimore Orioles?
A: The Baltimore Orioles are a species of bird.
> removed the line about replying Unknown to trick questions
Did you need to remove this because it identified your question as a trick question? If so, doesn’t that indicate that the model does in fact know the difference?
Mostly because I didn’t have a lot of time to prompt it and it’s hard to type new prompts on my phone. Though, a drawback of the “don’t say wrong answers” instruction is, if you keep on having a long enough conversation it’s going to go outside the token window.
Here's another try. (Notice it didn't answer the first question.)
--
I am a highly intelligent question answering bot. If you ask me a question that is rooted in truth, I will give you the answer. If you ask me a question that is nonsense, trickery, or has no clear answer, I will say it's a guess, but still try to answer.
Q: What species are the Baltimore Orioles' players?
A: The Baltimore Orioles are a professional baseball team in the American League East Division. As of the 2019 season, the Orioles have 25 players on their roster. These include 12 pitchers, 2 catchers, 6 infielders, and 5 outfielders. The Orioles have had players from all 30 Major League Baseball teams, as well as players from countries such as Canada, Cuba, the Dominican Republic, Japan, Mexico, Panama, Puerto Rico, South Korea, Taiwan, and Venezuela.
Q: What do they like to eat?
A: I'm not sure, but I guess they like to eat insects.
The basic design of an ML model is just one really big matrix multiplication that always takes the same time - I'm saying we haven't given GPT the ability to not do that, so it always has that arbitrary default.
It does have a system where it does beam search through the model instead of simply running it once, but it's not fully recurrent.
> That's a reasonable intuition and a fair experiment, but I think it hinges on the question of "What does a human do when they go away and think about a problem?". Could it be that they are just clearing their mind of other distractions, and trying to look at the problem from multiple perspectives, which is something that a machine could do more efficiently?
Well, some problems just take longer to think about or need knowledge that isn't already written down.
> Putting it another way, if you met someone who had never performed an experiment before, would you think they were not intelligent?
I think a prisoner or someone like that could demonstrate they're intelligent. If you're in the room with them, I mean, not like the Chinese room problem. But I think pure "intelligence" is just going to be less interesting that it looks, because it can't accomplish many tasks you'd expect it to if it can't actively interact with anything.
> Are you saying GPT-6 isn't intelligent because it doesn't have access to GPT-6's intelligence, but humans do? Or the fact that humans can go away and use Google means they are more intelligent than GPT-6 which has Google search results built into its "brain" in the first place?
Human + GPT will be more productive than GPT (at writing things that are actually logically correct) is all I mean. This isn't true of all AI - humans + chess programs are slower than chess programs - it's just because of the specific structure of GPT.
> You're just a big blob of inanimate knowledge. Ahem, sorry, what I mean is: Why does knowledge have to be "animate" to be legitimate?
I mean, it's cool that it's a big blob of knowledge, but that isn't the human capacity of "intelligence", it's "memory".
When it combines different inputs from its memory into a single one fluently, that shows something like intelligence, but it was very expensive to make that happen during training time. And GPT has flaws there you can't fix; for instance, it thinks orioles (the bird) and the Baltimore Orioles are the same thing:
https://www.aiweirdness.com/baltimore-orioles-effect/