> domain-specific models will always outperform generalist ones
That's only true assuming you habe enough data to train a domain-specific model / expertise to train it and test it correctly.
I've encountered cases where an image recognition task could be accomplished well with a very general model like CLIP, but people still fine-tuned another model on their own small data set because that's considered better.
A domain specific model might be more likely to fail on weird outliers not present in the small domain specific training data.
> could spell disaster for OpenAI
Nah I don't think so. They are not all in on one specific model architecture.
If the current architecture is found to have serious unfixable flaws then they'll just change architecture.
That's only true assuming you habe enough data to train a domain-specific model / expertise to train it and test it correctly.
I've encountered cases where an image recognition task could be accomplished well with a very general model like CLIP, but people still fine-tuned another model on their own small data set because that's considered better.
A domain specific model might be more likely to fail on weird outliers not present in the small domain specific training data.
> could spell disaster for OpenAI
Nah I don't think so. They are not all in on one specific model architecture. If the current architecture is found to have serious unfixable flaws then they'll just change architecture.