Deep learning really shines when the input is raw and at a very low abstraction level: pixels, byte pair encodings etc. Using deeply learning for classification on tabular data is just needless complexity, as the variables are often at a very high abstraction level already. Also with tabular data there are generally not many spatial or temporal relationship between the variables, which CNNs and transformers excel at.
Also images and text have tons of recurring patterns that can be exploited to train big models with lots of data. There is an internet worth of each modality that at least generally can all contribute helping a model build up a better overall understanding.
There is no analog for tabular data, it's all different.
I wonder if tree-based models would outperform in situations concerning source code as well; given it's already quite structured. Going a step further and supplying and AST may be even more beneficial.
Yes, what I meant was deep learning is great at deriving those higher level abstractions from low level raw data. Words can be seen as something in between, bag of words can be fairly effective at simpler tasks, but LLMs embed words into higher and higher abstractions.
