To add to this insight, there are many recent publications showing the dramatic improvements of adding another modality like vision to language models.

While this is my conjecture that is loosely supported by existing research, I personally believe that multimodality is the secret to understanding human intelligence.