2026-05-13 23:52 UTC
@simon_brooke@mastodon.scot But from a pragmatic and/or functional perspective, does it matter? We call a knowledge graph a "semantic layer", but it's also just a computer program without lived experience of the world (and a much simpler one at that).
And modern multi-modal LLMs encode so much more than just textual information. They're co-trained with all the audio, video, and image information available out on the interwebs. So their embeddings don't just encode the association between "black" and "queen" and "chess", they also encode the visual representation of those pieces across a multitude of chess sets. They "know" what a black queen looks like, how it's shaped, the colors it may be. (This is how NanoBanana and other image plagiarizers are able to work).
You can feed a picture into an LLM, and the LLM can trivially describe it. You can feed a piece of music in, and it will describe the key, timbre, and emotional tone. If that's not "semantics" (ie, having a referent to the real world), what is?
Replies (0)
No replies.