LLMs Text String - What It Contains?
Key Question: Does it Contain Causality
There’s been much debate about whether Large Language Models (LLMs) possess general intelligence comparable to human intelligence. Personally, my short answer is NO—and that’s also my gut feeling. However, before settling on that conclusion, perhaps it’s worth taking a moment to explore a deeper question: What exactly is embedded within the text generated by LLMs? The question I’m truly interested in asking is, “Do these text strings contain causality?”
Why Causality?
When we talk about “intelligence,” we often associate it with the ability to solve problems. If someone can solve problems that others cannot, they’re typically considered “intelligent”—a trait highly valued in society.
Now that we’ve set the stage, let’s dive in!
To determine whether causality is embedded in Large Language Models (LLMs), we need to examine the data they’re trained on and the context of that data. That said, it’s important to note that I’m speculating about the types of data used in training these models, and some of this data may not be used at all.
Scientific Text
Scientific Texts generally discuss science which is exploring and discovering the different parts of the environment model we live in. Physics, chemistry, and biology discovered the Laws that worked and reproduced with success. I can safely say that if the LLMs are trained solely on scientific text, there is a very high probability of causality built into the generated text strings.
The biggest assumption I am making is that the scientific text is of high quality and the results have been reproduced elsewhere too. Being in the academic world for a while now, my trust in research (or knowledge generation) is broken, but that is material for another issue. :)
Opinion Pieces
Opinion pieces are those that we find in newspapers or magazines where a writer shares his or her own opinion on issues that are related to society and the world at large. For example, in the Israel-Palestinian conflict, Should Euthanasia be legalized? What is contained in these opinion pieces does contain causality, but I will argue that it contains logical thinking rather than how the writer came to the conclusion expressed. Moreover, for opinion pieces, as they usually discuss something controversial and likely to be argued either way, there is a good chance that logical thinking might not be embedded “strongly” into these generated text strings.
News articles
News articles are written for the different audiences that the news agency is broadcasting to. News articles can be short, like reporting weather, sports results, disaster news, etc. Depending on the editor, journalists, and the style of writing that is accepted by the market they serve, details of the causality might not be written. What this means is there is a chance that causality might be embedded in the generated text strings if the LLMs are trained with news articles but it will not be explicit like scientific text.
Discussion Forums
Well, most of us are forum users thus will know how chaotic it is for the forum users to follow the different points of discussion and not forget the flaming that goes on there. Can we expect the machine to “follow” the discussion threads then and learn causality from it, with the huge assumption there is causality? So this is a big “No” on whether LLMs trained from discussion forums’ data can have causality embedded in generated text strings.
Concluding…
So far, we have been discussing the individual types of datasets that LLMs can be trained on, but we all know that LLMs are trained with an amalgamation of all these data. So I will conclude at the moment that I doubt that LLMs have causality embedded in them, and if we are to say that LLMs contain general intelligence, we will have to start proving that causality is embedded first, which by itself is a tall task. But let us see, perhaps a different model or memory architecture can help us with that!
What are your thoughts on this?
If this issue on LLMs resonated with you, here are more issues on LLMs I have written.
Consider supporting my work by commenting, sharing, subscribing, or making a “book” donation, and drop me some wisdom! Thanks in advance! :)