AI7/10
Michael Burry Argues LLMs Cannot Reach Understanding Due to Model Collapse
Michael Burry argues that human knowledge is too small for what AI is building, that LLMs will amplify propagation errors, and that language models cannot attain true understanding without reasoning that precedes language. He cites Ballard's test and a quoted Oxford and Cambridge study on model collapse.
Original post · 1 min read
This is to my point about compression being inevitable as human knowledge is too small for what we are building. As well, humans are too redundant in their wants needs and questions.
AI-generated content will clearly contain propagation errors just like human history of knowledge does. Only LLMs will iterate those propagation errors infinitely faster with less ability to self- correct, for want of understanding.
This gets to Ballard’s test. LLMs cannot attain understanding (AGI) as understanding cannot exist unless reason first exists without language. A likely impossibility for a language model.
Research on this is already focused on getting around this in some way. Though many also have not yet conceded the point.
AI-generated content will clearly contain propagation errors just like human history of knowledge does. Only LLMs will iterate those propagation errors infinitely faster with less ability to self- correct, for want of understanding.
This gets to Ballard’s test. LLMs cannot attain understanding (AGI) as understanding cannot exist unless reason first exists without language. A likely impossibility for a language model.
Research on this is already focused on getting around this in some way. Though many also have not yet conceded the point.
Superman @thesupermannxChatGPT has now a big problem.
Researchers at Oxford and Cambridge exposed a massive threat to large language models.”
They call it “model collapse."
Internet ecosystem is rapidly changing, and generative AI will soon contribute much of the text found online. This forces us to consider what happens to future iterations like gpt-n when they are trained on data scraped from the web that was already generated by an llm.
According to the research, indiscriminately using model-generated content in training causes "irreversible defects" in the resulting ai. the model loses the "tails of the orig…
♥ 3.3K · ⟲ 419 · 👁 660.1KView on X ↗
