Submission + - Could AI Cause âKnowledge Collapseâ(TM)? Study Finds A Troubling Patte (studyfinds.com)
alternative_right writes: A new study covering 27 popular AI systems and 70 million individual statements found that larger language models tended to produce narrower, more repetitive information than smaller versions of the same models, even though bigger models are generally built to perform better.
After accounting for other factors, larger models were linked to less diversity, meaning smaller versions within the same model family tended to generate more varied claims than their bigger siblings. The researchers have a possible explanation: bigger models may memorize more of their training data, which could narrow what they say when asked about the same topic again and again.
After accounting for other factors, larger models were linked to less diversity, meaning smaller versions within the same model family tended to generate more varied claims than their bigger siblings. The researchers have a possible explanation: bigger models may memorize more of their training data, which could narrow what they say when asked about the same topic again and again.