Shared Imagination: LLMs Hallucinate Alike

The preprint operates within the dominant paradigm of natural language processing, specifically focusing on large language models (LLMs). The core assumptions of this paradigm include the effectiveness of transformer-based architectures, the importance of large-scale pre-training data, and the viability of instruction tuning and alignment techniques.

The preprint 'Shared Imagination: LLMs Hallucinate Alike' presents an intriguing phenomenon that challenges certain assumptions within the current LLM paradigm. The high correctness rate among LLMs answering each other's fictional questions suggests an unexpected homogeneity among these models, despite their diverse performance on standard benchmarks. This homogeneity points to a potential model drift, where LLMs, while adhering to the existing paradigm, exhibit unexpected behaviors, such as converging towards a "shared imagination space" during hallucination. This convergence does not necessarily represent a model crisis or revolution yet, as it doesn't invalidate the core tenets of the LLM paradigm. Rather, it reveals a nuance that requires further investigation to fully understand its implications. Therefore, it's classified as 'Model Drift' as it identifies a drift from expected behavior within the confines of the current paradigm without signaling a full-blown crisis. While aspects of 'Normal Science' are present in the paper's methodology (empirical experimentation and data analysis), the core finding of shared hallucination points towards an anomaly requiring explanation within the current paradigm, aligning more closely with the concept of 'Model Drift'.