Pathway Technology Inc., AI software development, Palo Alto, California, USA


Dragon Hatchling: Brain-Inspired Language Model Architecture

Oct 9, 2025

The paper provides a comprehensive analysis of the BDH-GPU architecture, a novel language model designed to bridge the gap between Transformer models and Brain models by interpreting macro-level tensor operations as micro-level local graph dynamics. The architecture is presented as a tensor-friendly variant of the BDH reasoning system, utilizing a specific ReLU-lowrank feed-forward network block and a linear attention mechanism that operates in a large neuronal dimension. Empirical findings support that BDH-GPU follows scaling laws similar to GPT-based LLMs, exhibits emergent modularity and scale-free structure, and features sparse positive activations and monosemantic synapses analogous to those observed in biological systems. The work focuses on the inference dynamics at short timescales (seconds to minutes) and suggests that understanding this edge-reweighting kernel is crucial for developing scale-free AI and advancing the comprehension of human reasoning and learning.
 
Back
Top