#attention

OpenMythos is an open-source, theoretical implementation of the Anthropic Claude Mythos model, constructed by kyegomez based on publicly available research and speculation. It employs a Recurrent-Depth Transformer (RDT) architecture with three stages: a Prelude, a looped Recurrent Block, and a Coda. The core Recurrent Block allows for multiple iterations to achieve deep reasoning, enabling compute-adaptive and depth-variable reasoning. It supports switchable attention mechanisms (MLA/GQA) and utilizes a sparse Mixture-of-Experts (MoE) with routed and shared experts. OpenMythos aims to deliver performance comparable to models with higher parameter counts through greater parameter efficiency.