OpenMythos
by kyegomez
About
OpenMythos is an open-source, theoretical implementation of the Anthropic Claude Mythos model, constructed by kyegomez based on publicly available research and speculation. It employs a Recurrent-Depth Transformer (RDT) architecture with three stages: a Prelude, a looped Recurrent Block, and a Coda. The core Recurrent Block allows for multiple iterations to achieve deep reasoning, enabling compute-adaptive and depth-variable reasoning. It supports switchable attention mechanisms (MLA/GQA) and utilizes a sparse Mixture-of-Experts (MoE) with routed and shared experts. OpenMythos aims to deliver performance comparable to models with higher parameter counts through greater parameter efficiency.
Features
- Recurrent-Depth Transformer (RDT) Architecture
- Compute-Adaptive & Depth-Variable Reasoning
- Sparse Mixture-of-Experts (MoE) System
- MLA & GQA Attention Mechanism Support
- Depth Extrapolation & Parameter Efficiency
Supported Platforms
desktop