LLMs-from-scratch

About

"LLMs-from-scratch" is the official code repository for Sebastian Raschka's book *Build a Large Language Model (From Scratch)*. It guides users through developing, pretraining, and finetuning a GPT-like LLM from the ground up, mirroring the approach of large-scale foundational models. The project implements all code in PyTorch, avoiding external LLM libraries, and includes functionalities for loading larger pretrained models for finetuning. It's designed for educational purposes, focusing on in-depth understanding of LLM mechanics.

Features

• Build GPT-like LLMs from scratch
• Covers pretraining and finetuning techniques
• Pure PyTorch implementation without external LLM libraries
• Supports loading larger pretrained model weights
• Educational resource for in-depth LLM understanding

Supported Platforms