𝐁𝐮𝐢𝐥𝐝 𝐋𝐋𝐌𝐬 𝐟𝐫𝐨𝐦 𝐬𝐜𝐫𝐚𝐭𝐜𝐡

“ChatGPT” is everywhere—it’s a tool we use daily to boost productivity, streamline tasks, and spark creativity. But have you ever wondered how it knows so much and performs across such diverse fields? Like many, I've been curious about how it really works and if I could create a similar tool to fit specific needs. 🤔

To dive deeper, I found a fantastic resource: “Build a Large Language Model (From Scratch)” by Sebastian Raschka, which is explained with an insightful YouTube series “Building LLM from Scratch” by Dr. Raj Dandekar (MIT PhD). This combination offers a structured, approachable way to understand the mechanics behind LLMs—and even to try building one ourselves!

https://preview.redd.it/35sdlxdb2m0e1.jpg?width=1037&format=pjpg&auto=webp&s=dd228136fbf7cbdeeae253118ee7a46b04948c24

While AI and generative language models architecture shown in the figure can seem difficult to understand, I believe that by taking it step-by-step, it’s achievable—even for those without a tech background. 🚀

Learning one concept at a time can open the doors to this transformative field, and we at Vizuara.ai are excited to take you through the journey where each step is explained in detail for creating an LLM. For anyone interested, I highly recommend going through the following videos: 

Lecture 1: Building LLMs from scratch: Series introduction https://youtu.be/Xpr8D6LeAtw?si=vPCmTzfUY4oMCuVl 

Lecture 2: Large Language Models (LLM) Basics https://youtu.be/3dWzNZXA8DY?si=FdsoxgSRn9PmXTTz 

Lecture 3: Pretraining LLMs vs Finetuning LLMs https://youtu.be/-bsa3fCNGg4?si=j49O1OX2MT2k68pl 

Lecture 4: What are transformers? https://youtu.be/NLn4eetGmf8?si=GVBrKVjGa5Y7ivVY 

Lecture 5: How does GPT-3 really work? https://youtu.be/xbaYCf2FHSY?si=owbZqQTJQYm5VzDx 

Lecture 6: Stages of building an LLM from Scratch https://youtu.be/z9fgKz1Drlc?si=dzAqz-iLKaxUH-lZ 

Lecture 7: Code an LLM Tokenizer from Scratch in Python https://youtu.be/rsy5Ragmso8?si=MJr-miJKm7AHwhu9 

Lecture 8: The GPT Tokenizer: Byte Pair Encoding https://youtu.be/fKd8s29e-l4?si=aZzzV4qT_nbQ1lzk 

Lecture 9: Creating Input-Target data pairs using Python DataLoader https://youtu.be/iQZFH8dr2yI?si=lH6sdboTXzOzZXP9 

Lecture 10: What are token embeddings? https://youtu.be/ghCSGRgVB_o?si=PM2FLDl91ENNPJbd 

Lecture 11: The importance of Positional Embeddings https://youtu.be/ufrPLpKnapU?si=cstZgif13kyYo0Rc 

Lecture 12: The entire Data Preprocessing Pipeline of Large Language Models (LLMs) https://youtu.be/mk-6cFebjis?si=G4Wqn64OszI9ID0b 

Lecture 13: Introduction to the Attention Mechanism in Large Language Models (LLMs) https://youtu.be/XN7sevVxyUM?si=aJy7Nplz69jAzDnC 

Lecture 14: Simplified Attention Mechanism - Coded from scratch in Python | No trainable weights https://youtu.be/eSRhpYLerw4?si=1eiOOXa3V5LY-H8c 

Lecture 15: Coding the self attention mechanism with key, query and value matrices https://youtu.be/UjdRN80c6p8?si=LlJkFvrC4i3J0ERj 

Lecture 16: Causal Self Attention Mechanism | Coded from scratch in Python https://youtu.be/h94TQOK7NRA?si=14DzdgSx9XkAJ9Pp 

Lecture 17: Multi Head Attention Part 1 - Basics and Python code https://youtu.be/cPaBCoNdCtE?si=eF3GW7lTqGPdsS6y 

Lecture 18: Multi Head Attention Part 2 - Entire mathematics explained https://youtu.be/K5u9eEaoxFg?si=JkUATWM9Ah4IBRy2 

Lecture 19: Birds Eye View of the LLM Architecture https://youtu.be/4i23dYoXp-A?si=GjoIoJWlMloLDedg 

Lecture 20: Layer Normalization in the LLM Architecture https://youtu.be/G3W-LT79LSI?si=ezsIvNcW4dTVa29i 

Lecture 21: GELU Activation Function in the LLM Architecture https://youtu.be/d_PiwZe8UF4?si=IOMD06wo1MzElY9J 

Lecture 22: Shortcut connections in the LLM Architecture https://youtu.be/2r0QahNdwMw?si=i4KX0nmBTDiPmNcJ 

Lecture 23: Coding the entire LLM Transformer Block https://youtu.be/dvH6lFGhFrs?si=e90uX0TfyVRasvel 

Lecture 24: Coding the 124 million parameter GPT-2 model https://youtu.be/G3-JgHckzjw?si=peLE6thVj6bds4M0 

Lecture 25: Coding GPT-2 to predict the next token https://youtu.be/F1Sm7z2R96w?si=TAN33aOXAeXJm5Ro 

Lecture 26: Measuring the LLM loss function https://youtu.be/7TKCrt--bWI?si=rvjeapyoD6c-SQm3 

Lecture 27: Evaluating LLM performance on real dataset | Hands on project | Book data https://youtu.be/zuj_NJNouAA?si=Y_vuf-KzY3Dt1d1r 

Lecture 28: Coding the entire LLM Pre-training Loop https://youtu.be/Zxf-34voZss?si=AxYVGwQwBubZ3-Y9 

Lecture 29: Temperature Scaling in Large Language Models (LLMs) https://youtu.be/oG1FPVnY0pI?si=S4N0wSoy4KYV5hbv 

Lecture 30: Top-k sampling in Large Language Models https://youtu.be/EhU32O7DkA4?si=GKHqUCPqG-XvCMFG