Is my PC powerful enough for the exercises in this book?

[IDNBC]

Hi, I'm interested in using Build a Large Language Model (From Scratch) for a school project, but I’d like to make sure my PC specs are sufficient before purchasing the book.

Here are my specs:

CPU: 12th Gen Intel Core i7-12650H @ 2.30 GHz

RAM: 32 GB

GPU: NVIDIA GeForce RTX 3050 Laptop GPU (6GB VRAM)

Available storage: ~150 GB

Would this setup be enough to follow the practical parts of the book, at least for training small-scale models or running examples locally?

Thank you for your help!

[casinca]

Hi,

Imo, on CPU training with your ram, you'll have no problem, you'll be able to run all chapters smoothly without having to worry about memory. Training on CPU is a bit slow though.

If instead you want to use your Cuda GPU, some parts might lead you into OOM territory in f32, depending on the chapters, not sure.
@rasbt can give a better answer on that, I don't remember exactly. But It'll be feasible in bf16 model.to(torch.bfloat16).
You can also reduce some hyperparameters like batch_size or context_length to fit everything in VRAM. Even use Google Colab too, if needed.

Sebastian made the book with these limits and a broader audience in mind, I don't think you'll run into trouble tbh.

[IDNBC]

Thank you so much for your kind and detailed response — I really appreciate it!
It’s reassuring to hear that my setup should be sufficient with some adjustments, and I’ll definitely try using torch.bfloat16 and tweaking the hyperparameters if needed.
I'll go ahead and get the book and look forward to trying it out for my school project.

Thanks again for your help!

[rasbt]

Hi there,
I can confirm that this should be plenty. I tested the book on a 2020 MacBook Air with 16 GB RAM and it worked just fine.

Additionally, I think you will probably be able to use the GPU for most if not all chapters. 6GB RAM on the GPU might be tight but I think it is enough (the code will automatically use a CUDA-supported GPU if available). However, in case the 6 GB VRAM are too tight, you can always run the code on the CPU; that's what I've done on the 2020 MacBook Air mentioned above. Alternatively, you can use bf16 as recommended by another reader. (There are some more tips here: https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/10_llm-training-speed but I would not worry and read that until you have finished the book to be honest)

Long story short, your setup should be sufficient.

[IDNBC]

Thank you so much for your kind reply and confirmation — I’m very happy to hear that my setup should be sufficient!
It’s incredibly reassuring to know you tested the book on a MacBook Air, and I’ll definitely try using GPU when possible and fall back to CPU if needed.
Looking forward to diving into the book and using it for my school project. Thanks again!