AI course essential Memory components in LLM LLMs are memory hungry. GPUs are limited by memory more compared to compute.
AI course essential Decode-Maximal Batching [CHEATSHEET] This works best with continuous batching + chunk prefill.
AI course essential Chunked Prefill [CHEATSHEET] Optimization that makes server performance realistic.
AI course essential Lifecycle Aware Performance In safety-critical, long-lifecycle systems = obsolescence is the real bottleneck. Adaptive architectures (FPGAs, SoCs, stable CPUs) = keep innovation alive after deployment.
AI course essential System-Level Parallelism & Scaling [AI workload] When you have 100K GPUs, you want to utilize all of them.
AI course essential INT8 Quantization Understanding how to do calculations in INT8 instead of FP32 and still get same results is a core optimization in DL. DL is highly insensitive to local errors.
AI course essential [Calculate] Communication and compute overhead The embedding layer (before Attention layer) expands each token to a 1D vector of size hidden_dim. The common sizes of hidden dimension are 1024 to 8096.
AI course essential [Calculate] LLM memory calculations LLM is memory intensive. This limits the LLM that can run on given GPUs. Calculating maximum context length supported in a given hardware + model
AI course essential Global AI Race The country that will control AGI will control the future of humanity.
Artificial Intelligence Serve LLM in production on DigitalOcean GPU Droplet with SGLang Step-by-step: deploy a quantized, OpenAI-compatible LLM API on a DigitalOcean GPU Droplet with SGLang, Nginx, systemd, benchmarks, and a low-cost workflow.
Culture The man who refused $1M for his discovery (Why?) (Grigori Perelman) Grigori Perelman is a Russian Mathematician who who refused $1M and the equivalent of Nobel Prize (Fields Medal) for his proof to one of the most difficult problems in Mathematics.