AI course essential Memory components in LLM LLMs are memory hungry. GPUs are limited by memory more compared to compute.
AI course essential Decode-Maximal Batching [CHEATSHEET] This works best with continuous batching + chunk prefill.
AI course essential Chunked Prefill [CHEATSHEET] Optimization that makes server performance realistic.
AI course essential Lifecycle Aware Performance In safety-critical, long-lifecycle systems = obsolescence is the real bottleneck. Adaptive architectures (FPGAs, SoCs, stable CPUs) = keep innovation alive after deployment.
AI course essential System-Level Parallelism & Scaling [AI workload] When you have 100K GPUs, you want to utilize all of them.
AI course essential INT8 Quantization Understanding how to do calculations in INT8 instead of FP32 and still get same results is a core optimization in DL. DL is highly insensitive to local errors.
AI course essential [Calculate] Communication and compute overhead The embedding layer (before Attention layer) expands each token to a 1D vector of size hidden_dim. The common sizes of hidden dimension are 1024 to 8096.
AI course essential [Calculate] LLM memory calculations LLM is memory intensive. This limits the LLM that can run on given GPUs. Calculating maximum context length supported in a given hardware + model
AI course essential Global AI Race The country that will control AGI will control the future of humanity.
Artificial Intelligence Serve LLM in production on DigitalOcean GPU Droplet with SGLang Step-by-step: deploy a quantized, OpenAI-compatible LLM API on a DigitalOcean GPU Droplet with SGLang, Nginx, systemd, benchmarks, and a low-cost workflow.
Machine Learning (ML) Beyond Roofline: Mapping and smoothing GPU Performance Ruggedness by Intel A new Intel study shows GPU matrix-multiply performance is a rugged terrain, not a smooth ceiling and introduces "ruggedness analysis" to map it and an optimizer that makes it 30% faster.
Artificial Intelligence AI Engineer's Silicon Cheatsheet: Must have book for AI Engineers and Researchers The book "Artificial Intelligence: AI Engineer's Cheatsheet: Silicon edition" is the only book you need to master AI and ML concepts to become a ML Engineer or ML Researcher.
Random Number Generators: The Core of Online Gambling Fairness When players spin the reels of an online pokie or place a bet at a digital blackjack table, they expect one thing above all else: fairness. That sense of trust is made possible by Random Number Generators (RNGs), the unseen engines running behind every legitimate online casino game.
Blockchain Blockchain Nodes Demystified: A Technical Overview In the ever-evolving world of decentralized technologies, understanding the core infrastructure behind blockchain systems is essential.
Deep Learning Run Llama3.1-8B (LLM) on DigitalOcean CPU droplet In this guide at OpenGenus.org, we present the steps to setup the minimum required DigitalOcean CPU droplet, download Llama3.1-8B-Instruct model, prepare the script to run it on a sample input and generate the output. The computation time can vary from 2 to 10 minutes (or beyond).
books C++ Coding Cheatsheet: GOAT edition The book "C++: Coding Cheatsheet: GOAT edition" is the only book you need to master C++ programming concepts. The focus is on practical programming skills, design concepts and performance engineering ideas.
Deep Learning DL1943 Deep Learning Cheatsheet The book "Deep Learning: DL1943 Cheatsheet: DL/AI/ML Research, Engineering, Optimization & System Design" is the only book you need to master Deep Learning (DL) concepts.
Multi-Agent Cyber-Physical Systems So far we've been only talking about single-agent systems. However, as we attempt to model complex systems in the real world, we shall soon find that in most cases a single agent will not suffice.
Go Programming Language Introduction to Concurrency in Go A goroutine is a function that runs concurrently to other functions. Channels provide a medium for goroutines to communicate with each other.
PyTorch Understanding the differences and use Cases of torch.Tensor.max and torch.max in PyTorch torch.Tensor.max is an instance method that applies directly to a torch.Tensor object and is mainly used to find maximum values within a single tensor. torch.max is a module-level function used to compare two tensors element-wise.
