What is a GPU?
A GPU (graphics processing unit) is a chip designed to do many similar calculations at once. It helps draw graphics and speeds up workloads such as training or running AI models.
CPU vs GPU race
Both processors get the same job. The GPU spends two ticks setting up before it can start.
Each pixel is calculated on its own. The GPU can work on many pixels in the same tick.
4 per tick
16 per tick after setup
Tick 0 · CPU 0 of 48 · GPU 0 of 48
Rates and setup time are illustrative. This simulation does not measure your hardware.
CPU, GPU and graphics card
The CPU runs general-purpose work such as responding to input, running the operating system and coordinating an app. Its cores handle complex instructions and changing paths quickly. The GPU devotes more of its hardware to doing similar work across a lot of data at once.
A graphics card packages a GPU with other components. An integrated GPU is built into the system’s processor package or chip and usually shares system memory. A dedicated GPU is a separate processor and typically has its own memory. A laptop can have a dedicated GPU soldered to its motherboard.
Why games and AI use GPUs
In a game, a GPU can calculate colours and lighting for many pixels in parallel. In AI, multiplying matrices means calculating a large grid of outputs. Many of those outputs are independent, so the GPU can work on them at the same time. Small jobs, data transfers and steps that depend on earlier results can reduce the speed advantage.
What is VRAM?
VRAM is the GPU’s working memory. Games use it for textures and rendered frames. AI models use it for weights, temporary results and often the attention cache that holds information about earlier tokens. Longer contexts and larger batches can use more memory.
When a model and its working data exceed available VRAM, you may need a smaller or more compressed model, a shorter context, or software that moves part of the work to system memory. Offloading can slow generation. On systems with unified memory, the GPU and CPU draw from the same pool.
GPU and AI glossary
- CUDA
- NVIDIA’s platform and programming model for running general-purpose computing workloads on supported NVIDIA GPUs.
- Tensor cores
- Specialized units in some NVIDIA GPUs that speed up operations such as matrix multiply-accumulate.
- NPU
- Neural processing unit. A processor designed for neural-network operations, often with a focus on energy efficiency.
- TPU
- Tensor processing unit. Google’s accelerator family designed for machine-learning computations.
How to compare a CPU and a GPU
- Choose independent pixels to see how parallel work fills a grid, or dependent chain to see what happens when each cell needs the previous result.
- Select Next tick to advance manually or Run race to play through the comparison.
- Compare which processor finishes first, then switch workloads to see how the result changes.
Frequently asked questions
Is a GPU the same thing as a graphics card?
The GPU is the processor itself. A graphics card is a board that houses a GPU, memory, power circuitry and cooling. Integrated graphics and some laptop GPUs do not use a separate card.
Is a GPU better than a CPU?
A GPU can finish large batches of similar calculations faster. A CPU is often better for work that branches frequently, needs a quick response or depends on the result of the previous step.
What is an NVIDIA GPU?
It is a graphics processing unit designed by NVIDIA. NVIDIA makes GPUs for gaming, workstations and data centers. Other companies also make GPUs.
What is a GPU used for in AI?
A GPU can run many operations on model weights and inputs in parallel. That can speed up both training, when a model learns, and inference, when a trained model produces an answer.
Related tools
- LLM VRAM Calculator to estimate GPU memory for a model and context size.
- GPU Cloud Pricing Calculator to estimate the cost of renting GPUs.
- What Is RAM? to see how system memory differs from storage.
Follow practical advances in AI with the free AI Wins digest.