What is a GPU?

A GPU (graphics processing unit) is a chip designed to do many similar calculations at once. It helps draw graphics and speeds up workloads such as training or running AI models.

CPU vs GPU race

Both processors get the same job. The GPU spends two ticks setting up before it can start.

Choose a workload

Each pixel is calculated on its own. The GPU can work on many pixels in the same tick.

CPU0 / 48 pixels

4 per tick

GPU0 / 48 pixels

16 per tick after setup

Tick 0 · CPU 0 of 48 · GPU 0 of 48

Rates and setup time are illustrative. This simulation does not measure your hardware.

CPU, GPU and graphics card

The CPU runs general-purpose work such as responding to input, running the operating system and coordinating an app. Its cores handle complex instructions and changing paths quickly. The GPU devotes more of its hardware to doing similar work across a lot of data at once.

A graphics card packages a GPU with other components. An integrated GPU is built into the system’s processor package or chip and usually shares system memory. A dedicated GPU is a separate processor and typically has its own memory. A laptop can have a dedicated GPU soldered to its motherboard.

Why games and AI use GPUs

In a game, a GPU can calculate colours and lighting for many pixels in parallel. In AI, multiplying matrices means calculating a large grid of outputs. Many of those outputs are independent, so the GPU can work on them at the same time. Small jobs, data transfers and steps that depend on earlier results can reduce the speed advantage.

What is VRAM?

VRAM is the GPU’s working memory. Games use it for textures and rendered frames. AI models use it for weights, temporary results and often the attention cache that holds information about earlier tokens. Longer contexts and larger batches can use more memory.

When a model and its working data exceed available VRAM, you may need a smaller or more compressed model, a shorter context, or software that moves part of the work to system memory. Offloading can slow generation. On systems with unified memory, the GPU and CPU draw from the same pool.

GPU and AI glossary

CUDA
NVIDIA’s platform and programming model for running general-purpose computing workloads on supported NVIDIA GPUs.
Tensor cores
Specialized units in some NVIDIA GPUs that speed up operations such as matrix multiply-accumulate.
NPU
Neural processing unit. A processor designed for neural-network operations, often with a focus on energy efficiency.
TPU
Tensor processing unit. Google’s accelerator family designed for machine-learning computations.

How to compare a CPU and a GPU

  1. Choose independent pixels to see how parallel work fills a grid, or dependent chain to see what happens when each cell needs the previous result.
  2. Select Next tick to advance manually or Run race to play through the comparison.
  3. Compare which processor finishes first, then switch workloads to see how the result changes.

Frequently asked questions

Is a GPU the same thing as a graphics card?

The GPU is the processor itself. A graphics card is a board that houses a GPU, memory, power circuitry and cooling. Integrated graphics and some laptop GPUs do not use a separate card.

Is a GPU better than a CPU?

A GPU can finish large batches of similar calculations faster. A CPU is often better for work that branches frequently, needs a quick response or depends on the result of the previous step.

What is an NVIDIA GPU?

It is a graphics processing unit designed by NVIDIA. NVIDIA makes GPUs for gaming, workstations and data centers. Other companies also make GPUs.

What is a GPU used for in AI?

A GPU can run many operations on model weights and inputs in parallel. That can speed up both training, when a model learns, and inference, when a trained model produces an answer.

Follow practical advances in AI with the free AI Wins digest.

Get the free AI Wins digest

Only positive AI stories, delivered daily or weekly. Free, no spam, unsubscribe anytime.