Open-Source Internship opportunity by OpenGenus for programmers. Apply now.
Central Processing Unit (CPU), Graphics Processing Unit (GPU) and Tensor Processing Unit (TPU) are processors with a specialized purpose and architecture. We have compared these in respect to Memory Subsystem Architecture, Compute Primitive, Performance, Purpose, Usage and Manufacturers.
Note that if the correct compiler is present, all CPU, GPU and TPU can achieve the same task or result but by following a different path and different performance.
Memory Subsystem Architecture
This image captures the memory architecture of CPU, GPU and TPU:
Compute Primitive
This image summarizes the compute primitive (smallest unit) in CPU, GPU and TPU:
The dimension of data are:
- CPU: 1 X 1 data unit
- GPU: 1 X N data unit
- TPU: N X N data unit
Performance
As a comparision, consider this:
-
CPU can handle tens of operation per cycle
-
GPU can handle tens of thousands of operation per cycle
-
TPU can handle upto 128000 operations per cycle
Purpose,
-
Central Processing Unit (CPU): A processor designed to solve every computational problem in a general fashion. The cache and memory design is designed to be optimal for any general programming problem.
-
Graphics Processing Unit (GPU): A processor designed to accelerate the rendering of graphics.
-
Tensor Processing Unit (TPU): A co-processor designed to accelerate deep learning tasks develop using TensorFlow (a programming framework); Compilers have not been developed for TPU which could be used for general purpose programming; hence, it requires significant effort to do general programming on TPU
Usage
-
Central Processing Unit (CPU): General purpose programming problem
-
Graphics Processing Unit (GPU): Graphics rendering, Machine Learning model training and inference, efficient for programming problem with parallelization scope, General purpose programming problem
-
Tensor Processing Unit (TPU): Machine Learning model (only in TensorFlow model) training and inference
Manufacturers
-
Central Processing Unit (CPU): Intel, AMD, Qualcomm, NVIDIA, IBM, Samsung, Hewlett-Packard, VIA, Atmel and many others
-
Graphics Processing Unit (GPU): NVIDIA, AMD, Broadcom Limited, Imagination Technologies (PowerVR)
-
Tensor Processing Unit (TPU): Google
