What is Neural processing unit (NPU)?

A neural processing unit (NPU) is a microprocessor that specializes in the acceleration of machine learning algorithms, typically by operating on predictive models such as artificial neural networks (ANNs) or random forests (RFs). It is, also, known as neural processor.

It is important to note that it cannot be used for general purpose computing such as Central Processing Unit (CPU). This is mainly because the software support for this class of processors is not developed which can be used for any computing purpose. In fact, developing such a software/ compiler can be a challenge and at the same time, it may give low performance for tasks it is not designed for.

Need for NPU

For the last few years, we have made incredible progress in Machine Learning applications and have outperformed humans in some tasks such as playing games like Go and Chess.

At the same time, Machine Learning applications are taking human life to the next level. Some of the applications include:

• Self driving cars
• Monitoring a system or area from threats like a security system involving real time facial recognition
• Improving health care by accurate analysis and treatment

and many others

All these has exponentially increased the number of computations involved and the previous approaches of using GPUs could not scale well. This paved the way to design a processor which will perform better than GPUs and cope up with the progress we are making in the field of Machine Learning.

NPU are required for the following purpose:

• Accelerate the computation of Machine Learning tasks by several folds (nearly 10K times) as compared to GPUs

• Consume low power and improve resource utilization for Machine Learning tasks as compared to GPUs and CPUs

Examples

Real life implementations of Neural Processing Units (NPU) are:

• TPU by Google
• NNP, Myriad, EyeQ by Intel
• NVDLA by Nvidia
• AWS Inferentia by Amazon
• Ali-NPU by Alibaba
• Kunlun by Baidu
• Sophon by Bitmain
• MLU by Cambricon
• IPU by Graphcore
• Ascend by Huawei
• Neural Engine by Apple
• Neural Processing Unit (NPU) by Samsung