Currently, there are more than 100 companies all over the world building ASICs (Application Specific Integrated Circuit) or SOC’s (System on Chip) targeted towards deep learning applications. There is a long list of companies here. In addition to these startup big companies like Google (TPU), Facebook, Amazon (Inferentia), Tesla etc are all developing custom ASIC’s for deep learning training and inference.
These can be categorized into two types:
- Training and Inference — These ASIC’s are designed to handle both training the deep neural network and also performing inference. Training a large neural network like Resnet-50 is a much more compute-intensive task involving gradient descent and back-propagation. Compared to training inference is very simple and requires less computation. NVidia GPU’s, which are most popular today for deep learning, can do both training and inference. Some other examples are Graphcore IPU, Google TPU V3, Cerebras, etc. OpenAI has great analysis showing the recent increase in compute required for training large networks.
- Inference — These ASICs are designed to run DNN’s (Deep neural networks) which have been trained on GPU or other ASIC and then trained network is modified (quantized, pruned etc) to run on a different ASIC (like Google Coral Edge TPU, NVidia Jetson Nano). Most people say that the market for deep learning inference is much bigger than the training. Even very small microcontrollers (MCU’s) based on ARM Cortex-M0, M3, M4 etc can do inference as shown by the TensorFlow Lite team.
Making any chip (ASIC, SOC etc) is a costly, difficult and lengthy process typically done by teams of 10 to 1000’s of people depending on the size and complexity of the chip. Here I am only providing a brief overview specific to deep learning inference accelerator. If you have already designed chips you may find this too simple. If you are still interested, read on! If you like it share and 👏 …..
Full article: https://towardsdatascience.com/how-to-make-your-own-deep-learning-accelerator-chip-1ff69b78ece4