Challenges In Developing EAI Models

During the training phase, neural networks learn to provide the desired result by modifying their weights and biases. A trained network is then deployed to solve unknown problems in the real world in a process called inferencing.

Embedding complex algorithms in constrained devices affects the embedded design, latency, and power consumption in those devices. Running AI neural networks algorithms on resource constrained devices requires the algorithm and hardware designer to co-design a solution that addresses both the data engineering and data science needs. Acceleration and compression are some of the other critical problems to be addressed while developing intelligence on embedded devices.

According to research conducted by Tsinghua University, the convolutional layers within CNN models are computational-centric, and the fully connected layers are memory-centric. Thus, from a software/algorithm perspective, the memory footprint and the number of operations can be reduced by compressing CNN models while minimizing accuracy loss

From a hardware perspective, however, architectures are designed for data localization, reusability, and the acceleration of convolution operations

The bit widths of operators and weights of AI models are often reduced to target them for embedded devices.

In general, deep learning in embedded systems has three main challenges:

Deployment of models on memory constrained embedded devices
Training on MIPS constrained embedded devices
Power consumption of bulky models