Impact of Separable Convolutions on Performance in Deep Learning

In this paper, we propose a new neural network architecture called Ramp-CNN, designed specifically for automotive radar object recognition. The key innovation of Ramp-CNN is the incorporation of separable convolutions, which greatly improves performance in certain training scenarios.
To understand why this is important, imagine you’re trying to find specific objects in a big pile of junk. Traditional neural networks are like blindfolded people rummaging through the pile, unsure if they’re picking up a treasure or trash. Separable convolutions are like giving each person a flashlight and magnifying glass to help them find what they’re looking for more efficiently.
The impact of separable convolutions on performance depends on the chosen training objective. When trained with Binary Cross-Entropy (BCE), AENN (the neural network architecture) benefits greatly from separable convolutions, almost like a treasure hunter using a flashlight and magnifying glass to find rare coins hidden among trash. However, when trained with Mean Squared Error (MSE), performance actually deteriorates, like a blindfolded person trying to find a needle in a haystack without any tools.
This suggests that AENN is mainly performing template matching of characteristic object peaks while suppressing everything else when trained on BCE. These peaks can be represented in factorized form, which motivates the usage of separable convolutions. Meanwhile, restoring the clean complex-valued RADAR map is a more difficult task that seems to require generic convolutions. AENN trained on MAGMSE (a modified version of BCE) also benefits from separable convolutions, but performance gains are not as pronounced as with BCE.
Interestingly, performance of AENN trained with BCE increases when replacing generic convolutions with separable convolutions of the same size, even though AENN’s expressivity is reduced. This improvement suggests that the learned behavior of AENN is related to its ability to represent object peaks in a more efficient way.
In summary, Ramp-CNN is a simple yet effective modification for existing convolutional architectures that reduces computational complexity while maintaining performance. By incorporating the independence of range, velocity, and angle of objects into the NN architecture, we can improve the efficiency of automotive radar object recognition systems.