How to Improve Neural Networks by Using Complex Numbers?

What are Convolutional Neural Networks?

Convolutional Neural Networks are a type of deep learning neural network with multiple layers, including an input layer, hidden layers (fully connected layers, convolutional layer), a pooling layer, a padding layer, and an output layer, which is used for image segmentation, and extracting features from it.

Convolutional Neural Networks are used for the grid-like matrix dataset. The convolutional layer uses a filter and sends it to the input layer for extracting features from the input image. Then, the pooling layer is used to downsample the image, which helps in reducing the computation. The fully connected layers help to make the final predictions. The starting layers in the convolutional neural networks help to extract the features like edges and colours of the image. The hidden layers are used to extract the features more deeply and clearly.

Complex Numbers or Functions in Convolutional Neural Networks

It is obvious that the initial layers of the neural network extract minimal features from the images, like the colour and edges of the images. We can say it extracts the low level features of the images, and the later layers extract the high resolution features from the images. But, using the complex functions on the images, we can now extract low level features from the images. The convolutional layers extract the features from the images, which are expensive and require more investment. With the help of the complex functions applied to the neural networks, this process of extracting low level features becomes easy.

The neural networks that use complex values instead of real value weights are known as complex-valued Neural Networks. All the input and output signals and parameters, including weights and thresholds, are complex numbers. The activation function is also a complex value function. Each parameter has a real and imaginary part, making it a complex value. The complex-valued neural networks use Fourier Transformation to complete tasks like image processing and speech recognition.

Let's see some features of the complex-valued neural networks.

• The complex-valued neural networks have two hypersurfaces dividing the decision region into four equal sections.
• The complex-valued neural networks can represent two dimensional information as the parameters of input and output signals are complex numbers.
• The complex-valued neural networks learn faster than traditional real-valued neural networks. It used a backpropagation algorithm for the learning process.
• It needs fewer parameters than the real-valued neural networks. It is about half of the parameters, including the weights and thresholds.
• The complex-valued networks are much more flexible towards noise and interference produced while preprocessing the images, which makes it more efficient for fields like signal processing.

Training of the neural networks with complex numbers

Complex valued neural networks use backpropagation algorithms for training the data. It computes both real and imaginary parts of the parameters. The activation and loss functions must be complex differentiable functions such that the network can be optimised individually for both the complex and real values.

The complex-valued neural networks are trained with the complex value parameters. We use the input layer, which splits the complex values into real and imaginary parts, before sending the data to the further layers of the neural networks. After sending the data after splitting, the further layers split the data into various channels.

As we cannot feed complex numbers directly to the neural networks, we explicitly implement the complex and the real values to both the input and output layers of the network.

Applications and Use Cases of the Complex-valued Neural Networks

According to various studies, in neural networks that use complex numbers for the preprocessing and training, the complex part of the parameter is treated individually as real numbers. The complex values neural networks are assumed as two dimensional real-valued neural networks. The complex-valued neural networks are used in many fields, including:

1. Signal Processing: Complex-valued neural networks are used for signal processing like Fourier analysis and image processing with complex values.
2. Complex-valued neural networks are also implemented in Computer Vision, involving reconstruction of images, deblurring of images, etc.
3. Complex-valued neural networks can also be used in electromagnetics in designing antennas and representing electromagnetic waves.
4. Complex-valued neural networks are also implemented in machine learning to design algorithms and process huge amounts of data.

Complex-valued Neural Network in Python

Python provides different libraries like Keras, PyTorch and Tensorflow, which support complex valued operations on neural networks. We can easily implement the complex values in the neural networks, helping to improve their efficiency and accuracy.

Challenges of Complex-valued Neural Networks

Though complex-valued neural networks have many advantages and are used to improve the efficiency of the neural networks, they face various challenges:

• The complex-valued neural networks' activation functions cannot be complex-differentiable and bounded simultaneously.
• The complex-valued neural network needs specific deep-learning libraries that can be optimised for the complex value computations.
• The complex-valued neural networks need more expertise to interpret results from the complex-value computations.
• Though implementing complex values in the neural networks helps to improve their efficiency and accuracy, there has been less research till now.