Neural Networks

Full Text

Background. Nowadays neural networks [1] are very popular, due to their effectiveness in solving different problems. As a matter of fact, there is a variety of neural network's types, which differ in their principle of work. But before constructing a neural network for definite problem, it is needed to choose it's type, which sometimes is not that obvious that it seems, and the question arises: “Does choosing of proper type of neural network is really important for its proper performance?”

Aim. To discover if the choice of the neural network type is drastic for the future result.

Methods. Out hypothesis was “The choice of the neural network proper type is decisive for its satisfying performance”. For our experiment we have chosen the problem of recognising areas using images from the satellite (Object Classification problem with 10 classes) and 2 types of neural networks to be used in solving this problem: Multilayer Perceptron and Convolutional Neural Network.

Results. Firstly, we had to choose dataset for both, neural networks models validation, and for training. So, it was decided to use EuroSat [2, 3], which contains around 27,000 labeled photos of different types of areas taken from the satellite.

Firstly, we built the model of Perceptron, trained it on the training part of the dataset and tested it on the testing part of the dataset. Predictions accuracy was 0.601. To improve performance, the Dropout [4] training technique was used, which improved accuracy up to 0.619. The last step was to increase the number of learnable parameters from 40,261,450 to 85,569,684. The accuracy has risen to 0.623, which was not satisfying enough. The final look of the model’s architecture is represented in table 1.

 

Table 1. Configuration of the Perceptron model

Layer type	Activation function	Neurons in layer
Dropout 0.2	–	0
Dense	Rectified linear unit	6144
Dense	Rectified linear unit	1536
Dense	Rectified linear unit	384
Dense	Rectified linear unit	94
Dense	Softmax	10

 

Then we built the model of Convolutional Neural Network, trained and tested it on the same dataset. Configuration of CNN is represented in table 2.

 

Table 2. Configuration of Convolutional Neural Network

Layer	Activation function	Kernel size or neurons in layer
Convolutional	Rectified linear unit	32×(3.3)
Max pooling	–	2.2
Convolutional	Rectified linear unit	64×(3.3)
Max pooling	–	(2.2)
Dense	Rectified linear unit	512
Dense	Softmax	10

 

After training and testing the result was 0.81 accuracy, which was much better than the Perceptron’s with the above-mentioned configurations. The CNN used more than 10 times less learnable parameters than the last trained model of Perceptron, which means it to be faster and simpler. Moreover, it's accuracy was about 0.2 higher. The comparison is shown in table 3. It's important to note that Perceptron itself is not as bad as that, but it just does not suit this particular problem [5].

 

Table 3. Comparison of CNN model and Perceptron model

Model	Number of parameters	Accuracy
Perceptron	85,569,684	0.623
CNN	8,413,642	0.81

 

Conclusions. In conclusion, we would like to say that every problem solution should have its own particular approach, so if you use incorrect type of neural network, you can't even get close to the desired perfect result, despite all the efforts and modern training techniques. Finally, our hypothesis was confirmed, and the choice of the proper type of neural network for solving definite problem is essential for its proper performance.

About the authors

Samara University

Author for correspondence.
Email: lefrere123@gmail.com

student, group 6309-010302D, Faculty of Informatics

Russian Federation, Samara

Samara University

Email: fanya45@yandex.ru

scientific advisor, senior lecturer of the Department of foreign languages and Russian as a foreign language

Russian Federation, Samara

References

Supplementary files