Introduction to Neural Networks

In our previous post, we covered the basics of machine learning. Now it’s time to go a step further and introduce neural networks.

If you paid attention to the examples for each machine learning technique, you might have noticed that neural networks appeared in all three cases. This is because neural networks are a type of machine learning algorithm, and they are both powerful and widely used.

How Neural Networks Work

Neural networks in computers are inspired by the biological networks in our brains. When we receive a sensory signal, it goes to many neurons, and their outputs connect to even more neurons. Artificial neural networks copy this idea, using many connections to link artificial neurons together.

These connections are organized into layers. The input first goes to one group of neurons, which then pass their output to another group, and so on. In the example above, there are three layers: input, hidden, and output. Sometimes people call this a “2-layer network” because they do not count the input layer. To keep things simple, it is clearer to call it a neural network with one hidden layer.

Training with Backpropagation

As we discussed before, neural networks can be trained with supervised, unsupervised, or reinforcement learning. No matter which method is used, a technique called backpropagation helps update the network’s internal settings.

Deep Neural Networks

A neural network with more than one hidden layer is called a deep neural network.

Source: aiso-lab.com

Types of Neural Networks

The way neurons are connected and arranged can change how well a network works and what it can do. Researchers have discovered many ways to connect neurons to get useful results. Here are some of the most common types:

1. Feedforward Neural Network (Artificial Neural Network)
   • Single-layer perceptron
   • Multi-layer perceptron
2. Radial Basis Function Neural Network
3. Kohonen Self-Organizing Neural Network
4. Recurrent Neural Network
   • Long short-term memory (LSTM)
   • Gated recurrent unit (GRU)
   • Recurrent multilayer perceptron network (RMPN)
   • Neural Turing machines (NTM)
   • Continuous time recurrent neural network (CTRNN)
5. Convolutional Neural Network
   • LeNet
   • AlexNet
   • VGG
   • Inception
   • ResNet
   • DenseNet