Neural Networks

A neural network refers to a computer system whose modeling is inspired by the kinds of inter-connections in the human brain and nervous system. This chapter introduces you to two key neural networks, namely artificial neural networks and deep neural networks.

Artificial Neural Networks

Artificial Neural Networks (ANNs) is one of the most important concepts in Machine Learning. Just as suggested by the ‘neural’ appended to the name, the technology is inspired by the human brain which is made up of a series of inter-connected systems. There are both input layers and output layers in neural networks. In other cases, you will also find a hidden layer which has the capability to convert the input layer into a format understandable by the output layer. The essence of neural networks is to discover complex patterns that the human programmer finds hard to comprehend and extract. The machine is therefore taught on how to recognize it (Wikipedia, n.d.).

The history of neural networks can be traced to as far as the 1940s though their popularity was still dormant back then. They only became a part of Artificial Intelligence in the last few decades. The sudden interest in perceptron (another name for neural network) was initiated by back propagation. This is a technique in which hidden layers of neurons are adjusted by networks after they have failed to yield a desired outcome. For instance, if a network is designed to recognize dogs but it mistook it for a cat, back propagation enables it to redesign itself until it makes the right recognition (Wikipedia, n.d.).

Structure of Artificial Neural Networks

The human brain is made up of 86 billion neurons with connections via thousands of cells referred to as Axons. The brain accepts inputs from sensory organs or stimuli from external environment by means of connections referred to as dendrites. As the inputs come in, in the form of electric impulses, they get to travel at lightning speed using the neural network.

This is the same structure that an Artificial Neural Network assumes. In this case, the ANNs have several nodes which are the equivalent of the brain’s neurons. There are several links connecting the neurons, and the links have a capability of interacting with each other. The nodes have the ability of receiving data and conducting several simple operations on the data. When the operations yield certain results, they are transferred to other neurons. A node value or activation is the output of the operations at the node level.

There is a weight that is associated with each node. The Artificial Neural Networks are able to learn as a result of manipulation of the weight values.

Types of Artificial Neural Networks

There are different types of Artificial Neural Networks. Each of these is associated with its own levels of complexity and objectives. The feed-forward neural network is the basic type of ANNs. The other types include recurrent neural network, Hopfield networks, Boltzmann machine networks, and convolutional neural network (Analytics India Mag, n.d.).

Feed-forward neural network: In this type of ANN, the nodes are inter-connected in such a way that no cycle formed. Being the first and simplest ANN, information in the feed-forward neural network goes in only one direction, forward as implied by the name, via several output nodes and hidden nodes (if any). The network basically has no loops or cycles.

Recurrent neural network (RNN): In this type of ANN, the nodes are inter-connected in a manner that forms a directed graph with some form of sequence. In so doing, temporal dynamic behavior is exhibited for each time sequence. As opposed to the feed-forward neural network, the RNN has the ability of using memory while processing inputs. This makes them a valid choice when working with applications like speech recognition and handwriting recognition.

Hopfield networks: This is a type of RNN which was made popular by John Hopfield, hence the name. The Hopfield network introduces a model which could be relied on in understanding human memory. There are two different ways in which this type of neural network handles updates, namely: asynchronous and synchronous.

Convolutional neural network: This type of Artificial Neural Network is often used in analyzing visual imagery. The convolutional neural network (CNNs) utilizes a series of multi-layer perceptron designed in a way as to minimise pre-processing. The connectivity pattern developed by the neurons is the inspiring force behind the CNNs. As opposed to traditional algorithms which needed hand-programming to perform filtering, the CNNs learn on their own. This kind of ability which requires no human effort is a great advantage.

Boltzmann machine networks: Also referred to as stochastic Hopfield network, the Boltzmann machine networks is a type of RNN. It is ranked as one of the earliest types of neural network which could learn internal representations and provide solutions to tough combination problems. Their locality is one thing that makes them theoretically intriguing (Wikipedia, n.d.).

Deep Neural Networks

Most engineers claim that they find it hard to understand Deep Neural Networks. That is because of the level of complexity attached to them. To better understand these, you need to first consider the Deep Neural Networks. However, in these cases, they are more than two layers, hence a greater level of complexity. The sophistication of Deep Neural Networks is so that they can be explored when tackling complex data processing (Skymind, n.d.).

Generally, the definition of Deep Neural Networks revolves around the idea of these networks having an input layer, output layer and a hidden layer (between the input and output layer). The processing approach taken by each layer when ordering and sorting is designed in a manner that some describe as feature hierarchy.

The application of Deep Neural Networks is when handling unstructured or unlabeled data. This is a useful application especially in the modern generation where we have vast amounts of unstructured data floating everywhere thanks to social media content. As more of this data is yielded, it becomes essential that the technology be refined in order to process the data in an effective and meaningful manner (Neuralnetworksanddeeplearning, n.d.).

The working of deep neural networks

The processes followed by computers which use Deep Neural Networks are pretty much similar. There is a hierarchy and each algorithm involved makes application of non-linear transformation on the input and then gives an output based on what it has learned. The process goes on repeatedly until a desired outcome with a greater accuracy is obtained. The term deep is inspired by the many processing layers that the data gets subjected to.

The traditional Machine Learning involved supervised learning and this required the programmer to explicitly ‘tell’ the computer the kinds of things it ought to assess when evaluating an image.

For instance, if the machine has to determine if there is a cow in an image, the machine would be programmed to look at specific types of ears and legs for it to make an accurate output. This was a laborious process which required highly skilled programming for the computer operations to be successful.

Deep Neural Networks have the advantage of eliminating this input from the programmer. Rather, the computer discovers on its own the set of features in an unsupervised environment. The process not only becomes accurate but also faster.

For the accuracy level to be achieved there is the need for the Deep Neural Network to access vast amounts of data and compute power. The fact that these were not easily available to programmers until the emergence of cloud computing and big data, explains why the technology could not evolve. With Deep Neural Networks, complex statistical models can be created without the assistance of any human being. Large quantities of unlabeled (unstructured) data can be used in creating accurate predictive models.

The impact of the Internet of Things (IoT) cannot be ignored either. As the technology gets more omnipresent given that majority of the data employed is by machines and humans which tend to be unstructured, Deep Neural Networks are given a better opportunity to get more complex. Today’s research on Deep Neural Networks is aimed at bringing onboard all types of applications which run on big data analysis, medical diagnosis, language translation, stock market trading signals, image identification, network security, and natural language processing.

Even though Deep Neural Networks hold a stronger potential regarding making machines learn things on their own with greater accuracy, we ought to acknowledge the fact that this is a concept that is not easy to implement. Most of the things done are at the single hidden layer but the future points to a direction where more than one hidden layer would be used (Techopedia, n.d.).

Leave a Reply

Your email address will not be published. Required fields are marked *