Artificial Neural Networks: Basic Methods and Applications in Finance, Management and Decision Making, a Roadmap

The concept of simulating a thinking machine was introduced by Alan Turing in 1950, who proposed that if a computer-human interaction seems like human-human interaction then we should consider the computer in question to be intelligent. Generally, Artificial Intelligence systems act like humans, meaning that they need to access and use knowledge in order to solve problems by applying conditions, logic, hunches and intuition in order to make a decision [1]. Neural network methods are inspired by biology with analogous components of living organisms. While in biological Neural Networks dendrites collect signals and send a signal through an axon which in turn connects it with neurons which are excited of inhibited as a result, in Artificial Neural Networks input data are sent to a processing entity (simulating a neuron) which in turn sends output signals to other entities [2]. Such signals mainly follow the form of an “if-then” rule where Neural Networks analyze large number of variables identifying inter-actions and patterns [3], discovering rules and learning. Compared to other statistical models, Artificial Neural Networks constitute a better approach to deduce assumptions based on independent data [4,5]. Neural networks help to identify patterns among computer files, requiring assumptions and to organize and classify data achieving satisfactory predictions. These characteristics can make them a useful tool in finance [6], management [7], decisionmaking and strategic planning [8]. As demonstrated in (Figure 1):
a. The nerve cell consists of the body.
b. The body of the nerve cell also contains the nucleus.
c. Dendrites are many and short.
d. The axon is one and long.
e. The main functional feature of the nerve cell is that when it receives a stimulus it is activated.
The human brain, an extremely large, interconnected network of neurons, models the planet around us, as depicted in (Figure 1). A neuron collects input from other neurons using dendrites. It then summates the inputs and when the resulting value is higher than a threshold then an appropriate signal appears. This signal is finally sent to other connected neurons through the axon. Neural network algorithms are considered among the foremost powerful and widely used ones. In a macroscopic point of view, they could appear as a blind box: There is an input layer which transmits info to the “hidden layer(s)” which subsequently implements calculations to send results to the output layer. In order to implement and optimize a neural network it is crucial to understand how hidden layers operate. When a nerve cell is affected by a stimulus (E.g., electrical stimulation), then the membrane is potentially inverted. In other words, when under the influence of a stimulus the nerve cell is activated, the negative membrane loads move outwards and the positive inwards. But what was the problem-solving process before the development of the Neural Networks?

Figure 1:Human brain neural network.

a. Before Neural Networks, computers solved problems following specific commands provided by programmers.
The capability to solve problems was limited to those that is possible to provide a list of commands.
b. Unfortunately, many real-world problems are very complex, have no analytical solution and require alternative approaches, as for example heuristics. For instance, imagine that someone wants to recognize a handwritten letter.
c. It is practically impossible to describe (in programming language) how a letter may look like as different people write letters with different handwriting.

On the other hand, in order to digitally read handwritten letters, it is more helpful to use algorithms that recognize patterns through examples. The idea is to create a machine which will potentially be able to recognize patterns while its programmer probably would never understand the pattern itself. This means that a great number of different handwritten letters could be recognized discerning similarities and differences. Therefore, the aim of this work is to shed light on some applications of Artificial Neural Networks to forecast indicators used in specific fields, mainly finance, management and decision-making. Arguably, researchers as well as practitioners could use this study as an example of methodology in order to implement forecasting using artificial neural networks, in domains like the ones described above. Future researchers will not only find in the present article the basic introductory elements for understanding Neural Networks, but they will also be inspired to engage in further research in these or other research areas. The rest of the paper consists of a literature review and history of the artificial neural networks, including some significant milestones (section 2). After this review follows a short introduction to the basic Artificial Neural Network Models (section 3) and their applications to selected areas (section 4). Finally, section 5 includes an analysis of the models’ performance and section 6 concludes.

Kitcenham & Bereton [9] list three important reasons for conducting a literature review on information engineering and IT: 1. summarizing empirical facts, the advantages and disadvantages of a given technology, 2. to recognize any existing gaps in and recommend further research; and 3. to include a basis for the correct deployment of future study activities. This study is consistent with the second purpose, with the goal of shedding light on some of the applications of Artificial Neural Networks in different fields, especially economics, management and decision-making and also recognize gaps in these areas. Authors started the automated research in several databases (google scholar etc) reviewing all relevant studies (journals, conference, books etc). At the same time authors considered inclusion and exclusion criteria in order to reach the final insight. The key words that have been used for the present paper are: “Neural Networks application in management”, “Neural Networks application in decision making”, “Neural Networks application project management”, “Neural Networks application in finance”. The development of artificial neural networks has a noteworthy history. This study presents some major events and achievements during their evolution, in order to underline how research developed until now. In 1943, for the first time, Culloch & Pitts [10] proposed the idea of data processing through binary input elements, called “neurons”. These networks differ from those in the conventional computer science, in the sense that the steps of the programming process aren’t executed sequentially, but in parallel with “neurons” [10]. It was said that the model became the muse for deeper research.
In 1949, Donald Hebb [11] proposed algorithms that achieve learning by updating neurons’ connections [11]. McCulloch and Hebb contributed significantly to the development of neural sciences studies [10,11]. In 1954, Minsky [12] built and tested, the primary neurocomputer [12] and in 1958 Frank Rosenblatt [13] developed the Perceptron as an element instead of a neuron [13]. This was a trainable computer that could learn by changing the connections to the edge components and therefore identify patterns. The theory motivated scientists and became the basis for the essential algorithms of machine learning. This innovation led to the establishment of a public company for neural networks in 1960 [14]. Nils Nilsson [15] monograph on learning algorithms outlined any research development done so far [15] and any drawbacks found in learning algorithms. Sun Ichi Amari [16,17], Fukushima [18] and Miyaka (1980) discussed further the threshold components and the statistical principles of neural networks. Associative memory has further been reviewed by Tuevo Kohonen [19-25]. Later surveys in neural networking took into consideration Hopfield’s model as most appropriate for very large-scale integration designs [26] improved layered models [27]. The variety of applications where neural networks could be used has radically been increased from tiny examples to broad cognitive activities. Nowadays, scientists and practitioners develop complicated interconnected neural network chips of large scale. There is much research to be made in order to understand neural networks deeply and therefore achieve better results but there is still a satisfactory number of models ready to be issued to solve many problems.

Neural Network models are suitable to solve prediction and forecasting problems that have the following characteristics: i. the input is well understood, ii. the output is well understood and iii. experience is available [28]. These models consist of an input layer of neurons that sends signals to a hidden middle layer. The hidden layer of neurons computes weights and sends them to the output layer which aggregates data and generates the final output [2]. These models can classify outcomes [29] and identify patterns [30]. The procedure contains data to be fed and then processed within the layers. The structure described above is illustrated in (Figure 2). Having similar behavior to biological neurons, nodes combine inputs though an activation function to produce signal to the output layer. Each input has a different weight, which is acquired by training, taking into consideration the contribution or importance of each variable [4]. This function consists of two basic parts: i. a combination function that merges all input values to one and ii. a transfer function which produces the final outcome. A linear transfer function does not answer to most cases while sigmoid and hyperbolic tangents functions work better mainly because they can block function results between 0 and 1 (Figure 3).

Figure 2: The structure of a neural network.

Figure 3:Activation and transfer function.

Simple feedforward neural networks

Feedforward Neural Networks use the supervised learning method [31] which implements the learning process by specifying the desired output for every pattern encountered during training and by treating to differences between the network and training target as errors to be minimized [32]. The Perceptron is a simple topology of feedforward Neural Network without hidden layers. It is the first network introduced by Rosenblatt [13] in 1958 as a mechanism that is able to be trained in classification of models [33]. In the perceptron model, learning is an error-driven process calculating weights given a binary input vector to calculate output. The Perceptron learning algorithm has the following steps which are repeated until weights stop changing much:

i. For each x-> (input) and t-> (desired output) calculate y-> (output).

iii. If x-> ≠ t->

x-> =t-> then add to every weight that has x_i ≠ 0 , the difference Δw = d (t − y) , (d is the learning rate).

A generalization of the Perceptron model is the Delta Rule learning algorithm that was introduced by Widrow & Hoff [14]. It is also an error driven learning algorithm, like Perceptron, with the difference that in order to stop the algorithm the mean square error of the input vector should be minimized. Given a p -> learning vector the mean square error is calculated by the function:

In Delta Rule weights in the algorithm change by the following quantity:

Although Delta Rule learning algorithm is an improved version of the Perceptron model it cannot be used to Neural Networks with hidden layers because it is difficult to calculate the target output value t_k of each hidden node [33].

Backpropagation model

A highly successful and widely used [34] feedforward training model is backpropagation model (Figure 2), which identifies relationships between variables (inputs and outputs) [35]. The backpropagation model employs feedforward functions among a large variety of data examples driving to knowledge [36], it identifies relationships and errors and finally provides a possible outcome. Its algorithm drives the model to a learning procedure using trials and errors in order to determine the available correct answers [37]. It consists of two propagations: The forward pass, meaning the data imported to the input layer and the backward pass, meaning the feedback received and network response through error-correction knowledge [38]. The backpropagation algorithm begins the learning process using input vectors from past datasets. At first the model is initialized to small random values [39]. Each node of the input layer produces -through the transfer function- its own output which is considered as an input value to next hidden layer nodes. This continues to every hidden layer until it reaches the output layer which produces the final output value of the network. The input value of the j hidden, or output layer node is calculated by the function:

The output value of this node is calculated from the transfer function:

During the learning process for each input vector from past data the desired outcome is known. Using Delta Rule to the k output layer weights change:

Where d is the learning rate. When the calculation of new weights to the output layer is complete the value is now considered as desired output values to the first from the end hidden layer and then all weights are calculated using the same algorithm backwards until all weights change. This is a gradient descent optimization procedure which minimizes the mean square error E between output and desired output [40]: