- Email: [email protected]
A neural network approach to mutual fund net asset value forecasting
Omega, Int. J. Mgmt Sci. Vol.24. No. 2, pp. 205-215, 1996 Pergamon
0305-0483(95)00059-3
Copyright© 1996ElsevierScienceLtd Printedin GreatBritain.All fightsreserved 0305-0483/96$15.00+ 0.00
A Neural Network Approach to Mutual Fund Net Asset Value Forecasting W-C CHIANG TL URBAN The University of Tulsa, Tulsa, Okla, USA GW BALDRIDGE WorldCom, Inc., Tulsa, Okla, USA (Received March 1995; accepted after revision November 1995) In this paper, an artificial neural network method is applied to forecast the end-of-year net asset value (NAV) of mutual funds. The back-propagation neural network is identified and explained. Historical economic information is used for the prediction of NAV data. The results of the forecasting are compared to those of traditional econometric techniques (i.e. linear and nonlinear regression analysis), and it is shown that neural networks significantly outperform regression models in situations with limited data availability. Copyright © 1996 Elsevier Science Ltd
Key words--forecasting, neural networks, mutual funds
INTRODUCTION A MUTUAL FUND CONSISTS of a diversified portfolio of stocks and bonds that is managed by professionally trained individuals. It has become America's investment vehicle of choice. By 1990, the assets in the variety of US mutual funds, such as income and bond funds, money market funds, and equity funds, have exceeded $1 trillion [24]. Very much like portfolios of individual securities, it has provided an opportunity for investors (individual as well as institutional) to accommodate risk-return tradeoffs as well as diversification, liquidity, and tax purposes. Undoubtedly, a great deal of attention in the financial economics literature has been devoted to the evaluation of the performance of mutual funds. Most of these studies have focused on the forecasting skills of mutual fund managers, which are composed of OME24/2--G
(1) forecasts of price movements of selected stocks in the funds (stock selection) and (2) forecasts of price movements of the general stock market as a whole (market timing). However, conflicting results have been generated from these studies (see, for example, [6-8, 20-22, 36]). Empirical studies using multiple regression techniques are the major tools in these studies. Recently, Brockett et al. [2] formulated chance constrained programming models, exploring risk-return tradeoffs, to effect investors' choice of mutual funds. Efficient market theory states that all information relevant to an asset will be reflected in its price. In a perfectly efficient market, prices of an asset always reflect all known information, prices adjust instantaneously to new information, and speculation succeeds only as a matter of luck. In an economically efficient market, prices may not react to new information 205
2"06
Chiang et al.--Mutual Fund Net Asset Value Forecasting
instantaneously, but over the long run, speculation cannot be rewarded after transaction costs are paid. Although frequently challenged and sometimes refuted by prior studies, the hypothesis of efficient markets is still commonly believed to be true (see [18] for a review of the research in this area). Mutual funds are diversified portfolios of individual assets. Therefore, the prices of mutual funds should also reflect the known economic information either instantaneously or over the long run. However, economists and financial managers are at a loss to relate varieties of economic variables that are associated with market advance and decline. Ideally, one would like a derived equation relating these complex relationships. Neural network techniques can provide means by which these economic relationships may be exploited. A neural network is a system that performs a mathematical mapping from one domain to another by use of algorithmic techniques. In the case of mutual fund forecasting, the network would map from units such as nominal dollars, real dollars, percents, and indices to the net asset value (NAV) of the mutual fund (dollars). There has been a considerable amount of research conducted using neural networks for purposes of prediction. Tam and Kiang [33, 34] used neural networks to predict bank failures and found the results to be superior to discriminant analysis techniques. Salchenberger et al. [27] predicted thrift failures with neural networks and achieved better results than logit models. Dutta and Shekhar [1 l] illustrated that neural networks outperform regression analysis in predicting bond ratings. Brockett et al. [3] found that neural networks were better at predicting insolvency of insurance companies than a system designed specifically for this purpose that utilizes financial ratios. Other researchers have also used neural networks to predict stock price performance [39], software reliability [19], academic success of graduate students [15], power system load [23], 'out-of-control' production processes [31], and sludge bulking in wastewater treatment plants [4] and have generally found the results better than those of the traditional analytical techniques. Sharda [29] presented a survey of the literature on applications of neural networks and noted that neural network models performed better than the traditional
statistical techniques 71% of the time (30 of 42 studies). Most of the research concerning neural networks for economic forecasting purposes has been in the area of time-series forecasting. The use of neural networks has been frequently compared to the use of traditional time-series forecasting techniques (e.g. Box-Jenkins) and has been applied in a variety of situations (see, for example, [5, 10, 13, 17, 30, 35, 38]). Far less research has been conducted using neural networks for causal or econometric modeling. Hruschka [16] compared neural network models to an econometric model of market response and Marquez et al. [25] evaluated the performance of neural network models and regression models using generated data. Both of these studies, though, made the comparisons with regard only to the fit of the data, and did not extend the comparisons to encompass forecasting effectiveness. Sch6neburg [28] used neural networks to forecast stock prices using several 'independent' variables; however, these variables were all related to the stock price being predicted (e.g. the variation in the stock price from the previous day) and did not consider any exogenous variables. Bansal et al. [1] used neural networks to forecast prepayment rates in mortgage-backed securities portfolio management; however, they stated "forecasting prepayments requires large data sets" and utilized 1170 observations for their analysis. The purpose of this research is to evaluate the use of neural networks to develop forecasting models for predicting end-of-year net asset values of various US mutual funds. Various economic variables--such as gross national product, consumer price index, inflation rate, etc.--will be used as exogenous variables for the models. Since many mutual funds have only recently been established, it is important that a forecasting technique be able to perform reliably with minimal data requirements; thus, we will focus specifically on the situation in which a limited amount of data is available for analysis (in particular, using five years of data to forecast year six). Of course, many other practical situations arise in which forecasts must be made with few observations. The results will be compared to the use of traditional causal forecasting techniques--regression analysis, analyzing both linear and nonlinear models--to determine the relative performance under these conditions.
Omega, Vol. 24, No. 2
THE BACK-PROPAGATION NEURAL NETWORK FOR NAV FORECASTING
by using an algorithm known as back-propagation (for a detailed, introductory description see [37]).
The network architecture The specific back-propagation neural network used to forecast the NAVs consists of an input layer, a hidden layer, and an output layer, as shown in Fig. 1. Each neuron in the input layer contains known information (e.g. the unemployment rate). Lines, representing weights, connect the input layer to the hidden layer. These weights are used to model the synaptic strength between neurons in the human brain, thus portraying the importance of one neuron's 'effect' on the next layer's neuron. This 'effect' represents the 'activation level' of the neuron. The hidden layer is also made up of neurons, but the values of these continuously change as the network is training. Weights also connect the hidden layer to the output layer. The output layer consists of only one neuron because we are seeking only one piece of data--the NAV of a mutual fund at year's end. The open-circled neurons at the end of both the input layer and the hidden layer are indicative of the neuron biases which allow for a more rapid convergence of the training process. The neural network forecasts the NAV
Neuron: i I
Neuron: i 2
207
The back-propagation training method The back-propagation algorithm is used to train ('adjust') the weights in the neural network so that new, never-before seen information entering the input layer will yield astute results (correctly forecasted NAVs in our case) in the output layer. The actual training of the network is accomplished by 'back-propagating' the error from the output layer, to the hidden layer, and finally to the input layer while changing the weights simultaneously. The error, in our case, is simply the difference between the desired output (a known NAV dollar amount) and the output calculated during training, using the algorithm explained below. Essentially, steps are performed to map mathematically from 'economic variables' to 'Net Asset Value'. This is accomplished using equations that attempt to mimic the human brain. A neuron may activate strongly, weakly, or somewhere in between based upon the activation levels of the previous layer's neurons. There are two steps to the algorithm: the forward pass and the reverse pass.
Neuron: ik. 1
Neuron: i t
Neuron: bias
INPUT LAYER
HIDDEN LAYER as
OUTPU LAYER N{~uron." o Fig. 1. The back-propagation neural network.
Chiang et al.--Mutual Fund Net Asset Value Forecasting
208
During the forward pass, known economic information enters the input layer which is specified as i,, where n = 1, 2 ..... k, k being the number of neurons in the input layer. One economic variable is applied to each neuron in the input layer. A variable, N E T , is then calculated for each neuron in the hidden layer by summing the products of the input layer neurons and those weights, win.~, which are in common with the hidden neuron, where m = 1, 2 ..... j, j being the number of neurons in the hidden layer. For example, N E T for the first neuron in the hidden layer (NEThO is calculated
The same logistic function is also applied which produces an O UTo for this neuron. During the reverse pass, an error must be calculated--starting in the output layer--and back-propagated in order to adjust the weights of the neural network. This error is the difference between what the network produces in its imperfect state (OUTo) and what the network is training to achieve (TARGET). In order to calculate this error, 6 must be computed for the output layer: 6o = ( T A R G E T -
OOUTo OUT°) ~
(5)
as"
k
NETh, = ~ (i.w~..h~) + Wbias-hl
(1)
n=l
Note that the term, Wb~a~-hl,represents the contributory effect of the bias neuron. After the N E T value is calculated for each neuron in the hidden layer, an activation function (sigrnoid) is applied to modify it, producing a variable named O U T which reveals the neuron's contributory effect on the next layer's neurons. The activation function is used to compress the value of N E T so that O U T lies between 0 and 1, thus providing a mechanism for an 'activation' or a 'non-activation' in its role to contribute to the next layer's neurons. 1
OUT=
1 + exp(-NET)
(2)
Although other activation functions were tested (and produced very similar forecasting results), the sigmoid function has a simple derivative which requires a minimal amount of CPU time to calculate; that is: ~OUT ONET - OUT(1 - O U T )
(3)
For example, the value of O U T calculated for the third neuron in the hidden layer, OUT~3, would be: 1 0 UTh~ = 1 + exp( - NETh3)
(4)
Now, each neuron in the hidden layer has a value. Each of these neurons is then multiplied by its weight which connects to the output layer, and these products are then summed to result in a NETo for the lone neuron in the output layer.
The Aw (adjustment of weight) for each weight connecting the hidden layer and the output layer is calculated using the delta rule [26]: /IWo.hm
=
qhoOUTh,, + ~(previous{dWo.h,,})
m = l , 2 ..... j
(6)
The variables q and ~ are constants between 0 and 1. The constant, q, is the learning rate which dictates how greatly the weights are changed. The constant, ~, is called the momentum coefficient which filters out high frequency changes in Aw. It is desirable to choose these constants in a manner that optimizes learning speed and accuracy. The term, previous{Awo.h,,}, is the change in weight for the previous iteration. This Aw shown above is added to the previous weight that was calculated during the previous iteration. After all weights connecting the hidden layer and output layer have been computed, it is then necessary to calculate 6s for the hidden layer so that the weights connecting the input layer and the hidden layer may also be adjusted. To adjust the weights of the hidden layer, another approach must be used to calculate the 6's of the hidden neurons because there are no targets for this layer as there were for the output layer. This is accomplished by changing the equation of 6 for each neuron in the hidden layer: OOUTh,, 6~,, = (6oWo.~,,) ~ N E T
m = 1, 2 ..... j
(7)
The weights connecting the input layer to the hidden layer may now be adjusted by an equation almost identical to the Aw above. For
209
Omega, Vol. 24, No. 2
Normali~.the learningset
I
Randomlyinitializeall weights
First iterationbegins
Shufflelearningset so the network learns and does not memorize
~
Present next learningpair to the networq
Perform a forwardpass
Performa reverse pass
NO ~
YES
J Calculateroot meansquared error for the entirelearningset by performing q forwardpasses for each learningpair
L
Next iteration
NO
cceptabl
Fig. 2. The back-propagation algorithm.
e x a m p l e , AWhl-i2 is the weight adjustment for the
weight connecting the first neuron in the hidden layer and the second neuron in the input layer and is found as follows: dWh,-i2 -----r/fhli2 + ~(previous{Awhl.n})
(8)
Also shown in Fig. 1 are open circles at the end of the first two layers. These represent what are known as bias neurons. They offset the origin of the logistic function, which allows for a more rapid convergence of the training process. The difference between a neuron and a bias neuron is that the bias neuron always has an O U T = 1 and has no input signal, it may never be affected by other neurons. However, its
weights are adjusted as are all other weights in the neural network. Forward and reverse passes are continually executed for each learning pair. A learning pair refers to the input data (economic variables) coupled with its respective target (NAV) for a given year. A learning set refers to all learning pairs that are being presented to the neural network. In our particular application, the learning set contains five learning pairs--one for each year of the input data (as discussed below). The sixth year is the forecasted period. After the last learning pair has been presented to the network, a new iteration begins. All the learning pairs in the learning set are shuffled so that the learning pairs are presented to the network in a
Chiang et al.--Mutual Fund Net Asset Value Forecasting
210
random sequence, thus ensuring that the neural network learns and does not memorize. A new iteration then begins by presenting the first learning pair. Iterations are continued until an acceptable root mean squared error is achieved. Figure 2 illustrates this process. EVALUATION OF MODELS Data sample
To evaluate the effectiveness of neural networks in forecasting the net asset values of mutual funds, 15 economic variables were identified as input to the models. They are specified and defined in Table 1. For analytical purposes, we have obtained a 6-year economic data set (1981-1986) from [12]. In addition, we also obtained historical data of end-of-year net asset values for 101 US mutual funds. There are several reasons for using this particular data set. It is expected that in certain situations, particularly when there is a limited amount of data for analysis, regression analysis will not be as appropriate for forecasting as neural networks. Therefore, the data set was selected to test this hypothesis and compare the accuracy of neural networks to that of traditional forecasting techniques. In particular, the reasons for using these data are:
(1) These are relatively short time series with many variables. Regression models are not full rank when the number of estimated parameters becomes as large as the number of observations and are unable to utilize all of the available information. Furthermore, neural networks have been shown to outperform discriminant analysis when the sample size is small [32], but no studies have analyzed the forecasting performance of neural networks with small samples. (2) One problem that regression analysis frequently has with time-series data is that of autocorrelation. Unfortunately, with short time series, it is not easily identified. Gujarati [14, p. 238] notes that "in a sample of fewer than 15 observations, it becomes very difficult to draw any definitive conclusions about autocorrelation by examining the estimated residuals". (3) Since the independent variables are all economic data and are likely to be correlated to some extent, multicollinearity issues must also be considered with regression analysis. However, if the predicted model is used for forecasting, as opposed to parameter estimation, multicollinearity is not a serious problem based
Table 1. The economic variables used in the analysis Input
Economic variable
Neuron 1
Gross National Product
Neuron 2 Neunm 3 Neuron 4 Neuron 5
Consumption demand Investment demand Government demand Net exports
Neuron 6
Gross National Product in current dollars
Neunm 7
Consumer Price Index
Neuron 8
Money, MI
Neun)n 9 Neun)n l0
Money, M2 Potential output
Neuron 11
Gross National Product gap
Neuron 12
Unemployment rate
Neuron 13
Inflation rate
Neuron 14 Neuron 15
Treasury bill rate Long-term rate
Definition The market value of the goods and services produced within a given year by US-owned labor, land, machines, tools, buildings, and raw materials The level of spending by households The level of investment spending by firms The level of US government spending The value of US goods sold to foreigners minus the value of goods the US buys from foreigners The market value of the goods and services produced within a given year by US-owned labor, land, machines, tools, buildings, and raw materials 100 times the ratio of the dollar cost of a specified collection of goods and services in a given period to the cost of the same collection in a specified based period The sum of currency in the hands of the nonbank public plus checkable deposits Part of the other monetary aggregates: Ml and M3 The level at which output would be if all resources were full) employed The difference between potential and actual real GNP. In a recession, the GNP gap is positive because potential output exceeds actual output The percentage of the labor force who are out of a job and looking for work The percentage rate of increase of the general price level per year over a specified period of time The rate of return on short-term government bonds The rate of return set by the Federal Reserve Bank of the US
Units Dollars (billions) Dollars Dollars Dollars Dollars
(billions) (billions) (billions) (billions)
Current dollars (billions) Index Dollars (billions) Dollars (billions) 1982 Dollars (billions) % % % % %
211
Omega, Vol. 24, No. 2 on the assumption that the relationship
between the independent variables will continue into the future (see [14, p. 183]). (4) While all of the data series are for US mutual funds, we have purposely included various types of mutual funds (growth funds, income funds, etc.) to evaluate neural networks against regression analysis in a variety of situations. For analyzing the neural network and regression models, the first five periods of data are used to develop the models, while data from the sixth period is then used to determine ex post forecasts and evaluate the results. In the forecasting analysis, unconditional ex post forecasts were made. That is, we assumed that the input economic data for the forecasted period was known with certainty for both the neural network and regression models. In a true forecasting environment, conditional forecasts are made and the values of the independent variables are not known but must be forecasted using other techniques. Neural network model
The input and target values were normalized so that the weights would not blow up during training. The input values were all normalized based upon the largest value in each neuron for all six periods. The targets (NAV) were also normalized based upon a 10% increase in the largest value for the first five periods. This was done to compensate for the possibility that the actual NAV for the sixth period may be higher than that of any of the previous periods. The values of r/and ~ had to be adjusted in order for the network to be able to learn. Many executions of the algorithm were performed to correctly adjust both the learning rate, q, and the momentum coefficient, ct. The best combination of the two based on these trials were found to be: ~/= 0.9~ = 0.7 Also, the number of neurons in the hidden layer, j, had to be determined. Though several values were tried, 20 neurons yielded the best results. The maximum allowable root mean squared error (RMSE) was set at 0.005. The RMSE is calculated by adding the squares of the difference between the actual output and the target for all the learning pairs and then taking the square root. A C program was written to
simulate the neural network and to implement the back-propagation algorithm, and was executed on an Apollo workstation.
Linear regression model To compare the results of the neural network to 'traditional' forecasting methods, linear regression models were also developed for each of the 101 data sets, of the form: Y = flo + ~ fl~xi
(9)
i=l
Since we purposely provided only five observations to develop the model, all fifteen independent variables cannot be included in the regression analysis. Therefore, stepwise regression was conducted using SAS to utilize the most significant variables for each model. A 15% significance level (the SAS default) was used to bring the variables into the models. Individual models were developed for each of the data sets. The resulting models included from zero independent variables--in which none of the fifteen variables met the 15% level of significance for entry to the model--to three independent variables. The models generally had surprisingly high significance; furthermore, 66 models had an R Evalue of over 0.90, 40 of them had R 2 values exceeding 0.99. Due to the availability of data, intentionally limited for this analysis, no attempts were made to measure or correct for autocorrelation or multicollinearity or to test for normality (to test the significance of the entering and exiting variables in the stepwise regression).
Nonlinear regression model To evaluate the effectiveness of nonlinear models, models were also developed for each of the data sets using a power function, of the form: n
Y = fl0I-I x(,
(10)
i=1
This particular functional form was used due to its inherent richness (as it will allow many different shapes of the curved surface) and intrinsic linearity (a logarithmic transformation allows the use of simple linear regression). As with the linear models, stepwise regression was conducted to identify the most significant variables for each model; and no attempts were
Chiang et al.--Mutual Fund Net Asset Value Forecasting
212
Table 2. Performance summary of neural network and regression forecasting Performance statistic Mean absolute percent error Median absolute percent error Geometric mean absolute percent error Maximum absolute percent error Standard deviation of absolute percent error
Neural network
Linear regression
Nonlinear regression
8.76 6.59 5.71 57.28 8.40
15.17 10.97 7.56 92.22 14.84
21.93 13.58 10.46 90.54 22.57
made to detect autocorrelation, multicollinearity, or nonnormality due to the limited amount of data. The resulting nonlinear models also included from zero to three independent variables. As would be expected, these models appeared to fit the time series even better than the linear models since the nonlinear functional form allows the model to follow the data more closely. Overall, 68 models had an R 2 value of over 0.90 and 50 of them had R 2 values exceeding 0.99. RESULTS As previously mentioned, the neural network and regression models were developed using data from 1981 to 1985. They were then evaluated using the actual 1986 NAVs compared to the predicted NAVs of each model. The economic and NAV data sets as well as the forecasted NAV and the % error for both the neural network model and the linear and nonlinear regression models are available from the authors. Table 2 summarizes the results of the forecasting. As seen from the table, the neural network performed roughly 40% better than the linear regression analysis in the ex post forecast of the 1986 NAVs and 60% better than the nonlinear model. The mean absolute percent error (MAPE) between actual and forecasted values is 8.76% for the neural network model, 15.17% for the linear regression model, and 21.93% for the nonlinear regression model. The neural network model outperformed the linear regression model on 66 of the time series (both models resulted in the same forecast for one mutual fund) and the nonlinear model on 71 of the 101 series analyzed. The linear regression model resulted in an absolute error of over 30% on fourteen of the data sets; the nonlinear model performed this poorly thirty times. However, only twice did the neural network provide an error as large. To test for the statistical significance of the difference in performance of the techniques, a Friedman test was conducted to test for differences in the forecasting effectiveness as measured
by the absolute percent error. The use of parametric tests was dismissed due to the normality assumption, since we are considering the absolute value of the errors. These values cannot be negative and, since the mean is roughly equal to the standard deviation for each model, the resulting distribution is obviously skewed. The Friedman test indicated significant differences in the techniques at the 1% level of significance. Nonparametric multiple comparisons further indicate that the neural network model performs significantly better than the linear regression model which, in turn, performs significantly better than the nonlinear regression model (again at ~ = 0.01). Thus, we can conclude that the use of neural networks is superior to regression analysis for forecasting, particularly in situations with limited data availability. As previously mentioned, we have purposely included various types of mutual funds to evaluate the relative performance of the forecasting techniques in a variety of situations. Table 3 presents the MAPE of the neural network and regression models for each of five general types of mutual funds (other measures provide similar results). As seen by the results, the conclusions reached above for NAV forecasting in general, also hold for the various types of funds. That is, neural network models tend to outperform linear regression models which, in turn, tend to outperform nonlinear regression models. What is interesting in this analysis, however, is that all of the models tend to have higher forecast errors for the more aggressive mutual funds. As one referee commented, it may be argued that less
Table 3. Mean absolute percent errors of forecasting techniques by type of mutual fund Type of fund Growth Growth income Income Balanced Bond Overall
Number of funds
network
Linear regression
Nonlinear regression
45 27 l1 10 8 101
10.56 8.90 6.81 5.97 4.34 8.76
20.32 15.31 8.32 8.50 3.47 15.17
31.20 18.98 13.17 9.82 6.93 21.93
Neural
Omega, Vol. 24, No. 2
aggressive funds are easier to forecast because of correlation with the S&P 500, the relevant stock market index. IMPLICATIONS
Overfitting One of the benefits frequently cited for the application of neural networks is that it does not require a particular functional form to be specified, it "has the ability to classify where nonlinear separation surfaces may be present" [29, p. 117]. Therefore, it may be expected that situations in which neural networks perform particularly well would also be situations in which nonlinear regression would perform better than linear regression. However, as noted in the previous section, this does not appear to be the case. What seems to be occurring is that the nonlinear regression model is able to fit the given data quite precisely, but is less capable of extending the curved surface to make accurate ex post forecasts; apparently, this model tends to overfit the data. This is evident by noting that the median R 2 value was 0.989, but the median absolute percent error was 13.58. The neural network model, on the other hand, did not appear to have a problem with overfitting even though no deliberate attempt was made to minimize its effect. (Note that Curram and Mingers [9] discuss methods of preventing overfitting neural network models.) Extrapolation A common concern of using regression analysis for forecasting is in making predictions outside the range of data used to develop the model. For 56 of the 101 data sets used in this study, the actual NAV for the sixth period lies within the range of values of the first five periods. For these data sets, the neural network model outperformed the regression models 41 times while the linear regression model performed best for 6 time series and the nonlinear regression model 9 times. Thus, it is apparent that neural networks are superior to the regression models when extrapolation is not necessary. On the other hand, for the 45 data sets in which the value for the sixth period is outside the original range of data, the neural network model provided the best forecasts only 18 times, the linear regression model was best
213
for 16 data sets, and the nonlinear model for 11 data sets. Furthermore, a Friedman test indicates no significant difference between any of the three models at the 5% level of significance. The superiority of the neural network model is not as obvious in these cases. Of course, in this analysis, the neural network models are utilizing fifteen independent variables (neurons in the input layer) while the regression models each contain three or fewer independent variables; a 'compounding' effect may contribute to the increased error. On the other hand, extrapolation may simply be as much a problem with neural networks as it is with regression analysis. The effect of extrapolation when using neural network models for prediction is an area that warrants further research.
Turning points It is important to understand that large percentage errors may occur as a result of a change in a given fund's management or investment objective. If, for example, a fund changed objectives from 'growth' to 'income' during the years from 1981 to 1986, the network may forecast poorly. This is due to the fact that neural networks train to find patterns and complex relationships. So, from the detailed results, one may possibly conclude that certain funds held a steady objective during this time period. The effect of turning points in a time series on neural network forecasting performance has not yet been investigated and warrants further consideration. CONCLUSION A back-propagation neural network has been applied to forecast the NAV for US mutual funds--specifically a three-layer network has been provided with 15 neurons in the input layer, 20 neurons in the hidden layer, and 1 neuron in the output layer. Bias neurons as well as the use of momentum have decreased the training time and improved the quality of the results. The parameters of the network were fine tuned to their best values. These forecasts were then compared to forecasts developed using regression analysis. Six periods of data were used in the analysis-five values to develop the models and one value to evaluate the models--to determine the
214
Chiang et al.--MutualFundNet Asset Value Forecasting
relative performance of these techniques in situations with limited data availability. The results of the neural network were shown to be superior to traditional econometric forecasting techniques as the neural network is able to utilize all available information; regression analysis, however, is constrained by the degrees of freedom and by the ability to evaluate only given relationships between the variables (the functional form must be specified). It should be noted that a basic, back-propagation neural network was used in this analysis; while this is a very popular method, more powerful extensions have been developed and may outperform these models.
REFERENCES 1. Bansal A, Kauffman RJ and Weitz RR (1993) Comparing the modeling performance of regression and neural networks as data quality varies: a business value approach. J. Mgmt Inform. Syst. 10, 11-32. 2. Brockett PL, Charnes A, Cooper WW, Kwon KH and Ruefli TW (1992) Chance constrained programming approach to empirical analyses of mutual fund investment strategies. Decis. Sci. 23, 385-403. 3. Brockett PL, Pitaktong U, Golden LL and Cooper WW (1993) A comparison of neural network and alternative approaches to predicting insolvency for Texas domestic property and casualty insurers. Center for Cybernetic Studies Research Report No. 707, The University of Texas at Austin. 4. Capodagiio AG, Jones HV, Novotny V and Feng X (1991) Sludge bulking analysis and forecasting: application of system identification and artificial neural computing technologies. Wat. Res. 25, 12171224. 5. Chakraborty K, Mehrotra K, Mohan CK and Ranka S (1992) Forecasting the behavior of multivariate time series using neural networks. Neural Networks 5, 961-970. 6. Chan A. and Chen CR (1992) How well do asset allocation managers allocate assets? J. Portfolio Mgmt 18, 81-91. 7. Chang EC and Lewellen WG (1984) Market timing and mutual fund investment performance. J. Busin. 57, 57-72. 8. Chen CR and Stockum S (1986) Selectivity, market timing, and random beta behavior of mutual funds: a generalized model. J. Financ. Res. 9, 87-96. 9. Curram SP and Mengers J (1994) Neural networks, decision tree induction and discriminant analysis: an empirical comparison. J. Opl Res. Soc. 45, 440450. 10. Deppisch J, Bauer HU and Geisel T (1991) Hierarchical training of neural networks and prediction of chaotic time series. Phys. Lett. A 158, 57-62. 11. Dutta S and Shekhar S (1989) An artificial intelligence approach to predicting bond ratings. In Expert Systems in Economics, Banking and Management (Edited by Pau LF, Motiwalla J, Pao YH and Teh HH), pp. 59-68. North-Holland, Amsterdam. 12. Fischer S, Dornbusch R and Schmalensee R (1988) Economics, 2nd edition. McGraw-Hill, New York.
13. Foster WR, Collopy F and Ungar LH (1992) Neural network forecasting of short, noisy time series. Computers chem. Engng 16, 293-297. 14. Gujarati D (1978) Basic Econometrics. McGraw-Hill, New York. 15. Hardgrave BC, Wilson RL and Walstrom KA (1994) Predicting graduate student success: a comparison of neural networks and traditional techniques. Computers Opns Res. 21, 249-263. 16. Hruschka H (1993) Determining market response functions by neural network modeling: a comparison to econometric techniques. Eur. J. Opl Res. 66, 27-35. 17. Huntley DG (1991) Neural nets: an approach to the forecasting of time series. Soc. Sci. Computer Rev. 9, 27-38. 18. Ippolito RA (1993) On studies of mutual fund performance, 1962-1991. Finane. Analysts J. 49, 42-50. 19. Karunanithi N, Whitley D and Malaiya YK (1992) Prediction of software reliability using connectionist models. IEEE Trans. Software Engng 18, 563-574. 20. Kon SJ (1983) The market-timing performance of mutual fund managers. J. Busin. 56, 323-347. 21. Kon SJ and Jen FC (1979) The investment performance of mutual funds: an empirical investigation of timing, selectivity and market efficiency. J. Busin. 52, 263-289. 22. Lee CF and Rahman S (1991) New evidence on timing and security selection skill of mutual fund managers. J. Portfolio Mgmt 17, 80-83. 23. Lu CN, Wu HT and Vemuri S (1993) Neural network based short term load forecasting. IEEE Trans. Power Syst. 8, 336-342. 24. Mack TH (1991) Mutual funds: a perspective for the 1990's. J. Financ. Plann. 4, 158-162. 25. Marquez L, Hill T, Worthley R and Remus W (1993) Neural network models as an alternative to regression. In Neural Networks in Finance and Investing (Edited by Trippi RR and Turban E), pp. 435-449. Probus, Chicago. 26. Rumelhart DE, Hinton GE and Williams RJ (1986) Learning internal representations by error propagation. In Parallel Distributed Processing: Explorations in the Microstructures of Cognition (Edited by Rumelhart DE, McClelland JL and the PDP Research Group), Vol. 1, pp. 318-362. MIT Press, Cambridge, Mass. 27. Salchenberger LM, Cinar EM and Lash NA (1992) Neural networks: a new tool for predicting thrift failures. Decis. Sci. 23, 899-916. 28. Sch6neburg E (1990) Stock price prediction using neural networks: a project report. Neurocomputing 2, 17-27. 29. Sharda R (1994) Neural networks for the MS/OR analyst: an application bibliography. Interfaces 24, 116-130. 30. Sharda R and Patil RB (1990) Connectionist approach to time series prediction: an empirical test. In Neural Networks in Finance and Investing (Edited by Trippi RR and Turban E), pp. 451-464. Probus, Chicago. 31. Smith AE (1994) X-bar and R control chart interpretation using neural computing. Int. J. Prod. Res. 32, 309-320. 32. Subramanian V, Hung MS and Hu MY (1993) An experimental evaluation of neural networks for classification. Computers Opns Res. 20, 769-782. 33. Tam KY (1991) Neural network models and the prediction of bank bankruptcy. Omega 19, 429-445. 34. Tam KY and Kiang MY (1992) Managerial applications of neural networks: the case of bank failure predictions. Mgmt Sci. 38, 926-947.
Omega, Vol. 24, No. 2 35. Tang Z, de Almeida C and Fishwick PA (1991) Time series forecasting using neural networks vs Box-Jenkins methodology. Simulation 57, 303-310. 36. Treynor JL and Mazuy KK (1966) Can mutual funds outguess the market? Harvard Busin. Rev. 44, 131-136. 37. Wasserman PD (1989) Neural Computing. Van Nostrand Reinhold, New York. 38. Wong FS, Wang PZ, Goh TH and Quek BK (1992) Fuzzy neural systems for stock selection. Financ. Analysts J. 48, 47-52.
215
39. Yoon Y, Swales G and Margavio TM (1993) A comparison of discriminant analysis versus artificial neural networks. J. Opl Res. Soc. 44, 51-60. Dr Wen-Chyuan Chiang, Department of Quantitative Methods and Management Information Systems, College of Business Administration, The University of Tulsa, 600 South College Avenue, Tulsa, OK 74104-3189, USA.
ADDRESS FOR CORRESPONDENCE:
0305-0483(95)00059-3
Copyright© 1996ElsevierScienceLtd Printedin GreatBritain.All fightsreserved 0305-0483/96$15.00+ 0.00
A Neural Network Approach to Mutual Fund Net Asset Value Forecasting W-C CHIANG TL URBAN The University of Tulsa, Tulsa, Okla, USA GW BALDRIDGE WorldCom, Inc., Tulsa, Okla, USA (Received March 1995; accepted after revision November 1995) In this paper, an artificial neural network method is applied to forecast the end-of-year net asset value (NAV) of mutual funds. The back-propagation neural network is identified and explained. Historical economic information is used for the prediction of NAV data. The results of the forecasting are compared to those of traditional econometric techniques (i.e. linear and nonlinear regression analysis), and it is shown that neural networks significantly outperform regression models in situations with limited data availability. Copyright © 1996 Elsevier Science Ltd
Key words--forecasting, neural networks, mutual funds
INTRODUCTION A MUTUAL FUND CONSISTS of a diversified portfolio of stocks and bonds that is managed by professionally trained individuals. It has become America's investment vehicle of choice. By 1990, the assets in the variety of US mutual funds, such as income and bond funds, money market funds, and equity funds, have exceeded $1 trillion [24]. Very much like portfolios of individual securities, it has provided an opportunity for investors (individual as well as institutional) to accommodate risk-return tradeoffs as well as diversification, liquidity, and tax purposes. Undoubtedly, a great deal of attention in the financial economics literature has been devoted to the evaluation of the performance of mutual funds. Most of these studies have focused on the forecasting skills of mutual fund managers, which are composed of OME24/2--G
(1) forecasts of price movements of selected stocks in the funds (stock selection) and (2) forecasts of price movements of the general stock market as a whole (market timing). However, conflicting results have been generated from these studies (see, for example, [6-8, 20-22, 36]). Empirical studies using multiple regression techniques are the major tools in these studies. Recently, Brockett et al. [2] formulated chance constrained programming models, exploring risk-return tradeoffs, to effect investors' choice of mutual funds. Efficient market theory states that all information relevant to an asset will be reflected in its price. In a perfectly efficient market, prices of an asset always reflect all known information, prices adjust instantaneously to new information, and speculation succeeds only as a matter of luck. In an economically efficient market, prices may not react to new information 205
2"06
Chiang et al.--Mutual Fund Net Asset Value Forecasting
instantaneously, but over the long run, speculation cannot be rewarded after transaction costs are paid. Although frequently challenged and sometimes refuted by prior studies, the hypothesis of efficient markets is still commonly believed to be true (see [18] for a review of the research in this area). Mutual funds are diversified portfolios of individual assets. Therefore, the prices of mutual funds should also reflect the known economic information either instantaneously or over the long run. However, economists and financial managers are at a loss to relate varieties of economic variables that are associated with market advance and decline. Ideally, one would like a derived equation relating these complex relationships. Neural network techniques can provide means by which these economic relationships may be exploited. A neural network is a system that performs a mathematical mapping from one domain to another by use of algorithmic techniques. In the case of mutual fund forecasting, the network would map from units such as nominal dollars, real dollars, percents, and indices to the net asset value (NAV) of the mutual fund (dollars). There has been a considerable amount of research conducted using neural networks for purposes of prediction. Tam and Kiang [33, 34] used neural networks to predict bank failures and found the results to be superior to discriminant analysis techniques. Salchenberger et al. [27] predicted thrift failures with neural networks and achieved better results than logit models. Dutta and Shekhar [1 l] illustrated that neural networks outperform regression analysis in predicting bond ratings. Brockett et al. [3] found that neural networks were better at predicting insolvency of insurance companies than a system designed specifically for this purpose that utilizes financial ratios. Other researchers have also used neural networks to predict stock price performance [39], software reliability [19], academic success of graduate students [15], power system load [23], 'out-of-control' production processes [31], and sludge bulking in wastewater treatment plants [4] and have generally found the results better than those of the traditional analytical techniques. Sharda [29] presented a survey of the literature on applications of neural networks and noted that neural network models performed better than the traditional
statistical techniques 71% of the time (30 of 42 studies). Most of the research concerning neural networks for economic forecasting purposes has been in the area of time-series forecasting. The use of neural networks has been frequently compared to the use of traditional time-series forecasting techniques (e.g. Box-Jenkins) and has been applied in a variety of situations (see, for example, [5, 10, 13, 17, 30, 35, 38]). Far less research has been conducted using neural networks for causal or econometric modeling. Hruschka [16] compared neural network models to an econometric model of market response and Marquez et al. [25] evaluated the performance of neural network models and regression models using generated data. Both of these studies, though, made the comparisons with regard only to the fit of the data, and did not extend the comparisons to encompass forecasting effectiveness. Sch6neburg [28] used neural networks to forecast stock prices using several 'independent' variables; however, these variables were all related to the stock price being predicted (e.g. the variation in the stock price from the previous day) and did not consider any exogenous variables. Bansal et al. [1] used neural networks to forecast prepayment rates in mortgage-backed securities portfolio management; however, they stated "forecasting prepayments requires large data sets" and utilized 1170 observations for their analysis. The purpose of this research is to evaluate the use of neural networks to develop forecasting models for predicting end-of-year net asset values of various US mutual funds. Various economic variables--such as gross national product, consumer price index, inflation rate, etc.--will be used as exogenous variables for the models. Since many mutual funds have only recently been established, it is important that a forecasting technique be able to perform reliably with minimal data requirements; thus, we will focus specifically on the situation in which a limited amount of data is available for analysis (in particular, using five years of data to forecast year six). Of course, many other practical situations arise in which forecasts must be made with few observations. The results will be compared to the use of traditional causal forecasting techniques--regression analysis, analyzing both linear and nonlinear models--to determine the relative performance under these conditions.
Omega, Vol. 24, No. 2
THE BACK-PROPAGATION NEURAL NETWORK FOR NAV FORECASTING
by using an algorithm known as back-propagation (for a detailed, introductory description see [37]).
The network architecture The specific back-propagation neural network used to forecast the NAVs consists of an input layer, a hidden layer, and an output layer, as shown in Fig. 1. Each neuron in the input layer contains known information (e.g. the unemployment rate). Lines, representing weights, connect the input layer to the hidden layer. These weights are used to model the synaptic strength between neurons in the human brain, thus portraying the importance of one neuron's 'effect' on the next layer's neuron. This 'effect' represents the 'activation level' of the neuron. The hidden layer is also made up of neurons, but the values of these continuously change as the network is training. Weights also connect the hidden layer to the output layer. The output layer consists of only one neuron because we are seeking only one piece of data--the NAV of a mutual fund at year's end. The open-circled neurons at the end of both the input layer and the hidden layer are indicative of the neuron biases which allow for a more rapid convergence of the training process. The neural network forecasts the NAV
Neuron: i I
Neuron: i 2
207
The back-propagation training method The back-propagation algorithm is used to train ('adjust') the weights in the neural network so that new, never-before seen information entering the input layer will yield astute results (correctly forecasted NAVs in our case) in the output layer. The actual training of the network is accomplished by 'back-propagating' the error from the output layer, to the hidden layer, and finally to the input layer while changing the weights simultaneously. The error, in our case, is simply the difference between the desired output (a known NAV dollar amount) and the output calculated during training, using the algorithm explained below. Essentially, steps are performed to map mathematically from 'economic variables' to 'Net Asset Value'. This is accomplished using equations that attempt to mimic the human brain. A neuron may activate strongly, weakly, or somewhere in between based upon the activation levels of the previous layer's neurons. There are two steps to the algorithm: the forward pass and the reverse pass.
Neuron: ik. 1
Neuron: i t
Neuron: bias
INPUT LAYER
HIDDEN LAYER as
OUTPU LAYER N{~uron." o Fig. 1. The back-propagation neural network.
Chiang et al.--Mutual Fund Net Asset Value Forecasting
208
During the forward pass, known economic information enters the input layer which is specified as i,, where n = 1, 2 ..... k, k being the number of neurons in the input layer. One economic variable is applied to each neuron in the input layer. A variable, N E T , is then calculated for each neuron in the hidden layer by summing the products of the input layer neurons and those weights, win.~, which are in common with the hidden neuron, where m = 1, 2 ..... j, j being the number of neurons in the hidden layer. For example, N E T for the first neuron in the hidden layer (NEThO is calculated
The same logistic function is also applied which produces an O UTo for this neuron. During the reverse pass, an error must be calculated--starting in the output layer--and back-propagated in order to adjust the weights of the neural network. This error is the difference between what the network produces in its imperfect state (OUTo) and what the network is training to achieve (TARGET). In order to calculate this error, 6 must be computed for the output layer: 6o = ( T A R G E T -
OOUTo OUT°) ~
(5)
as"
k
NETh, = ~ (i.w~..h~) + Wbias-hl
(1)
n=l
Note that the term, Wb~a~-hl,represents the contributory effect of the bias neuron. After the N E T value is calculated for each neuron in the hidden layer, an activation function (sigrnoid) is applied to modify it, producing a variable named O U T which reveals the neuron's contributory effect on the next layer's neurons. The activation function is used to compress the value of N E T so that O U T lies between 0 and 1, thus providing a mechanism for an 'activation' or a 'non-activation' in its role to contribute to the next layer's neurons. 1
OUT=
1 + exp(-NET)
(2)
Although other activation functions were tested (and produced very similar forecasting results), the sigmoid function has a simple derivative which requires a minimal amount of CPU time to calculate; that is: ~OUT ONET - OUT(1 - O U T )
(3)
For example, the value of O U T calculated for the third neuron in the hidden layer, OUT~3, would be: 1 0 UTh~ = 1 + exp( - NETh3)
(4)
Now, each neuron in the hidden layer has a value. Each of these neurons is then multiplied by its weight which connects to the output layer, and these products are then summed to result in a NETo for the lone neuron in the output layer.
The Aw (adjustment of weight) for each weight connecting the hidden layer and the output layer is calculated using the delta rule [26]: /IWo.hm
=
qhoOUTh,, + ~(previous{dWo.h,,})
m = l , 2 ..... j
(6)
The variables q and ~ are constants between 0 and 1. The constant, q, is the learning rate which dictates how greatly the weights are changed. The constant, ~, is called the momentum coefficient which filters out high frequency changes in Aw. It is desirable to choose these constants in a manner that optimizes learning speed and accuracy. The term, previous{Awo.h,,}, is the change in weight for the previous iteration. This Aw shown above is added to the previous weight that was calculated during the previous iteration. After all weights connecting the hidden layer and output layer have been computed, it is then necessary to calculate 6s for the hidden layer so that the weights connecting the input layer and the hidden layer may also be adjusted. To adjust the weights of the hidden layer, another approach must be used to calculate the 6's of the hidden neurons because there are no targets for this layer as there were for the output layer. This is accomplished by changing the equation of 6 for each neuron in the hidden layer: OOUTh,, 6~,, = (6oWo.~,,) ~ N E T
m = 1, 2 ..... j
(7)
The weights connecting the input layer to the hidden layer may now be adjusted by an equation almost identical to the Aw above. For
209
Omega, Vol. 24, No. 2
Normali~.the learningset
I
Randomlyinitializeall weights
First iterationbegins
Shufflelearningset so the network learns and does not memorize
~
Present next learningpair to the networq
Perform a forwardpass
Performa reverse pass
NO ~
YES
J Calculateroot meansquared error for the entirelearningset by performing q forwardpasses for each learningpair
L
Next iteration
NO
cceptabl
Fig. 2. The back-propagation algorithm.
e x a m p l e , AWhl-i2 is the weight adjustment for the
weight connecting the first neuron in the hidden layer and the second neuron in the input layer and is found as follows: dWh,-i2 -----r/fhli2 + ~(previous{Awhl.n})
(8)
Also shown in Fig. 1 are open circles at the end of the first two layers. These represent what are known as bias neurons. They offset the origin of the logistic function, which allows for a more rapid convergence of the training process. The difference between a neuron and a bias neuron is that the bias neuron always has an O U T = 1 and has no input signal, it may never be affected by other neurons. However, its
weights are adjusted as are all other weights in the neural network. Forward and reverse passes are continually executed for each learning pair. A learning pair refers to the input data (economic variables) coupled with its respective target (NAV) for a given year. A learning set refers to all learning pairs that are being presented to the neural network. In our particular application, the learning set contains five learning pairs--one for each year of the input data (as discussed below). The sixth year is the forecasted period. After the last learning pair has been presented to the network, a new iteration begins. All the learning pairs in the learning set are shuffled so that the learning pairs are presented to the network in a
Chiang et al.--Mutual Fund Net Asset Value Forecasting
210
random sequence, thus ensuring that the neural network learns and does not memorize. A new iteration then begins by presenting the first learning pair. Iterations are continued until an acceptable root mean squared error is achieved. Figure 2 illustrates this process. EVALUATION OF MODELS Data sample
To evaluate the effectiveness of neural networks in forecasting the net asset values of mutual funds, 15 economic variables were identified as input to the models. They are specified and defined in Table 1. For analytical purposes, we have obtained a 6-year economic data set (1981-1986) from [12]. In addition, we also obtained historical data of end-of-year net asset values for 101 US mutual funds. There are several reasons for using this particular data set. It is expected that in certain situations, particularly when there is a limited amount of data for analysis, regression analysis will not be as appropriate for forecasting as neural networks. Therefore, the data set was selected to test this hypothesis and compare the accuracy of neural networks to that of traditional forecasting techniques. In particular, the reasons for using these data are:
(1) These are relatively short time series with many variables. Regression models are not full rank when the number of estimated parameters becomes as large as the number of observations and are unable to utilize all of the available information. Furthermore, neural networks have been shown to outperform discriminant analysis when the sample size is small [32], but no studies have analyzed the forecasting performance of neural networks with small samples. (2) One problem that regression analysis frequently has with time-series data is that of autocorrelation. Unfortunately, with short time series, it is not easily identified. Gujarati [14, p. 238] notes that "in a sample of fewer than 15 observations, it becomes very difficult to draw any definitive conclusions about autocorrelation by examining the estimated residuals". (3) Since the independent variables are all economic data and are likely to be correlated to some extent, multicollinearity issues must also be considered with regression analysis. However, if the predicted model is used for forecasting, as opposed to parameter estimation, multicollinearity is not a serious problem based
Table 1. The economic variables used in the analysis Input
Economic variable
Neuron 1
Gross National Product
Neuron 2 Neunm 3 Neuron 4 Neuron 5
Consumption demand Investment demand Government demand Net exports
Neuron 6
Gross National Product in current dollars
Neunm 7
Consumer Price Index
Neuron 8
Money, MI
Neun)n 9 Neun)n l0
Money, M2 Potential output
Neuron 11
Gross National Product gap
Neuron 12
Unemployment rate
Neuron 13
Inflation rate
Neuron 14 Neuron 15
Treasury bill rate Long-term rate
Definition The market value of the goods and services produced within a given year by US-owned labor, land, machines, tools, buildings, and raw materials The level of spending by households The level of investment spending by firms The level of US government spending The value of US goods sold to foreigners minus the value of goods the US buys from foreigners The market value of the goods and services produced within a given year by US-owned labor, land, machines, tools, buildings, and raw materials 100 times the ratio of the dollar cost of a specified collection of goods and services in a given period to the cost of the same collection in a specified based period The sum of currency in the hands of the nonbank public plus checkable deposits Part of the other monetary aggregates: Ml and M3 The level at which output would be if all resources were full) employed The difference between potential and actual real GNP. In a recession, the GNP gap is positive because potential output exceeds actual output The percentage of the labor force who are out of a job and looking for work The percentage rate of increase of the general price level per year over a specified period of time The rate of return on short-term government bonds The rate of return set by the Federal Reserve Bank of the US
Units Dollars (billions) Dollars Dollars Dollars Dollars
(billions) (billions) (billions) (billions)
Current dollars (billions) Index Dollars (billions) Dollars (billions) 1982 Dollars (billions) % % % % %
211
Omega, Vol. 24, No. 2 on the assumption that the relationship
between the independent variables will continue into the future (see [14, p. 183]). (4) While all of the data series are for US mutual funds, we have purposely included various types of mutual funds (growth funds, income funds, etc.) to evaluate neural networks against regression analysis in a variety of situations. For analyzing the neural network and regression models, the first five periods of data are used to develop the models, while data from the sixth period is then used to determine ex post forecasts and evaluate the results. In the forecasting analysis, unconditional ex post forecasts were made. That is, we assumed that the input economic data for the forecasted period was known with certainty for both the neural network and regression models. In a true forecasting environment, conditional forecasts are made and the values of the independent variables are not known but must be forecasted using other techniques. Neural network model
The input and target values were normalized so that the weights would not blow up during training. The input values were all normalized based upon the largest value in each neuron for all six periods. The targets (NAV) were also normalized based upon a 10% increase in the largest value for the first five periods. This was done to compensate for the possibility that the actual NAV for the sixth period may be higher than that of any of the previous periods. The values of r/and ~ had to be adjusted in order for the network to be able to learn. Many executions of the algorithm were performed to correctly adjust both the learning rate, q, and the momentum coefficient, ct. The best combination of the two based on these trials were found to be: ~/= 0.9~ = 0.7 Also, the number of neurons in the hidden layer, j, had to be determined. Though several values were tried, 20 neurons yielded the best results. The maximum allowable root mean squared error (RMSE) was set at 0.005. The RMSE is calculated by adding the squares of the difference between the actual output and the target for all the learning pairs and then taking the square root. A C program was written to
simulate the neural network and to implement the back-propagation algorithm, and was executed on an Apollo workstation.
Linear regression model To compare the results of the neural network to 'traditional' forecasting methods, linear regression models were also developed for each of the 101 data sets, of the form: Y = flo + ~ fl~xi
(9)
i=l
Since we purposely provided only five observations to develop the model, all fifteen independent variables cannot be included in the regression analysis. Therefore, stepwise regression was conducted using SAS to utilize the most significant variables for each model. A 15% significance level (the SAS default) was used to bring the variables into the models. Individual models were developed for each of the data sets. The resulting models included from zero independent variables--in which none of the fifteen variables met the 15% level of significance for entry to the model--to three independent variables. The models generally had surprisingly high significance; furthermore, 66 models had an R Evalue of over 0.90, 40 of them had R 2 values exceeding 0.99. Due to the availability of data, intentionally limited for this analysis, no attempts were made to measure or correct for autocorrelation or multicollinearity or to test for normality (to test the significance of the entering and exiting variables in the stepwise regression).
Nonlinear regression model To evaluate the effectiveness of nonlinear models, models were also developed for each of the data sets using a power function, of the form: n
Y = fl0I-I x(,
(10)
i=1
This particular functional form was used due to its inherent richness (as it will allow many different shapes of the curved surface) and intrinsic linearity (a logarithmic transformation allows the use of simple linear regression). As with the linear models, stepwise regression was conducted to identify the most significant variables for each model; and no attempts were
Chiang et al.--Mutual Fund Net Asset Value Forecasting
212
Table 2. Performance summary of neural network and regression forecasting Performance statistic Mean absolute percent error Median absolute percent error Geometric mean absolute percent error Maximum absolute percent error Standard deviation of absolute percent error
Neural network
Linear regression
Nonlinear regression
8.76 6.59 5.71 57.28 8.40
15.17 10.97 7.56 92.22 14.84
21.93 13.58 10.46 90.54 22.57
made to detect autocorrelation, multicollinearity, or nonnormality due to the limited amount of data. The resulting nonlinear models also included from zero to three independent variables. As would be expected, these models appeared to fit the time series even better than the linear models since the nonlinear functional form allows the model to follow the data more closely. Overall, 68 models had an R 2 value of over 0.90 and 50 of them had R 2 values exceeding 0.99. RESULTS As previously mentioned, the neural network and regression models were developed using data from 1981 to 1985. They were then evaluated using the actual 1986 NAVs compared to the predicted NAVs of each model. The economic and NAV data sets as well as the forecasted NAV and the % error for both the neural network model and the linear and nonlinear regression models are available from the authors. Table 2 summarizes the results of the forecasting. As seen from the table, the neural network performed roughly 40% better than the linear regression analysis in the ex post forecast of the 1986 NAVs and 60% better than the nonlinear model. The mean absolute percent error (MAPE) between actual and forecasted values is 8.76% for the neural network model, 15.17% for the linear regression model, and 21.93% for the nonlinear regression model. The neural network model outperformed the linear regression model on 66 of the time series (both models resulted in the same forecast for one mutual fund) and the nonlinear model on 71 of the 101 series analyzed. The linear regression model resulted in an absolute error of over 30% on fourteen of the data sets; the nonlinear model performed this poorly thirty times. However, only twice did the neural network provide an error as large. To test for the statistical significance of the difference in performance of the techniques, a Friedman test was conducted to test for differences in the forecasting effectiveness as measured
by the absolute percent error. The use of parametric tests was dismissed due to the normality assumption, since we are considering the absolute value of the errors. These values cannot be negative and, since the mean is roughly equal to the standard deviation for each model, the resulting distribution is obviously skewed. The Friedman test indicated significant differences in the techniques at the 1% level of significance. Nonparametric multiple comparisons further indicate that the neural network model performs significantly better than the linear regression model which, in turn, performs significantly better than the nonlinear regression model (again at ~ = 0.01). Thus, we can conclude that the use of neural networks is superior to regression analysis for forecasting, particularly in situations with limited data availability. As previously mentioned, we have purposely included various types of mutual funds to evaluate the relative performance of the forecasting techniques in a variety of situations. Table 3 presents the MAPE of the neural network and regression models for each of five general types of mutual funds (other measures provide similar results). As seen by the results, the conclusions reached above for NAV forecasting in general, also hold for the various types of funds. That is, neural network models tend to outperform linear regression models which, in turn, tend to outperform nonlinear regression models. What is interesting in this analysis, however, is that all of the models tend to have higher forecast errors for the more aggressive mutual funds. As one referee commented, it may be argued that less
Table 3. Mean absolute percent errors of forecasting techniques by type of mutual fund Type of fund Growth Growth income Income Balanced Bond Overall
Number of funds
network
Linear regression
Nonlinear regression
45 27 l1 10 8 101
10.56 8.90 6.81 5.97 4.34 8.76
20.32 15.31 8.32 8.50 3.47 15.17
31.20 18.98 13.17 9.82 6.93 21.93
Neural
Omega, Vol. 24, No. 2
aggressive funds are easier to forecast because of correlation with the S&P 500, the relevant stock market index. IMPLICATIONS
Overfitting One of the benefits frequently cited for the application of neural networks is that it does not require a particular functional form to be specified, it "has the ability to classify where nonlinear separation surfaces may be present" [29, p. 117]. Therefore, it may be expected that situations in which neural networks perform particularly well would also be situations in which nonlinear regression would perform better than linear regression. However, as noted in the previous section, this does not appear to be the case. What seems to be occurring is that the nonlinear regression model is able to fit the given data quite precisely, but is less capable of extending the curved surface to make accurate ex post forecasts; apparently, this model tends to overfit the data. This is evident by noting that the median R 2 value was 0.989, but the median absolute percent error was 13.58. The neural network model, on the other hand, did not appear to have a problem with overfitting even though no deliberate attempt was made to minimize its effect. (Note that Curram and Mingers [9] discuss methods of preventing overfitting neural network models.) Extrapolation A common concern of using regression analysis for forecasting is in making predictions outside the range of data used to develop the model. For 56 of the 101 data sets used in this study, the actual NAV for the sixth period lies within the range of values of the first five periods. For these data sets, the neural network model outperformed the regression models 41 times while the linear regression model performed best for 6 time series and the nonlinear regression model 9 times. Thus, it is apparent that neural networks are superior to the regression models when extrapolation is not necessary. On the other hand, for the 45 data sets in which the value for the sixth period is outside the original range of data, the neural network model provided the best forecasts only 18 times, the linear regression model was best
213
for 16 data sets, and the nonlinear model for 11 data sets. Furthermore, a Friedman test indicates no significant difference between any of the three models at the 5% level of significance. The superiority of the neural network model is not as obvious in these cases. Of course, in this analysis, the neural network models are utilizing fifteen independent variables (neurons in the input layer) while the regression models each contain three or fewer independent variables; a 'compounding' effect may contribute to the increased error. On the other hand, extrapolation may simply be as much a problem with neural networks as it is with regression analysis. The effect of extrapolation when using neural network models for prediction is an area that warrants further research.
Turning points It is important to understand that large percentage errors may occur as a result of a change in a given fund's management or investment objective. If, for example, a fund changed objectives from 'growth' to 'income' during the years from 1981 to 1986, the network may forecast poorly. This is due to the fact that neural networks train to find patterns and complex relationships. So, from the detailed results, one may possibly conclude that certain funds held a steady objective during this time period. The effect of turning points in a time series on neural network forecasting performance has not yet been investigated and warrants further consideration. CONCLUSION A back-propagation neural network has been applied to forecast the NAV for US mutual funds--specifically a three-layer network has been provided with 15 neurons in the input layer, 20 neurons in the hidden layer, and 1 neuron in the output layer. Bias neurons as well as the use of momentum have decreased the training time and improved the quality of the results. The parameters of the network were fine tuned to their best values. These forecasts were then compared to forecasts developed using regression analysis. Six periods of data were used in the analysis-five values to develop the models and one value to evaluate the models--to determine the
214
Chiang et al.--MutualFundNet Asset Value Forecasting
relative performance of these techniques in situations with limited data availability. The results of the neural network were shown to be superior to traditional econometric forecasting techniques as the neural network is able to utilize all available information; regression analysis, however, is constrained by the degrees of freedom and by the ability to evaluate only given relationships between the variables (the functional form must be specified). It should be noted that a basic, back-propagation neural network was used in this analysis; while this is a very popular method, more powerful extensions have been developed and may outperform these models.
REFERENCES 1. Bansal A, Kauffman RJ and Weitz RR (1993) Comparing the modeling performance of regression and neural networks as data quality varies: a business value approach. J. Mgmt Inform. Syst. 10, 11-32. 2. Brockett PL, Charnes A, Cooper WW, Kwon KH and Ruefli TW (1992) Chance constrained programming approach to empirical analyses of mutual fund investment strategies. Decis. Sci. 23, 385-403. 3. Brockett PL, Pitaktong U, Golden LL and Cooper WW (1993) A comparison of neural network and alternative approaches to predicting insolvency for Texas domestic property and casualty insurers. Center for Cybernetic Studies Research Report No. 707, The University of Texas at Austin. 4. Capodagiio AG, Jones HV, Novotny V and Feng X (1991) Sludge bulking analysis and forecasting: application of system identification and artificial neural computing technologies. Wat. Res. 25, 12171224. 5. Chakraborty K, Mehrotra K, Mohan CK and Ranka S (1992) Forecasting the behavior of multivariate time series using neural networks. Neural Networks 5, 961-970. 6. Chan A. and Chen CR (1992) How well do asset allocation managers allocate assets? J. Portfolio Mgmt 18, 81-91. 7. Chang EC and Lewellen WG (1984) Market timing and mutual fund investment performance. J. Busin. 57, 57-72. 8. Chen CR and Stockum S (1986) Selectivity, market timing, and random beta behavior of mutual funds: a generalized model. J. Financ. Res. 9, 87-96. 9. Curram SP and Mengers J (1994) Neural networks, decision tree induction and discriminant analysis: an empirical comparison. J. Opl Res. Soc. 45, 440450. 10. Deppisch J, Bauer HU and Geisel T (1991) Hierarchical training of neural networks and prediction of chaotic time series. Phys. Lett. A 158, 57-62. 11. Dutta S and Shekhar S (1989) An artificial intelligence approach to predicting bond ratings. In Expert Systems in Economics, Banking and Management (Edited by Pau LF, Motiwalla J, Pao YH and Teh HH), pp. 59-68. North-Holland, Amsterdam. 12. Fischer S, Dornbusch R and Schmalensee R (1988) Economics, 2nd edition. McGraw-Hill, New York.
13. Foster WR, Collopy F and Ungar LH (1992) Neural network forecasting of short, noisy time series. Computers chem. Engng 16, 293-297. 14. Gujarati D (1978) Basic Econometrics. McGraw-Hill, New York. 15. Hardgrave BC, Wilson RL and Walstrom KA (1994) Predicting graduate student success: a comparison of neural networks and traditional techniques. Computers Opns Res. 21, 249-263. 16. Hruschka H (1993) Determining market response functions by neural network modeling: a comparison to econometric techniques. Eur. J. Opl Res. 66, 27-35. 17. Huntley DG (1991) Neural nets: an approach to the forecasting of time series. Soc. Sci. Computer Rev. 9, 27-38. 18. Ippolito RA (1993) On studies of mutual fund performance, 1962-1991. Finane. Analysts J. 49, 42-50. 19. Karunanithi N, Whitley D and Malaiya YK (1992) Prediction of software reliability using connectionist models. IEEE Trans. Software Engng 18, 563-574. 20. Kon SJ (1983) The market-timing performance of mutual fund managers. J. Busin. 56, 323-347. 21. Kon SJ and Jen FC (1979) The investment performance of mutual funds: an empirical investigation of timing, selectivity and market efficiency. J. Busin. 52, 263-289. 22. Lee CF and Rahman S (1991) New evidence on timing and security selection skill of mutual fund managers. J. Portfolio Mgmt 17, 80-83. 23. Lu CN, Wu HT and Vemuri S (1993) Neural network based short term load forecasting. IEEE Trans. Power Syst. 8, 336-342. 24. Mack TH (1991) Mutual funds: a perspective for the 1990's. J. Financ. Plann. 4, 158-162. 25. Marquez L, Hill T, Worthley R and Remus W (1993) Neural network models as an alternative to regression. In Neural Networks in Finance and Investing (Edited by Trippi RR and Turban E), pp. 435-449. Probus, Chicago. 26. Rumelhart DE, Hinton GE and Williams RJ (1986) Learning internal representations by error propagation. In Parallel Distributed Processing: Explorations in the Microstructures of Cognition (Edited by Rumelhart DE, McClelland JL and the PDP Research Group), Vol. 1, pp. 318-362. MIT Press, Cambridge, Mass. 27. Salchenberger LM, Cinar EM and Lash NA (1992) Neural networks: a new tool for predicting thrift failures. Decis. Sci. 23, 899-916. 28. Sch6neburg E (1990) Stock price prediction using neural networks: a project report. Neurocomputing 2, 17-27. 29. Sharda R (1994) Neural networks for the MS/OR analyst: an application bibliography. Interfaces 24, 116-130. 30. Sharda R and Patil RB (1990) Connectionist approach to time series prediction: an empirical test. In Neural Networks in Finance and Investing (Edited by Trippi RR and Turban E), pp. 451-464. Probus, Chicago. 31. Smith AE (1994) X-bar and R control chart interpretation using neural computing. Int. J. Prod. Res. 32, 309-320. 32. Subramanian V, Hung MS and Hu MY (1993) An experimental evaluation of neural networks for classification. Computers Opns Res. 20, 769-782. 33. Tam KY (1991) Neural network models and the prediction of bank bankruptcy. Omega 19, 429-445. 34. Tam KY and Kiang MY (1992) Managerial applications of neural networks: the case of bank failure predictions. Mgmt Sci. 38, 926-947.
Omega, Vol. 24, No. 2 35. Tang Z, de Almeida C and Fishwick PA (1991) Time series forecasting using neural networks vs Box-Jenkins methodology. Simulation 57, 303-310. 36. Treynor JL and Mazuy KK (1966) Can mutual funds outguess the market? Harvard Busin. Rev. 44, 131-136. 37. Wasserman PD (1989) Neural Computing. Van Nostrand Reinhold, New York. 38. Wong FS, Wang PZ, Goh TH and Quek BK (1992) Fuzzy neural systems for stock selection. Financ. Analysts J. 48, 47-52.
215
39. Yoon Y, Swales G and Margavio TM (1993) A comparison of discriminant analysis versus artificial neural networks. J. Opl Res. Soc. 44, 51-60. Dr Wen-Chyuan Chiang, Department of Quantitative Methods and Management Information Systems, College of Business Administration, The University of Tulsa, 600 South College Avenue, Tulsa, OK 74104-3189, USA.
ADDRESS FOR CORRESPONDENCE: