- Email: [email protected]
A psychophysical model of decision making
Physica A 389 (2010) 3580–3587
Contents lists available at ScienceDirect
Physica A journal homepage: www.elsevier.com/locate/physa
A psychophysical model of decision making B.J. West a,∗ , P. Grigolini b a
Information Science Directorate, US Army Research Office, Durham, NC 27709, United States
b
Center for Nonlinear Science, University of North Texas, United States
article
info
Article history: Received 26 January 2010 Received in revised form 12 March 2010 Available online 21 April 2010 Keywords: Decision making Probability Inverse power law 1/f noise
abstract Herein we develop a psychophysical model of decision making based on the difference between objective clock time and the human brain’s perception of time. In this model the utility function is given by the survival probability, which is shown to be a generalized hyperbolic distribution. The parameters of the utility function are fit to intertemporal choice model experimental data and decision making is determined to be a 1/f -noise process. Published by Elsevier B.V.
1. Introduction The history of probability is partly a history of how we make choices with incomplete knowledge and under conditions of uncertainty. Consequently, it is not surprising that the mathematical formalism of probability theory was initiated in that most mysterious of human activities: gambling. Choice enters gambling in the determination and selection of the most likely outcomes in games of chance. The implicit assumption in a wager is that the antagonists want to win and therefore in these initial efforts at formulating a general theory a great deal of attention was paid to developing measures of play that could be used to enhance the likelihood of winning or at least decrease the likelihood of losing. One measure of play is, of course, the expected outcome of a wager. If the possible outcomes of play are indexed j = 1 to M, then the probability of the jth outcome is pj , wj is the wager for this outcome and the expected winnings are
hW i =
M X
pj wj .
(1)
j =1
From this perspective the expected winnings provided a guide for the bettor and was historically indicative of the outcome of a single play since the probability had a subjective interpretation. Even with this non-binding estimate of how well a bettor may do, it was quite a surprise when in 1713 N. Bernoulli introduced what appeared to be a reasonable game that resulted in a diverging expectation value hW i = ∞. In general this divergence of the average occurs when the relative sizes of the terms in series (1) satisfy the condition p j +1 w j +1 pj wj
≥ 1 as M → ∞.
(2)
The particular case pj = 1/2j with the wager of wj = 2j ducats is called the St. Petersburg Paradox, named after the journal in which the problem was eventually published. The paradox arose because the bettor and the house could not agree on an
∗
Corresponding author. E-mail addresses: [email protected], [email protected] (B.J. West).
0378-4371/$ – see front matter. Published by Elsevier B.V. doi:10.1016/j.physa.2010.03.039
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
3581
ante. The house wanted a large ante because the expected winnings are infinite and the bettor wanted a small ante because half the time he would only win 2 ducats. In 1731 D. Bernoulli gave one resolution of the paradox by introducing the concept of a utility function and published the result in 1738 [1]. He argued that not everyone should wager in the same way since the subjective value or utility associated with a unit of currency depends on one’s total wealth. Consequently, he proposed estimating the ‘winnings’ by using the average utility of a wager
hU i =
M X
pj U wj
(3)
j=1
and not the wager itself. It is the average utility that one ought to maximize and not the expected winnings. The mathematical wrangling over the resolution of the St. Petersburg Paradox lasted another two centuries, but that discussion does not concern us here except to note that much of the argument was focused on the difference between subjective and objective probability. The final resolution of what constitutes a probability in the physical sciences is that we interpret the probability objectively as a relative frequency; the number of times the play lands on the jth outcome Nj relative to the total large number of plays N Nj
. (4) N However the concern over the interpretation of probability has today been again taken up by the cognitive sciences, which distinguish between the probability used in the physical sciences from that used in the determination of decision-making activities. Hess and Hastie [2] point out that behavioral-decision research indicates that people fail to choose what makes them happy, either because they fail to predict what will make them happy or they fail to act on their predictions. They go on to examine five biases that distort the ability to make accurate predictions and four mechanism that inhibit acting on the predictions made. Ariely [3] takes the position that undesirable outcomes can be predicted and avoided, as he argues in his interesting book, which explains the fundamentally irrational nature of many decisions. Wittmann and Paulus [4] point out that psychology does not dismiss rationality, but emphasize that other factors such as satisfaction and impulsivity become of equal or more importance in the psychology of decision making. One of the most studied emotions that mitigate rational decision making is regret [5] with regret aversion often dictate the decision-making process [6]. However, it was also shown that curiosity, which can be associated with impulsive behavior, can suppress or kill regret [6]. On a broader stage sociology examines how peer pressure and leadership often dominate the discussion for decision making in social networks, as discussed by Yaniv and Milyavsy [7]. Herein we assume that decisions are tied to rewards and the delay time separating a decision from its resulting reward plays a significant role in the decision-making process. This dependence of a decision on the delay time is only part of the story, however. It is not unreasonable to expect that the time between decision and reward is mitigated by the magnitude of the reward. In a rational world increasing the delay time or decreasing the size of the reward, would lead one to expect a monotonic decrease in the experienced size of a reward with the time delay. Consequently, the longer one must wait for a reward the smaller is its experienced value at the time of the decision. A functional form for the value experienced (utility function) by the individual making the decision was given by Samuelson in 1937 to be an exponential function of the delay time [8]. But the world does not always conform to our notions of consistency. The independent variable of interest in the intertemporal decision-making process, the way it is presently formulated, is the length of time between when an individual makes a choice and when the reward for that choice is realized. As mentioned, a form for a discounted utility function in economic theory [8], is the exponential pj =
hU (D)i = e−kD
(5)
where the measured delay between the time of decision until the reward is delivered is denoted by D. The zero-delay value of the decision is normalized to unity. The monotonic behavior of the exponential function with time is consistent with the notion of rationality in which the longer the delay time, the greater the discounting of the reward. The spectrum of individual responses to a specific delay time is characterized by the phenomenological impulsivity rate parameter k, so that the utility is discounted at the same rate k in each interval of time 1
d hU (D)i
. (6) dD Cajueiro [9] gave mathematical form to the observation that behavioral economic studies do not follow the economist’s assumed consistency in intertemporal choice, but rather there is an inconsistency in human intertemporal choice and an implied irrationality. Consequently, the consistent intertemporal choice predicted by expected utility theory is not what is observed experimentally. The traditional method of explaining this preference reversal over time is by modeling the expected utility function, that is, the expected value an individual perceives obtaining with the choice they make, with a hyperbolic function rather than an exponential function of delay: k = discount rate = −
hU (D)i =
1 1 + kD
.
hU (D)i
(7)
3582
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
The simple hyperbolic model (7) fits data significantly better than does the exponential (5), but it too does not capture the rich behavior of decision making, suggesting that additional extensions of the theory are necessary. In Section 2 we use arguments from psychophysics to discuss how the objective delay of reward can be related to subjective uncertainty. We introduce a probability density for waiting a given time before realizing the reward generated by a specific decision and associate this with the subjective estimate of that time. This discussion of the intertemporal choice utility function yields a generalized hyperbolic distribution based on renewal theory. In Section 3 we compare the theoretical form of the newly derived utility function with that obtained from Takahashi’s experiments [10,11] and obtain surprisingly good correspondence. In Section 4 the present model is compared with a number of other approaches. In Section 5 we draw some conclusions regarding the mechanism operating in the decision-making process of the individual, including some mechanisms operating in the human brain. 2. Stochastic utility function Psychophysics was developed by Fechner [12] in the nineteenth century to associate objective physical measures with experiential phenomena in psychology. Historically such activity was directed toward sensitivity to changes in light, heat, electric shocks and other such physical stimulations. Recently however, with the advent of sophisticated instruments to measure brain activity, such as functional magnetic resonance imaging (fMRI) more subtle experiences have been addressed such as the perception of time and the pursuit of happiness. The failure to act on a prediction that will insure success is related to the notion of rationality; Wittmann and Paulus [4] point out that the time between a decision and a beneficial outcome can be viewed as a cost to be weighed against the benefits of the outcome. They focus on the notion of impulsivity, whereas here we are concerned with the mathematical formulation of the relation between the perception of time and the time as measured by a clock and how that difference between the two influences the decision-making process. A related but distinct chain of argument was developed by Takahashi [10] to obtain and use the q-exponential discount model of Cajueiro [9]. 2.1. Physical and mental time The notion of delay enters naturally into the description of complex stochastic phenomena, describing the length of time a given configuration of a phenomenon persists, or in the present case, the time experienced between when a decision is made and when the decision is rewarded. This perspective requires the subjective delay time to be a stochastic variable rather than a deterministic parameter. The probability that the subjective delay time τ is in the interval (τ , τ + dτ ) is given by ψ(τ )dτ where ψ(τ ) is the subjective delay time probability density. The probability that no reward has been received in the subjective delay time interval (0, τ ) is given by
Ψ (τ ) =
∞
Z τ
ψ(τ 0 )dτ 0 .
(8)
This function is known as the survival probability in the statistics literature. The probability of the delay, as appropriate for the subjective utility function, is here constructed using Fechner’s Law. This law relates the subjectively experienced magnitude of a stimulus to the objectively measured magnitude of that stimulus. Herein we relate the physical time D, as measured by a clock, to the subjective time τ , experienced by the individual, in the form of Fechner’s Law [13]
τ = Ts ln [T + D] + C .
(9)
Here the constant C is determined by the zero of the perceived time to yield
D τ = Ts ln 1 +
(10)
T
where T is the time scale characteristic of the measurement and Ts is characteristic of the mental estimates of the time intervals. Roberts [13] states in a footnote that a crucial assumption made by Fechner in constructing the logarithmic form of his law was that sensation can scale based on variability or confusion or error; if two pairs of stimuli are equally often confused, then psychologically they are equally far apart. He also discusses the fact that Fechner’s Law is based on a probabilistic form of the utility function, which he refers to as the Fechnerian utility model. We assume that in our perception of time the discount rate is constant from one interval to the next. This assumption was made previously [14] to explain the 1/f -noise observed in human cognition [15]. Consequently, the distribution density of the time intervals between events estimated by the brain is assumed to be exponential,
ψ(τ ) =
1 Tm
exp −
τ Tm
.
(11)
This subjective time distribution density function can be related to the unknown distribution density of measurement times φ(D) through the equality of the probabilities that the subjective delay time τ is in the interval (τ , τ + dτ ) and the objective time D is in the interval (D, D + dD)
ψ(τ )dτ = φ(D)dD.
(12)
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
3583
Inserting (10) into (11) we obtain from (12) after a little algebra the Pareto distribution
φ(D) = ψ(τ )
dτ dD
= (µ − 1)
T µ−1
(13)
(T + D)µ
with the Pareto index expressed in terms of the subjective time parameters
µ=1+
Ts Tm
.
(14)
As Takahashi et al. [16] point out the Weber–Fechner type of time perception is mathematically equivalent to the q-exponent discount model based on Tsallis’ non-extensive entropy. Consequently, using the integral for the survival probability (8) we obtain
Φ (D) =
T µ−1
(15)
(T + D)µ−1
and therefore the expected utility function for the intertemporal choice model is a generalized hyperbolic distribution
hU (D)i = Φ (D).
(16)
The subjective utility function is the probability that the reward for a decision has not been received up to the delay time D. Fechner’s Law has allowed us to transform the assumed rational (dynamically consistent) subjective time to the irrational (dynamically inconsistent) time given by the generalized hyperbolic (Pareto) distribution. 2.2. Renewal theory It is useful to relate the Pareto distribution to renewal theory by introducing the age-specific failure rate g (t ) expressed by g (t ) =
φ(t ) Φ (t )
(17)
so that the survival probability is written D
Z Φ (D) = exp −
g (t )dt 0
0
.
(18)
0
The Pareto form of the survival probability is determined by the time for the nth event occurring in the interval tn+1 > t > tn generated by [14] g (t ) =
r0 1 + r1 (t − tn )
.
(19)
Note that every time there is an event we reset the time origin to the time of occurrence of the last event tn before the observation time t. Using the renewal nature of the process we define the clock times D through the prescription D(n) = tn+1 − tn and consequently substituting (19) into (18) obtain the desired form (15) for the survival probability with the parameters appropriately chosen. Consequently, as pointed out elsewhere [14] the Weber–Fechner law used in the last subsection generates a renewal process. A general derivation of the discount rate resulting in (19) among others was subsequently given by Yukalov and Sornette [17]. The average delay time in the intertemporal choice model, based on the Pareto distribution, is given by ∞
Z
D0 φ D dD0 =
0
hDi = 0
T
µ−2 ∞
for µ > 2 for µ ≤ 2
(20)
from which we see that the mean delay time diverges for µ ≤ 2, but the distribution is still normalizable for µ > 1. Thus, the intertemporal delay process is non-ergodic if the Pareto index is in the interval 1 < µ < 2 [18]. A non-ergodic process is one for which averages calculated with the ensemble distribution function are not equal to those calculated with the time series. 3. Experimental results The generalized hyperbolic form of the utility function was postulated by Takahashi [10] and the parameters in the probability densities are determined from the two distinct experiments conducted by him. In these experiments individuals first made decisions regarding their own utility and subsequently they made decisions regarding what they estimated to be the utility of the decisions of others. From this experiment Takahashi deduced two sets of parameter values, which were
3584
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
Fig. 1. The logarithm of the utility function is plotted as a function of the logarithm of the delay time. The dots are the experimental points for an individual choosing their own rewards with delay and the solid line segment is the fit of a Pareto distribution to the data using (15). The numerical values of the parameters determined by the least-square fit to the data are recorded in Table 1. Table 1 Empirical values of the parameters in the two representations of the subjective value function as obtained by least-square fitting the data to (15) shown in Figs. 1 and 2.
Decision for self Decision for other
T
µ
k
31.01 1.85
1.28 1.11
0.009 0.058
found to be quite different. We independently fit these data to the stochastic utility model developed in the previous section. The experiment described by Takahashi [11] was conducted using 20 graduate students at the University of Tokyo and took approximately one hour per student to complete. The students were asked to make decisions about hypothetical money to be received either immediately or at a later time and concerning (a) the subjects themselves and (b) other people not known to the decision-making subjects. The base value for the decisions was 1000 yen and there were seven delays ranging from 1 week to 25 years. Each of the students participated in the two types of intertemporal choice tasks, (a) and (b) above. Moreover it probably needs to be stressed that there exists a large number of ways to generate the generalized hyperbolic distribution [18]. What is of interest presently is the model presented in the previous section and its subsequent interpretation using Takahashi’s experimental data. The method previously used to arrive at the Pareto form of the subjective value function given by (15) in the present context did not consider the stochastic dynamics of the process, but was made by ansatz [10]. We point out that the values of the parameters fit using least-square error methods are in essential agreement with those obtained by Takahashi [11] and there is no qualitative difference in the interpretation of the parameter values. We received the group median values of the utility function for both experiment types (a) and (b) from Takahashi by personal communication and used these values to calculate the parameter T and the inverse power-law index µ by leastsquare fitting the Pareto functional form to the experimental data points. These values are plotted in Fig. 1 for the self-delay and in Fig. 2 for the delay of others. The log–log form of the graph in Fig. 1 clearly indicates the asymptotic inverse power-law form of the subjective utility function (15). From Table 1 the empirical power-law index is µ = 1.28 and the characteristic time scale is T = 31 days that determines the curvature of the probability density in Fig. 1. It should also be pointed out that the parameter T varies inversely with the impulsivity parameter ([µ − 1] /k) and therefore the more impulsive the individual the shorter the characteristic time relating the subjective and objective times in Eq. (10). It is evident that the asymptotic form of the Pareto distribution is the inverse power-law
Φ (D) ≈ (µ − 1)
µ−1 T
D
.
(21)
This variability in the decision-making process has an associated spectrum that can be determined using a Tauberian theorem to be [19] S (f ) ∝
1 f 3−µ
(22)
so that since µ < 2 the utility function manifests 1/f -noise behavior with spectral index given by 3 − µ = 1.72. Such 1/f -noise with an index far from µ = 2 have been found in complex psychophysical phenomena spanning the range of scales from the macroscopic behavioral level down to the microscopic level. It has been observed that 1/f -noise appears in body movements such as walking, postural sway, and movement in synchronization with external stimulation such as a metronome; also in physiologic networks as manifest in heart rate variability, human vision, the dynamics of the human brain and in human cognition; finally 1/f -noise is considered by many to be the signature of complexity, see, for example, West et al. [18] for a discussion of the relation between 1/f -noise and general complex networks including references to the above mentioned phenomena.
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
3585
Fig. 2. The logarithm of the utility function is plotted as a function of the logarithm of the delay time. The dots are the experimental points for an individual choosing the rewards for an unknown other with delay and the solid line segment is the fit of the Pareto distribution to the data using (15). The numerical values of the parameters determined by the least-square fit to the data are recorded in Table 1.
Fig. 2 depicts the estimate of the subjective utility function individuals make for unknown others. The Pareto distribution is also obtained in this case but the parameters characterizing the distribution are quite different from those determined by the data in Fig. 1. The Pareto index for the data in Fig. 2 is µ = 1.11 and the characteristic time scale is T = 1.85 days indicating a much shorter characteristic time than the subjects determined for themselves and a much steeper slope in the spectrum. 4. On the dynamic origin of discount evaluation We point out that the need of going beyond the hyperbolic picture with a fitting parameter µ < 2 seems to be well assessed [10,11]. Its origin, however, is still the subject of discussion and further investigation. Takahashi used Tsallis statistics, thereby implying that non-extensive thermodynamics is the source of this hyperbolic property. It is important to point out that Takahashi adopted Tsallis statistics after the publication of an earlier work interpreting this effect by means of the Weber–Fechner law (compare his Eqs. (3)–(10) of this paper). From the point of view of the fitting function procedure these different theoretical approaches are equivalent, since they all generate the same analytical function (16). More recently, the same fitting function has been re-derived by Yukalov and Sornette [17] using arguments based on the quantum-mechanical procedures for defining probability amplitudes to take interference uncertainty into account (see their Eq. (74)). All these different approaches to the same fitting function leaves open the problem of whether or not the process is renewal; the basis of the model presented herein. In an ideal experiment where the same subjects are requested to produce a series of discounting times D1 , D2 , . . . Di , . . . , in sequence, would these times be correlated to one another or not? The use of different theories would lead to different conclusions. For instance, if we accept superstatistics as the dynamical foundation of non-extensive thermodynamics [20], we would reach the conclusion that the process is not renewal [21]. If, on the other hand, we assume that the drawing of a discounting time D has an intimate connection with an idealized Manneville map [22], the resulting time series would be renewal. Consequently, the sequence would be characterized by the important property of renewal aging [23] with the interesting effect of turning the power-law index µ in the survival probability into µ − 1. As Allegrini et al. [24] show the survival probability for an aging time ta has the asymptotic forms
Φ (ta = 0, D) =
µ−1
T
(23)
T +D
and
Φ (ta = ∞, D) =
T T +D
µ−2
.
(24)
Here ta denotes the time interval between ‘‘preparation’’ (the first event) and ‘‘observation’’ (D = 0), and the second expression denotes the experimental observation at very long times. On the basis of the discount evaluation experiment alone it is not possible to establish whether the renewal or the superstatistics condition applies. There are, however, other experiments involving brain dynamics that lead us to make the cautious conjecture that the renewal condition does in fact apply. Allegrini et al. [24] have recently analyzed the electroencephalogram of 30 subjects under the closed-eyes condition with a technique that allows them to establish the global properties of the brain. This approach led them to conclude that the brain dynamics are renewal with µ = 2.13. If we conjecture that the discount evaluation experiment activates the aging process [23], we find that this power index of the earlier experiment is turned into µ = 1.13, which is very close to the decision-making values recorded in Table 1 for the interevent choice experiments. On the other hand, Grigolini et al. [14] have recently argued that there may exist a very important connection between the phenomenological Weber–Fechner law and the critical dynamics of a network of interacting units [25], which generates
3586
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
a renewal sequence of waiting times. The model of interacting units [25] has more recently been applied to the physics of blinking quantum dots [26] with the interesting conclusion that this model accounts for the generation of 1/f -noise revealed by the experiments on these networks. It is important to observe that the 1/f -noise spectrum experimentally detected cannot be explained by using stationary correlation functions due to the non-ergodic and non-equilibrium condition corresponding to µ ≤ 3 [19], even if the non-ergodic effects are less significant with µ > 2. All this is confirmed by the recent experiments on liquid crystals [27] where again the same perspective of many interacting units applies. On the basis of these remarks we reach the tentative conclusion that the discount evaluation experiment may be interpreted as a technique to reveal the brain dynamics. Furthermore, the power-law indices recorded in Table 1 may be connected to those of Allegrini et al. [24] through the aging effect, an important consequence of the renewal condition. 5. Implications for decision making In the present psychophysical model the index µ provides a measure of dynamic inconsistency, which is to say how soon individuals change their minds with the passage of time, but without new information. The change in the discount rate over time measures irrationality in terms of the deviation of the generalized hyperbolic (Pareto) from the exponential utility function. The empirical estimates using Takahashi’s data show that irrationality in intertemporal choice is stronger when the outcome is irrelevant to the decision maker, with smaller values of the index µ corresponding to greater irrationality. It is well known that such generalized hyperbolic delay time distributions have diverging mean times for µ < 2, which is the situation for decision making. According to Frederick et al. [28]: Intertemporal choices–decisions involving tradeoffs among costs and benefits occurring at different times—are important and ubiquitous. Such decisions not only affect one’s health, wealth, and happiness, but, may also, as Adam Smith first recognized, determine the economic prosperity of nations. As we mentioned earlier, the exponential form of the discounted utility model proposed by Samuelson rests on a time-independent discount rate. Moreover, even if ordinary people are internally rational, as assumed here, the measured decisions they make in the external world are characterized by the generalized hyperbolic utility function (15) and often appear to the world as irrational Refs. [29,30]. Wittmann et al. [31] seem to have been the first to explicitly capture the processing of time as a dependent variable in fMRI studies during delay discounting experiments. Their experimental observations suggest that different portions of the brain are characterized by different inverse power-law indices. However, we have to keep in mind that these experiments describe the brain’s response to a stimulus. It may be true that the brain at rest has a different distribution of µ-values. The intertemporal choice experiments using fMRI to show brain activity when subjects choose between smaller earlier rewards and later larger rewards lead to the conclusion: . . . that limbic and paralimbic cortical areas are involved when having to choose between immediate and delayed rewards, whereas areas relating to executive control seem to be more important for delaying gratification. It should also be pointed out that the neurobiology underlying delay discounting is not well understood, even though it is possible to relate the generalized hyperbolic utility function obtained in experiments that simultaneously measure brain activity, as done by Wittmann et al. [31]. In particular, they showed experimentally that the delay of rewards activate certain parts of the brain using fMRI. One of the results determined in this latter set of experiments was that the subjects’ preference judgments resulted in inverse power-law slopes that have one value for delay time less than one year and a substantially larger value for delay times greater than one year. Specifically, µ = 1.17 for delay of reward of one year or less and µ = 1.50 for delay of reward of one year or more. We cannot rule out the possibility that the discrepancy between these two values and those recorded in Table 1 may be due to the arbitrary nature of the fitting procedure, but the overall closeness of the values to those in Table 1 is encouraging and is consistent with the renewal theory interpretation of brain activity [14,18,19]. We have emphasized that the concept of time is subjective and consequently a strongly impulsive patient may have a perception of time very different from a control individual. It is interesting to observe that according to Wittmann et al. [31] stimulant-dependent individuals estimate that a given time interval is larger than the corresponding estimates made by control subjects. Consequently, postponing gratification has a telescoping effects on their discomfort. Moreover the impulsivity, as measured by the response time T in the stochastic intertemporal choice model, is an order of magnitude shorter when the outcome is irrelevant to the decision maker. The interpretation of this latter observation is intriguing, since it appears that a given individual estimates others to be less rational and more impulsive than themselves, leading to a number of interesting speculations. Lawmakers, for example, may unconsciously (or even consciously) assume they know what is best for their constituency given their experience and position. Thus, it is also important to emphasize that the response to these stimuli change according to whether the brain is that of a healthy person or is that of a stimulantdependent subject. In a military context this finding, if it is borne out by subsequent experiments, is even more significant. The perceived rationality of others influences such emotional states as trust. A leader may be reluctant to trust his subordinates, and therefore not be willing to delegate authority if their rationality is suspect. It remains for future studies to examine how the irrationality in the choices people make for others can be mitigated.
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
3587
Acknowledgements P.G. thanks ARO (grant W911NF-08-1-0117) and Welch (grant B-1577) for financial support of this research work. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31]
D. Bernoulli, Econometrica 22 (1954). Translated from Latin to English by L. Sommer from Comm. Acad. Sca. Imp. Pet. V, 175 (1738). C.K. Hess, R. Hastie, Decision and experience: why don’t we choose what makes us happy? TRENDS in Cog. Sci. 10 (2006) 31. D. Ariely, Predictably Irrational: The Hidden Forces That Shape Our Decisions, Harper Collins, New York, 2008. M. Wittmann, M.P. Paulus, Decision making, impulsivity and time perception, TRENDS in Cog. Sci. 12 (2007) 7. G. Coricelli, R.J. Dolan, A. Sirigu, Brain, emotion and decision making the paradigmatic example of regret, TRENDS in Cog. Sci. 11 (2007) 258. E. van Dijk, M. Zeelenberg, When curiosity killed regret: avoiding or seeking the unknown in decision-making under uncertainty, J. Exp. Soc. Psyh. 43 (2007) 656. I. Yaniv, M. Milyavsy, Using advice from multiple sources to revise and improve judgments, Org. Beh. Hum. Dec. Proc. 103 (2007) 104. P.A. Samuelson, A note on measurement of utility, The Rev. Econom. Stud. 4 (1937) 155. D.O. Cajueiro, A note on the relevance of the q-exponential function in the context of intertemporal choices, Physica A 364 (2006) 385–388. T. Takahashi, A comparison of intertemporal choices of oneself versus some else based on Tsallis’ statistics, Physica A 385 (2007) 637–644. T. Takahashi, A probabilistic model based on Tsallis’ statistics, Physica A 386 (2007) 335–338. G.T. Fechner, Elemente der Psychophysik, Breitkopf und Hartel, Leipzig, 1860. F.S. Roberts, Measurement Theory, Encyclopedia of Mathematics and Its Applications, Addison-Wesley, Reading, MA, 1979. P. Grigolini, G. Aquino, M. Bologna, M. Lukovic, B.J. West, A theory of 1/f noise in human cognition, Physica A 388 (2009) 4192. D.L. Gilden, T. Thornton, M.W. Mallon, 1/f noise in human cognition, Science 267 (1995) 1837–1839. T. Takahashi, H. Oono, M.H.B. Radford, Psychophysics of time perception and intertemporal choice models, Physica A 387 (2008) 2066. V.I. Yukalov, D. Sornette, Physics or risk and uncertainty in quantum decision making, Eur. Phys. J. B 71 (2009) 533. B.J. West, E. Geneston, P. Grigolini, Maximizing information exchange between complex networks, Phys. Rep. 468 (2008) 1. M. Lukovic, P. Grigolini, Power spectra for both interrupted and perennial aging processes, J. Chem. Phys. 129 (2008) 184102. C. Beck, Dynamical foundatins of nonextensive statistical mechanics, Phys. Rev. Lett. 87 (2001) 180601. P. Grigolini, in: C. Beck, G. Benedek, A. Rapisardand, T. Tsallis (Eds.), Complexity, Metastability and Nonextensivity, World Scientific, Singapore, 2005, p. 72. P. Allegrini, V. Benci, P. Grigolini, P. Hamilton, M. Ignaccolo, G. Menconi, L. Palatila, G. Raffaelli, N. Scafetta, M. Virgilio, J. Yang, Compression and diffusion: a joint approach to detect complexity, Chaos Solitons Fractals 15 (2003) 517. G. Aquino, M. Bologna, P. Grigolini, B.J. West, Phys. Rev. E 70 (2004) 036105. P. Allegrini, D. Menicucci, R. Bedni, L. Fronzoni, A. Gemiginani, P. Grigolini, B.J. West, P. Paradsi, Spontaneous brain activity as a source of ideal 1/f noise, Phys. Rev. E 80 (2009) 061914. S. Bianco, E. Geneston, P. Grigolini, M. Ignaccola, Renewal aging as emerging property of phase synchronization, Physica A 387 (2008) 1387. P.A. Frantsuzov, S. Volkan-Kacso, B. Janko, Phys. Rev. Lett. 103 (2009) 207402. L. Silvestri, L. Fronzoni, P. Grigolini, P. Allegrini, Event-driven power-law relaxation in weak turbulence, Phys. Rev. Lett. 102 (2009) 104502. S. Frederick, G. Loewenstein, T. O’Donoghue, Time discounting and time preference: a critical review, J. Econ. Lit. 40 (2002) 351. R. Thaler, Some empirical evidence on dynamic inconsistency, Economics Lett. 8 (1981) 201. A.G. Samfey, G. Loewenstein, S.M. McClure, J.D. Cohen, Neuroeconomics; cross currents in research on decision making, TRENDS in Cog. Sci. 10 (2006) 108. M. Wittmann, D.S. Leland, M.P. Paulus, Time and decision making: differential contribution of the posterior insular cortex and the stiatum during a delay discounting task, Exp. Brain Res. 179 (2007) 643.
Contents lists available at ScienceDirect
Physica A journal homepage: www.elsevier.com/locate/physa
A psychophysical model of decision making B.J. West a,∗ , P. Grigolini b a
Information Science Directorate, US Army Research Office, Durham, NC 27709, United States
b
Center for Nonlinear Science, University of North Texas, United States
article
info
Article history: Received 26 January 2010 Received in revised form 12 March 2010 Available online 21 April 2010 Keywords: Decision making Probability Inverse power law 1/f noise
abstract Herein we develop a psychophysical model of decision making based on the difference between objective clock time and the human brain’s perception of time. In this model the utility function is given by the survival probability, which is shown to be a generalized hyperbolic distribution. The parameters of the utility function are fit to intertemporal choice model experimental data and decision making is determined to be a 1/f -noise process. Published by Elsevier B.V.
1. Introduction The history of probability is partly a history of how we make choices with incomplete knowledge and under conditions of uncertainty. Consequently, it is not surprising that the mathematical formalism of probability theory was initiated in that most mysterious of human activities: gambling. Choice enters gambling in the determination and selection of the most likely outcomes in games of chance. The implicit assumption in a wager is that the antagonists want to win and therefore in these initial efforts at formulating a general theory a great deal of attention was paid to developing measures of play that could be used to enhance the likelihood of winning or at least decrease the likelihood of losing. One measure of play is, of course, the expected outcome of a wager. If the possible outcomes of play are indexed j = 1 to M, then the probability of the jth outcome is pj , wj is the wager for this outcome and the expected winnings are
hW i =
M X
pj wj .
(1)
j =1
From this perspective the expected winnings provided a guide for the bettor and was historically indicative of the outcome of a single play since the probability had a subjective interpretation. Even with this non-binding estimate of how well a bettor may do, it was quite a surprise when in 1713 N. Bernoulli introduced what appeared to be a reasonable game that resulted in a diverging expectation value hW i = ∞. In general this divergence of the average occurs when the relative sizes of the terms in series (1) satisfy the condition p j +1 w j +1 pj wj
≥ 1 as M → ∞.
(2)
The particular case pj = 1/2j with the wager of wj = 2j ducats is called the St. Petersburg Paradox, named after the journal in which the problem was eventually published. The paradox arose because the bettor and the house could not agree on an
∗
Corresponding author. E-mail addresses: [email protected], [email protected] (B.J. West).
0378-4371/$ – see front matter. Published by Elsevier B.V. doi:10.1016/j.physa.2010.03.039
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
3581
ante. The house wanted a large ante because the expected winnings are infinite and the bettor wanted a small ante because half the time he would only win 2 ducats. In 1731 D. Bernoulli gave one resolution of the paradox by introducing the concept of a utility function and published the result in 1738 [1]. He argued that not everyone should wager in the same way since the subjective value or utility associated with a unit of currency depends on one’s total wealth. Consequently, he proposed estimating the ‘winnings’ by using the average utility of a wager
hU i =
M X
pj U wj
(3)
j=1
and not the wager itself. It is the average utility that one ought to maximize and not the expected winnings. The mathematical wrangling over the resolution of the St. Petersburg Paradox lasted another two centuries, but that discussion does not concern us here except to note that much of the argument was focused on the difference between subjective and objective probability. The final resolution of what constitutes a probability in the physical sciences is that we interpret the probability objectively as a relative frequency; the number of times the play lands on the jth outcome Nj relative to the total large number of plays N Nj
. (4) N However the concern over the interpretation of probability has today been again taken up by the cognitive sciences, which distinguish between the probability used in the physical sciences from that used in the determination of decision-making activities. Hess and Hastie [2] point out that behavioral-decision research indicates that people fail to choose what makes them happy, either because they fail to predict what will make them happy or they fail to act on their predictions. They go on to examine five biases that distort the ability to make accurate predictions and four mechanism that inhibit acting on the predictions made. Ariely [3] takes the position that undesirable outcomes can be predicted and avoided, as he argues in his interesting book, which explains the fundamentally irrational nature of many decisions. Wittmann and Paulus [4] point out that psychology does not dismiss rationality, but emphasize that other factors such as satisfaction and impulsivity become of equal or more importance in the psychology of decision making. One of the most studied emotions that mitigate rational decision making is regret [5] with regret aversion often dictate the decision-making process [6]. However, it was also shown that curiosity, which can be associated with impulsive behavior, can suppress or kill regret [6]. On a broader stage sociology examines how peer pressure and leadership often dominate the discussion for decision making in social networks, as discussed by Yaniv and Milyavsy [7]. Herein we assume that decisions are tied to rewards and the delay time separating a decision from its resulting reward plays a significant role in the decision-making process. This dependence of a decision on the delay time is only part of the story, however. It is not unreasonable to expect that the time between decision and reward is mitigated by the magnitude of the reward. In a rational world increasing the delay time or decreasing the size of the reward, would lead one to expect a monotonic decrease in the experienced size of a reward with the time delay. Consequently, the longer one must wait for a reward the smaller is its experienced value at the time of the decision. A functional form for the value experienced (utility function) by the individual making the decision was given by Samuelson in 1937 to be an exponential function of the delay time [8]. But the world does not always conform to our notions of consistency. The independent variable of interest in the intertemporal decision-making process, the way it is presently formulated, is the length of time between when an individual makes a choice and when the reward for that choice is realized. As mentioned, a form for a discounted utility function in economic theory [8], is the exponential pj =
hU (D)i = e−kD
(5)
where the measured delay between the time of decision until the reward is delivered is denoted by D. The zero-delay value of the decision is normalized to unity. The monotonic behavior of the exponential function with time is consistent with the notion of rationality in which the longer the delay time, the greater the discounting of the reward. The spectrum of individual responses to a specific delay time is characterized by the phenomenological impulsivity rate parameter k, so that the utility is discounted at the same rate k in each interval of time 1
d hU (D)i
. (6) dD Cajueiro [9] gave mathematical form to the observation that behavioral economic studies do not follow the economist’s assumed consistency in intertemporal choice, but rather there is an inconsistency in human intertemporal choice and an implied irrationality. Consequently, the consistent intertemporal choice predicted by expected utility theory is not what is observed experimentally. The traditional method of explaining this preference reversal over time is by modeling the expected utility function, that is, the expected value an individual perceives obtaining with the choice they make, with a hyperbolic function rather than an exponential function of delay: k = discount rate = −
hU (D)i =
1 1 + kD
.
hU (D)i
(7)
3582
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
The simple hyperbolic model (7) fits data significantly better than does the exponential (5), but it too does not capture the rich behavior of decision making, suggesting that additional extensions of the theory are necessary. In Section 2 we use arguments from psychophysics to discuss how the objective delay of reward can be related to subjective uncertainty. We introduce a probability density for waiting a given time before realizing the reward generated by a specific decision and associate this with the subjective estimate of that time. This discussion of the intertemporal choice utility function yields a generalized hyperbolic distribution based on renewal theory. In Section 3 we compare the theoretical form of the newly derived utility function with that obtained from Takahashi’s experiments [10,11] and obtain surprisingly good correspondence. In Section 4 the present model is compared with a number of other approaches. In Section 5 we draw some conclusions regarding the mechanism operating in the decision-making process of the individual, including some mechanisms operating in the human brain. 2. Stochastic utility function Psychophysics was developed by Fechner [12] in the nineteenth century to associate objective physical measures with experiential phenomena in psychology. Historically such activity was directed toward sensitivity to changes in light, heat, electric shocks and other such physical stimulations. Recently however, with the advent of sophisticated instruments to measure brain activity, such as functional magnetic resonance imaging (fMRI) more subtle experiences have been addressed such as the perception of time and the pursuit of happiness. The failure to act on a prediction that will insure success is related to the notion of rationality; Wittmann and Paulus [4] point out that the time between a decision and a beneficial outcome can be viewed as a cost to be weighed against the benefits of the outcome. They focus on the notion of impulsivity, whereas here we are concerned with the mathematical formulation of the relation between the perception of time and the time as measured by a clock and how that difference between the two influences the decision-making process. A related but distinct chain of argument was developed by Takahashi [10] to obtain and use the q-exponential discount model of Cajueiro [9]. 2.1. Physical and mental time The notion of delay enters naturally into the description of complex stochastic phenomena, describing the length of time a given configuration of a phenomenon persists, or in the present case, the time experienced between when a decision is made and when the decision is rewarded. This perspective requires the subjective delay time to be a stochastic variable rather than a deterministic parameter. The probability that the subjective delay time τ is in the interval (τ , τ + dτ ) is given by ψ(τ )dτ where ψ(τ ) is the subjective delay time probability density. The probability that no reward has been received in the subjective delay time interval (0, τ ) is given by
Ψ (τ ) =
∞
Z τ
ψ(τ 0 )dτ 0 .
(8)
This function is known as the survival probability in the statistics literature. The probability of the delay, as appropriate for the subjective utility function, is here constructed using Fechner’s Law. This law relates the subjectively experienced magnitude of a stimulus to the objectively measured magnitude of that stimulus. Herein we relate the physical time D, as measured by a clock, to the subjective time τ , experienced by the individual, in the form of Fechner’s Law [13]
τ = Ts ln [T + D] + C .
(9)
Here the constant C is determined by the zero of the perceived time to yield
D τ = Ts ln 1 +
(10)
T
where T is the time scale characteristic of the measurement and Ts is characteristic of the mental estimates of the time intervals. Roberts [13] states in a footnote that a crucial assumption made by Fechner in constructing the logarithmic form of his law was that sensation can scale based on variability or confusion or error; if two pairs of stimuli are equally often confused, then psychologically they are equally far apart. He also discusses the fact that Fechner’s Law is based on a probabilistic form of the utility function, which he refers to as the Fechnerian utility model. We assume that in our perception of time the discount rate is constant from one interval to the next. This assumption was made previously [14] to explain the 1/f -noise observed in human cognition [15]. Consequently, the distribution density of the time intervals between events estimated by the brain is assumed to be exponential,
ψ(τ ) =
1 Tm
exp −
τ Tm
.
(11)
This subjective time distribution density function can be related to the unknown distribution density of measurement times φ(D) through the equality of the probabilities that the subjective delay time τ is in the interval (τ , τ + dτ ) and the objective time D is in the interval (D, D + dD)
ψ(τ )dτ = φ(D)dD.
(12)
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
3583
Inserting (10) into (11) we obtain from (12) after a little algebra the Pareto distribution
φ(D) = ψ(τ )
dτ dD
= (µ − 1)
T µ−1
(13)
(T + D)µ
with the Pareto index expressed in terms of the subjective time parameters
µ=1+
Ts Tm
.
(14)
As Takahashi et al. [16] point out the Weber–Fechner type of time perception is mathematically equivalent to the q-exponent discount model based on Tsallis’ non-extensive entropy. Consequently, using the integral for the survival probability (8) we obtain
Φ (D) =
T µ−1
(15)
(T + D)µ−1
and therefore the expected utility function for the intertemporal choice model is a generalized hyperbolic distribution
hU (D)i = Φ (D).
(16)
The subjective utility function is the probability that the reward for a decision has not been received up to the delay time D. Fechner’s Law has allowed us to transform the assumed rational (dynamically consistent) subjective time to the irrational (dynamically inconsistent) time given by the generalized hyperbolic (Pareto) distribution. 2.2. Renewal theory It is useful to relate the Pareto distribution to renewal theory by introducing the age-specific failure rate g (t ) expressed by g (t ) =
φ(t ) Φ (t )
(17)
so that the survival probability is written D
Z Φ (D) = exp −
g (t )dt 0
0
.
(18)
0
The Pareto form of the survival probability is determined by the time for the nth event occurring in the interval tn+1 > t > tn generated by [14] g (t ) =
r0 1 + r1 (t − tn )
.
(19)
Note that every time there is an event we reset the time origin to the time of occurrence of the last event tn before the observation time t. Using the renewal nature of the process we define the clock times D through the prescription D(n) = tn+1 − tn and consequently substituting (19) into (18) obtain the desired form (15) for the survival probability with the parameters appropriately chosen. Consequently, as pointed out elsewhere [14] the Weber–Fechner law used in the last subsection generates a renewal process. A general derivation of the discount rate resulting in (19) among others was subsequently given by Yukalov and Sornette [17]. The average delay time in the intertemporal choice model, based on the Pareto distribution, is given by ∞
Z
D0 φ D dD0 =
0
hDi = 0
T
µ−2 ∞
for µ > 2 for µ ≤ 2
(20)
from which we see that the mean delay time diverges for µ ≤ 2, but the distribution is still normalizable for µ > 1. Thus, the intertemporal delay process is non-ergodic if the Pareto index is in the interval 1 < µ < 2 [18]. A non-ergodic process is one for which averages calculated with the ensemble distribution function are not equal to those calculated with the time series. 3. Experimental results The generalized hyperbolic form of the utility function was postulated by Takahashi [10] and the parameters in the probability densities are determined from the two distinct experiments conducted by him. In these experiments individuals first made decisions regarding their own utility and subsequently they made decisions regarding what they estimated to be the utility of the decisions of others. From this experiment Takahashi deduced two sets of parameter values, which were
3584
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
Fig. 1. The logarithm of the utility function is plotted as a function of the logarithm of the delay time. The dots are the experimental points for an individual choosing their own rewards with delay and the solid line segment is the fit of a Pareto distribution to the data using (15). The numerical values of the parameters determined by the least-square fit to the data are recorded in Table 1. Table 1 Empirical values of the parameters in the two representations of the subjective value function as obtained by least-square fitting the data to (15) shown in Figs. 1 and 2.
Decision for self Decision for other
T
µ
k
31.01 1.85
1.28 1.11
0.009 0.058
found to be quite different. We independently fit these data to the stochastic utility model developed in the previous section. The experiment described by Takahashi [11] was conducted using 20 graduate students at the University of Tokyo and took approximately one hour per student to complete. The students were asked to make decisions about hypothetical money to be received either immediately or at a later time and concerning (a) the subjects themselves and (b) other people not known to the decision-making subjects. The base value for the decisions was 1000 yen and there were seven delays ranging from 1 week to 25 years. Each of the students participated in the two types of intertemporal choice tasks, (a) and (b) above. Moreover it probably needs to be stressed that there exists a large number of ways to generate the generalized hyperbolic distribution [18]. What is of interest presently is the model presented in the previous section and its subsequent interpretation using Takahashi’s experimental data. The method previously used to arrive at the Pareto form of the subjective value function given by (15) in the present context did not consider the stochastic dynamics of the process, but was made by ansatz [10]. We point out that the values of the parameters fit using least-square error methods are in essential agreement with those obtained by Takahashi [11] and there is no qualitative difference in the interpretation of the parameter values. We received the group median values of the utility function for both experiment types (a) and (b) from Takahashi by personal communication and used these values to calculate the parameter T and the inverse power-law index µ by leastsquare fitting the Pareto functional form to the experimental data points. These values are plotted in Fig. 1 for the self-delay and in Fig. 2 for the delay of others. The log–log form of the graph in Fig. 1 clearly indicates the asymptotic inverse power-law form of the subjective utility function (15). From Table 1 the empirical power-law index is µ = 1.28 and the characteristic time scale is T = 31 days that determines the curvature of the probability density in Fig. 1. It should also be pointed out that the parameter T varies inversely with the impulsivity parameter ([µ − 1] /k) and therefore the more impulsive the individual the shorter the characteristic time relating the subjective and objective times in Eq. (10). It is evident that the asymptotic form of the Pareto distribution is the inverse power-law
Φ (D) ≈ (µ − 1)
µ−1 T
D
.
(21)
This variability in the decision-making process has an associated spectrum that can be determined using a Tauberian theorem to be [19] S (f ) ∝
1 f 3−µ
(22)
so that since µ < 2 the utility function manifests 1/f -noise behavior with spectral index given by 3 − µ = 1.72. Such 1/f -noise with an index far from µ = 2 have been found in complex psychophysical phenomena spanning the range of scales from the macroscopic behavioral level down to the microscopic level. It has been observed that 1/f -noise appears in body movements such as walking, postural sway, and movement in synchronization with external stimulation such as a metronome; also in physiologic networks as manifest in heart rate variability, human vision, the dynamics of the human brain and in human cognition; finally 1/f -noise is considered by many to be the signature of complexity, see, for example, West et al. [18] for a discussion of the relation between 1/f -noise and general complex networks including references to the above mentioned phenomena.
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
3585
Fig. 2. The logarithm of the utility function is plotted as a function of the logarithm of the delay time. The dots are the experimental points for an individual choosing the rewards for an unknown other with delay and the solid line segment is the fit of the Pareto distribution to the data using (15). The numerical values of the parameters determined by the least-square fit to the data are recorded in Table 1.
Fig. 2 depicts the estimate of the subjective utility function individuals make for unknown others. The Pareto distribution is also obtained in this case but the parameters characterizing the distribution are quite different from those determined by the data in Fig. 1. The Pareto index for the data in Fig. 2 is µ = 1.11 and the characteristic time scale is T = 1.85 days indicating a much shorter characteristic time than the subjects determined for themselves and a much steeper slope in the spectrum. 4. On the dynamic origin of discount evaluation We point out that the need of going beyond the hyperbolic picture with a fitting parameter µ < 2 seems to be well assessed [10,11]. Its origin, however, is still the subject of discussion and further investigation. Takahashi used Tsallis statistics, thereby implying that non-extensive thermodynamics is the source of this hyperbolic property. It is important to point out that Takahashi adopted Tsallis statistics after the publication of an earlier work interpreting this effect by means of the Weber–Fechner law (compare his Eqs. (3)–(10) of this paper). From the point of view of the fitting function procedure these different theoretical approaches are equivalent, since they all generate the same analytical function (16). More recently, the same fitting function has been re-derived by Yukalov and Sornette [17] using arguments based on the quantum-mechanical procedures for defining probability amplitudes to take interference uncertainty into account (see their Eq. (74)). All these different approaches to the same fitting function leaves open the problem of whether or not the process is renewal; the basis of the model presented herein. In an ideal experiment where the same subjects are requested to produce a series of discounting times D1 , D2 , . . . Di , . . . , in sequence, would these times be correlated to one another or not? The use of different theories would lead to different conclusions. For instance, if we accept superstatistics as the dynamical foundation of non-extensive thermodynamics [20], we would reach the conclusion that the process is not renewal [21]. If, on the other hand, we assume that the drawing of a discounting time D has an intimate connection with an idealized Manneville map [22], the resulting time series would be renewal. Consequently, the sequence would be characterized by the important property of renewal aging [23] with the interesting effect of turning the power-law index µ in the survival probability into µ − 1. As Allegrini et al. [24] show the survival probability for an aging time ta has the asymptotic forms
Φ (ta = 0, D) =
µ−1
T
(23)
T +D
and
Φ (ta = ∞, D) =
T T +D
µ−2
.
(24)
Here ta denotes the time interval between ‘‘preparation’’ (the first event) and ‘‘observation’’ (D = 0), and the second expression denotes the experimental observation at very long times. On the basis of the discount evaluation experiment alone it is not possible to establish whether the renewal or the superstatistics condition applies. There are, however, other experiments involving brain dynamics that lead us to make the cautious conjecture that the renewal condition does in fact apply. Allegrini et al. [24] have recently analyzed the electroencephalogram of 30 subjects under the closed-eyes condition with a technique that allows them to establish the global properties of the brain. This approach led them to conclude that the brain dynamics are renewal with µ = 2.13. If we conjecture that the discount evaluation experiment activates the aging process [23], we find that this power index of the earlier experiment is turned into µ = 1.13, which is very close to the decision-making values recorded in Table 1 for the interevent choice experiments. On the other hand, Grigolini et al. [14] have recently argued that there may exist a very important connection between the phenomenological Weber–Fechner law and the critical dynamics of a network of interacting units [25], which generates
3586
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
a renewal sequence of waiting times. The model of interacting units [25] has more recently been applied to the physics of blinking quantum dots [26] with the interesting conclusion that this model accounts for the generation of 1/f -noise revealed by the experiments on these networks. It is important to observe that the 1/f -noise spectrum experimentally detected cannot be explained by using stationary correlation functions due to the non-ergodic and non-equilibrium condition corresponding to µ ≤ 3 [19], even if the non-ergodic effects are less significant with µ > 2. All this is confirmed by the recent experiments on liquid crystals [27] where again the same perspective of many interacting units applies. On the basis of these remarks we reach the tentative conclusion that the discount evaluation experiment may be interpreted as a technique to reveal the brain dynamics. Furthermore, the power-law indices recorded in Table 1 may be connected to those of Allegrini et al. [24] through the aging effect, an important consequence of the renewal condition. 5. Implications for decision making In the present psychophysical model the index µ provides a measure of dynamic inconsistency, which is to say how soon individuals change their minds with the passage of time, but without new information. The change in the discount rate over time measures irrationality in terms of the deviation of the generalized hyperbolic (Pareto) from the exponential utility function. The empirical estimates using Takahashi’s data show that irrationality in intertemporal choice is stronger when the outcome is irrelevant to the decision maker, with smaller values of the index µ corresponding to greater irrationality. It is well known that such generalized hyperbolic delay time distributions have diverging mean times for µ < 2, which is the situation for decision making. According to Frederick et al. [28]: Intertemporal choices–decisions involving tradeoffs among costs and benefits occurring at different times—are important and ubiquitous. Such decisions not only affect one’s health, wealth, and happiness, but, may also, as Adam Smith first recognized, determine the economic prosperity of nations. As we mentioned earlier, the exponential form of the discounted utility model proposed by Samuelson rests on a time-independent discount rate. Moreover, even if ordinary people are internally rational, as assumed here, the measured decisions they make in the external world are characterized by the generalized hyperbolic utility function (15) and often appear to the world as irrational Refs. [29,30]. Wittmann et al. [31] seem to have been the first to explicitly capture the processing of time as a dependent variable in fMRI studies during delay discounting experiments. Their experimental observations suggest that different portions of the brain are characterized by different inverse power-law indices. However, we have to keep in mind that these experiments describe the brain’s response to a stimulus. It may be true that the brain at rest has a different distribution of µ-values. The intertemporal choice experiments using fMRI to show brain activity when subjects choose between smaller earlier rewards and later larger rewards lead to the conclusion: . . . that limbic and paralimbic cortical areas are involved when having to choose between immediate and delayed rewards, whereas areas relating to executive control seem to be more important for delaying gratification. It should also be pointed out that the neurobiology underlying delay discounting is not well understood, even though it is possible to relate the generalized hyperbolic utility function obtained in experiments that simultaneously measure brain activity, as done by Wittmann et al. [31]. In particular, they showed experimentally that the delay of rewards activate certain parts of the brain using fMRI. One of the results determined in this latter set of experiments was that the subjects’ preference judgments resulted in inverse power-law slopes that have one value for delay time less than one year and a substantially larger value for delay times greater than one year. Specifically, µ = 1.17 for delay of reward of one year or less and µ = 1.50 for delay of reward of one year or more. We cannot rule out the possibility that the discrepancy between these two values and those recorded in Table 1 may be due to the arbitrary nature of the fitting procedure, but the overall closeness of the values to those in Table 1 is encouraging and is consistent with the renewal theory interpretation of brain activity [14,18,19]. We have emphasized that the concept of time is subjective and consequently a strongly impulsive patient may have a perception of time very different from a control individual. It is interesting to observe that according to Wittmann et al. [31] stimulant-dependent individuals estimate that a given time interval is larger than the corresponding estimates made by control subjects. Consequently, postponing gratification has a telescoping effects on their discomfort. Moreover the impulsivity, as measured by the response time T in the stochastic intertemporal choice model, is an order of magnitude shorter when the outcome is irrelevant to the decision maker. The interpretation of this latter observation is intriguing, since it appears that a given individual estimates others to be less rational and more impulsive than themselves, leading to a number of interesting speculations. Lawmakers, for example, may unconsciously (or even consciously) assume they know what is best for their constituency given their experience and position. Thus, it is also important to emphasize that the response to these stimuli change according to whether the brain is that of a healthy person or is that of a stimulantdependent subject. In a military context this finding, if it is borne out by subsequent experiments, is even more significant. The perceived rationality of others influences such emotional states as trust. A leader may be reluctant to trust his subordinates, and therefore not be willing to delegate authority if their rationality is suspect. It remains for future studies to examine how the irrationality in the choices people make for others can be mitigated.
B.J. West, P. Grigolini / Physica A 389 (2010) 3580–3587
3587
Acknowledgements P.G. thanks ARO (grant W911NF-08-1-0117) and Welch (grant B-1577) for financial support of this research work. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31]
D. Bernoulli, Econometrica 22 (1954). Translated from Latin to English by L. Sommer from Comm. Acad. Sca. Imp. Pet. V, 175 (1738). C.K. Hess, R. Hastie, Decision and experience: why don’t we choose what makes us happy? TRENDS in Cog. Sci. 10 (2006) 31. D. Ariely, Predictably Irrational: The Hidden Forces That Shape Our Decisions, Harper Collins, New York, 2008. M. Wittmann, M.P. Paulus, Decision making, impulsivity and time perception, TRENDS in Cog. Sci. 12 (2007) 7. G. Coricelli, R.J. Dolan, A. Sirigu, Brain, emotion and decision making the paradigmatic example of regret, TRENDS in Cog. Sci. 11 (2007) 258. E. van Dijk, M. Zeelenberg, When curiosity killed regret: avoiding or seeking the unknown in decision-making under uncertainty, J. Exp. Soc. Psyh. 43 (2007) 656. I. Yaniv, M. Milyavsy, Using advice from multiple sources to revise and improve judgments, Org. Beh. Hum. Dec. Proc. 103 (2007) 104. P.A. Samuelson, A note on measurement of utility, The Rev. Econom. Stud. 4 (1937) 155. D.O. Cajueiro, A note on the relevance of the q-exponential function in the context of intertemporal choices, Physica A 364 (2006) 385–388. T. Takahashi, A comparison of intertemporal choices of oneself versus some else based on Tsallis’ statistics, Physica A 385 (2007) 637–644. T. Takahashi, A probabilistic model based on Tsallis’ statistics, Physica A 386 (2007) 335–338. G.T. Fechner, Elemente der Psychophysik, Breitkopf und Hartel, Leipzig, 1860. F.S. Roberts, Measurement Theory, Encyclopedia of Mathematics and Its Applications, Addison-Wesley, Reading, MA, 1979. P. Grigolini, G. Aquino, M. Bologna, M. Lukovic, B.J. West, A theory of 1/f noise in human cognition, Physica A 388 (2009) 4192. D.L. Gilden, T. Thornton, M.W. Mallon, 1/f noise in human cognition, Science 267 (1995) 1837–1839. T. Takahashi, H. Oono, M.H.B. Radford, Psychophysics of time perception and intertemporal choice models, Physica A 387 (2008) 2066. V.I. Yukalov, D. Sornette, Physics or risk and uncertainty in quantum decision making, Eur. Phys. J. B 71 (2009) 533. B.J. West, E. Geneston, P. Grigolini, Maximizing information exchange between complex networks, Phys. Rep. 468 (2008) 1. M. Lukovic, P. Grigolini, Power spectra for both interrupted and perennial aging processes, J. Chem. Phys. 129 (2008) 184102. C. Beck, Dynamical foundatins of nonextensive statistical mechanics, Phys. Rev. Lett. 87 (2001) 180601. P. Grigolini, in: C. Beck, G. Benedek, A. Rapisardand, T. Tsallis (Eds.), Complexity, Metastability and Nonextensivity, World Scientific, Singapore, 2005, p. 72. P. Allegrini, V. Benci, P. Grigolini, P. Hamilton, M. Ignaccolo, G. Menconi, L. Palatila, G. Raffaelli, N. Scafetta, M. Virgilio, J. Yang, Compression and diffusion: a joint approach to detect complexity, Chaos Solitons Fractals 15 (2003) 517. G. Aquino, M. Bologna, P. Grigolini, B.J. West, Phys. Rev. E 70 (2004) 036105. P. Allegrini, D. Menicucci, R. Bedni, L. Fronzoni, A. Gemiginani, P. Grigolini, B.J. West, P. Paradsi, Spontaneous brain activity as a source of ideal 1/f noise, Phys. Rev. E 80 (2009) 061914. S. Bianco, E. Geneston, P. Grigolini, M. Ignaccola, Renewal aging as emerging property of phase synchronization, Physica A 387 (2008) 1387. P.A. Frantsuzov, S. Volkan-Kacso, B. Janko, Phys. Rev. Lett. 103 (2009) 207402. L. Silvestri, L. Fronzoni, P. Grigolini, P. Allegrini, Event-driven power-law relaxation in weak turbulence, Phys. Rev. Lett. 102 (2009) 104502. S. Frederick, G. Loewenstein, T. O’Donoghue, Time discounting and time preference: a critical review, J. Econ. Lit. 40 (2002) 351. R. Thaler, Some empirical evidence on dynamic inconsistency, Economics Lett. 8 (1981) 201. A.G. Samfey, G. Loewenstein, S.M. McClure, J.D. Cohen, Neuroeconomics; cross currents in research on decision making, TRENDS in Cog. Sci. 10 (2006) 108. M. Wittmann, D.S. Leland, M.P. Paulus, Time and decision making: differential contribution of the posterior insular cortex and the stiatum during a delay discounting task, Exp. Brain Res. 179 (2007) 643.