Removing spikes from samples
I am reading a book called Fooled by Randomness by a derivatives trader Nassim Nicholas Taleb.
In one of the chapters he mentions about how sampling and removal of spikes from the samples can lead to improper interpretation of facts...
In the section called Symmerty and Science on Page 105, he explains how most people remove the highest and lowest values while calculating averages. Below I summarize his arguement
Example 1:
A professor would remove the highest and lowest marks as spikes and uses only the rest of the values to find an average of class marks. The author feels it is an unsound practice.
Example 2:
A whether forcaster removes the spikes in the temperature while calculating the average temperature assuming that these are rare events and would not occur again.
Suppose somebody uses this average temperature to forcast global warming and the rate of melting of polar ice caps, the guy might horribly go wrong because while calculating the average temperature, the spikes were ignored as one off event. But this one off event could have caused disproportionate melting of ice. By ignoring this spike in temperature you ignored the irreversible affect this specific temperature rise could have caused on the ice caps.
Example 3:
He uses the above two arguements to nail the stock/derivatives markets forcasters and says:
I quote
"So people in finance borrow the technique and ignore infrequent events, not noticing that the effect of a rare event can bankrupt a company"
So such events although rare, brings large consequencs that cannot be just ignored"
unquote
A recent example of one such thing that immediately comes to my mind is the subprime crisis. In the history of Baer Sterns They might have made so many intelligent investments, stratergic buyouts. But Subprime Crisis brought the investment bank on it knees. So one rare event crippled the whole bank. If you were to remove this one event from the history of Bear Sterns and look at the performance of Baer Sterns you would feel that it is an excellent investment company... Then why on earth did it crash from close to $100 to $2 !!
In one of the chapters he mentions about how sampling and removal of spikes from the samples can lead to improper interpretation of facts...
In the section called Symmerty and Science on Page 105, he explains how most people remove the highest and lowest values while calculating averages. Below I summarize his arguement
Example 1:
A professor would remove the highest and lowest marks as spikes and uses only the rest of the values to find an average of class marks. The author feels it is an unsound practice.
Example 2:
A whether forcaster removes the spikes in the temperature while calculating the average temperature assuming that these are rare events and would not occur again.
Suppose somebody uses this average temperature to forcast global warming and the rate of melting of polar ice caps, the guy might horribly go wrong because while calculating the average temperature, the spikes were ignored as one off event. But this one off event could have caused disproportionate melting of ice. By ignoring this spike in temperature you ignored the irreversible affect this specific temperature rise could have caused on the ice caps.
Example 3:
He uses the above two arguements to nail the stock/derivatives markets forcasters and says:
I quote
"So people in finance borrow the technique and ignore infrequent events, not noticing that the effect of a rare event can bankrupt a company"
So such events although rare, brings large consequencs that cannot be just ignored"
unquote
A recent example of one such thing that immediately comes to my mind is the subprime crisis. In the history of Baer Sterns They might have made so many intelligent investments, stratergic buyouts. But Subprime Crisis brought the investment bank on it knees. So one rare event crippled the whole bank. If you were to remove this one event from the history of Bear Sterns and look at the performance of Baer Sterns you would feel that it is an excellent investment company... Then why on earth did it crash from close to $100 to $2 !!
Labels: Fooled By Randomness, Nassim Nicholas Taleb, Removing spikes from samples
posted by amitkeerti | 9:21 PM
0 Comments:
Post a Comment
Subscribe to Post Comments [Atom]
<< Home