1872-2020 SP500 index : a [time, mind]-bending perspective

"... History never repeats itself, but it rhymes. ..."
(originators : ) TradingView member ??

Table of Contents

"Key [results, comments]"

Standard & Poor's index of stock market prices for 500 large capitalisation companies (SP500) from 1872 through 2020 is shown in the semi-log plot below. Two "eyeballed" linear trend lines are shown for the periods [1872-1926, 1926-2020].

The ratio of the historical SP500 price to its semi-log trend price for the same date is plotted in the graph below. The price scaling is according to Fibonnacci ratios, as commonly used by many stock market technical analysts, which I often find to be helpful or at least interesting, as it so often provides convenient matches with data series.

The real fun starts when I rather [arbitrarily, haphazardly] copy-pasted sections of the graph from the 1950-2020 timeframe, and shuffled them to lie over sections of the 1872-1960 section of the original graph. Here is one possible result :

You will notice that I was too cowardly to fill in the 1890-1900 section with an overlay, perhaps from ~1970-1980. That would be equivalent to a forecast of sorts, but what I learned from commodity chemical market research is that "Forecasting is a particularly hazardous occupation, especially if you try to apply it to the future". I also feel that it is "dangerous" to have only one forecast, just as it is dangerous to "believe" only one scientific theory, even if it is an overwhelmingly dominant mainstream consensus science theory-cum-cult-cum-religion. I try to force a "multiple conflicting hypothesis" attitude on myself, even if I have to find a really [defunct, dishonest, abandoned] theory. Otherwise, I often find that I become a tool of the theory, rather than the theory being a tool to be used if and when [desired, needed, appropriate]. Most of all, a theory should NOT become an impediment to [thinking, progress], as is far too often the case.

So maybe YOU can generate forecasts, not necessarily with the transparency overlays.

Play with the [time, mind]-bending perspective yourself

You can move these transparencies around in gimp (but not in the image above, which is a jpg format). Furthermore, you can do your own models by copy-paste of sections of a graph, and moving them around. Of course, with the [spreadsheet, .dot text] files, you can impose your wown data transformations as well.

Deja Vu all over again (Yogi Berra) : - This "chopping up a time series" and omitting sections to recreat a time seeries with a model is eerily familiar : Perhaps this section is saying something important about an over-reliance of modern approaches on [modelling, statistics], at the expense of simple pattern-matching? And perhaps it is just me who has this problem...

Furthermore, a key lesson may be what I have long felt for [physics, astronomy, climate, etc] series : the real challenge isn't prediction of the next steps in a time series, but to predict [when, to which state] a system will change in the future.

Ratio of actual to semi-log detrended data : [advantages, disadvantages]

Time greatly distorts the measuring sticks for [money, economics, finance, markets], so it is very important to convert the data before making comparisons across time. This web-page uses a simple semi-log conversion of the S&P500 index opening prices from 1872 to 2020. While other basis such as adjustments with [Producer, Consumer,]-Price Indexes or populations might be better in some ways, the current approach has the advantage that it does not [distort, polute] the data at an early stage, and it is very [easy, clear] to double-check results.

For sure, one would normally do a simple linear regression to generate the [xStart, yStart, slope] parameters for the trend lines for [1872-1926, 1926-2020], however, at this preliminary stage it is more important for me to look at the data and to avoid facetious levels of [accuracy, certainty, self-delusion]. The eye is often much better than ordinary analysis.

line This "semi-log detrended" format has many advantages :

Some disadvantages of the format are :

Future potential work

It's unlikely that I will get much time for this, but here are some ideas that could be interesting.

Comparison of [TradingView, Yahoo finance] data

It's always important to use several sources of data, where possible. [errors, fraud, deception, incomplete disclosure, mis-understood] data treatments aren't rare in science (I don't know about [economics, markets]). But more important is that I may not have understood the [data, analysis, modelling, reality] or it's limitations well enough. Here I merely compare two sources of the same data, which brings some confidence.

TradingView versus yahoo finance - the data series are essentially the same, as no black curve excursions above, below the blue yahoo curve are seen.

TradingView versus 2 times yahoo finance - by multiplying the yahoo data by 2, it is much easier to see the essentially perfect similarity of the data series. This helps as the first comparison above leaves me thinking that maybe something is missing.

[data, software] cart [description, links]

While most results are provided in sections above, links to data [spreadsheets, text files] and software [???, source code] are listed below along with brief comments. A full listing of files (including other SP500 web-pages) can be seen via this Directory's listing. Hopefully this will help those who want to do something different, as the programs etc mayhelp with [learning, debugging].

Data :
Software :

17Sep2020 initial posting

Directory of available files for this webpage

Copyright © 2007 through 2020
All website content is owned by William Neil Howell of Alberta, Canada, except when it isn't.

Permission is granted to copy, distribute and/or modify Howell's content under either:
The GNU Free Documentation License; with no Invariant Sections, Front-Cover Texts, or Back-Cover Texts.
Creative Commons License Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
It is expected that reusers of this web-site will:
  • Acknowledge William Neil Howell and/or the specific author of web-site content,
  • Provide a link or reference back to the specific web-page carrying the content that is being used,
  • Allow any modifications made to the content to also be reused under the terms of either of the two standard licences (GNU or Creative Commons).