The Evolution of the Market

I was always curious of what the market was like pre algorithms and computerized trading from market makers, and this is what prompted this research / article. Through the pursuit of this quesiton, I discovered some really surprising things. Mostly, the impact of the retail influx is actually quite visible in the data and statistics when you break down the market to its components. The exact time and the effect of the influx of retail and the "meme stock era" actually fundamentally changed market dynamics, as you will see how in this article!

So here I am to talk about the market evolution, as told by Statistics using the S&P. A very special thanks to Tradingview for giving such rich data on the S&P, allowing me to pull data as far back at 1888. Thanks so much Tradingview!

Now, lets get into it!

Introduction

The story of market evolution is really the story of how information is processed into price. From the ticker-tape era of the late 19th century to today’s machine-driven trading, each wave of innovation has left fingerprints in the data. With access to SPX data stretching back to 1888 (thanks again Tradingview!), we can actually test for these regime shifts.

My question was simple: did the rise of algorithmic and computer-driven trading — starting in the 1980s — measurably change the character of market price action? My thesis is that computer based alogirthms should have acted on the market in the following ways:

• a) Should make the market more linear in nature via use of computer algorithms heavily based in linear algebra.
  
• b) Reduced randomness in the data structure.
  
• c) Made markets more efficient.
  

To answer these questions and find whether my theses were in fact valid, I applied a battery of statistical tests, regime backtests, and structural break analyses across defined eras of market history.

Descriptive Statistics: Shifting Return Distributions

I first grouped the data into six eras:

• 1888–1910
  
• 1910–1950
  
• 1950–1970
  
• 1970–1990
  
• 1990–2010
  
• 2011–current
  

For each, I computed mean returns, volatility, skewness, kurtosis, and a normality test.

You can see the results in the table below:



Findings:

• Early markets (1888–1910) show wide swings and near-normal distribution.
  
• Post-WW2 (1950–1970) returns were calmer, with reduced volatility.
  
• From 1970–1990, skewness and kurtosis exploded, showing fat-tailed events — think oil crisis, stagflation, and 1987 crash.
  
• 2011–current is defined by higher kurtosis and volatility clustering, consistent with an environment dominated by algorithmic and high-frequency trading.
  

💡 Trading Tip: When kurtosis is high, risk is concentrated in rare but violent moves. Simple VaR (volatility) measures understate risk. Options traders often exploit this by buying long-dated wings (cheap out-of-the-money puts/calls) in high-kurtosis regimes.

Autocorrelation and Randomness

I then ran the Ljung–Box test (serial correlation) and Runs test (randomness), which you can see the results in the table below.

• Pre-1970 markets often failed randomness tests → returns weren’t fully efficient, suggesting exploitable patterns.
  
• Post-1990, autocorrelation is near-zero (high-frequency traders and quants arbitrage away serial dependence quickly).
  
• However, runs test still showed occasional streakiness, especially in 2011–current (momentum bursts).
  

💡 Trading Tip: Don’t fight market efficiency. In modern data, intraday edges based on lagged correlations vanish quickly. Better edge: look for volatility regime shifts or structural breaks rather than naive mean reversion. And we will get into this more later in this article!

Variance Ratio & Hurst Exponent: Random Walk vs. Persistence

Variance Ratio tests showed early markets >1 (predictable mean reversion), but after 1990 values dipped negative, which tends to signify momentum behavior. See the tables below:



Looking at the Hurst exponent, it hovered at ~0.55 pre-2000 (persistent trending), but dropped toward 0.48 post-2011 indicating a move from randomness to more stability.

💡 Trading Tip: Momentum is not dead, but its timescale is compressed. Where trends once lasted months, they now play out in days or weeks. Swing traders should shorten holding horizons in the modern era based on these results. And I am sure we all can relate after the initial crash we saw at the beginning of 2025 and how quickly it recovered! This quick recovery without retracement of lows showed up as a market rule from 2018 and on (more on that later).

GARCH Volatility Clustering

Before I get into this analysis, I just want to clarify what GARCH is, as it is discussed a bit among quant traders and chances are you may have heard it but not quite sure what it's all about.

GARCH — short for Generalized Autoregressive Conditional Heteroskedasticity — is a model designed to capture how market volatility clusters in time. Essentially, it recognizes that periods of calm trading are usually followed by more calm, and turbulent days are usually followed by more turbulence.

Instead of assuming volatility is constant, GARCH lets it “breathe” with the market:

When shocks hit (e.g., 2008 crisis, 2020 COVID crash), volatility spikes, and the model expects more big moves ahead.

When markets settle, volatility decays slowly rather than instantly snapping back.

This persistence — where high volatility begets high volatility — is one of the defining features of financial time series, and GARCH is the workhorse model used to measure it.

So keeping this in mind, let's discuss the results.

I Fitted multiple GARCH(1,1) models which gave me alpha + beta ≈ persistence.
What this means is summarized below by era:

• Pre-1980: persistence ~0.95 (long-lived volatility shocks).
  
• Post-2010: persistence ~0.97 — extremely sticky volatility.
  

This shows that volatility has become a regime in itself — shocks last longer and decay more slowly.

💡 Trading Tip: In persistent volatility regimes, selling short-dated options (expecting “vol will collapse”) is dangerous. Instead, structured spreads (calendars/diagonals) are safer because they profit from persistence.

Regime-Based Backtests: Momentum vs. Mean Reversion

I backtested two toy strategies inside each era:

Momentum: buy after up days.

Reversion: fade after up days.

Results:



Interpretation Tip: This chart shows 2 toy strategies applied, one based on momentum (i.e. last day was positive, I am going to just go ahead and long the next day, inverse if last day was negative) vs mean reversion (essentially playing to major SMAs). The lower the number, the better the strategy (negative numbers in this case are GOOD and positive are BAD, 0 is net flat).

Here is the summary of the results:

• 1910–1950: reversion dominated (thin markets, order-driven).
  
• 1970–1990: momentum exploded (indexing, funds, trend-followers).
  
• 2011–current: momentum again shows dominance, possibly linked to retail trading waves post-2018 (e.g., meme stocks, option gamma squeezes).
  

This is the result that shocked me the most. You can literally see from this chart, at about 2018, the market abandoned mean reversion in favour of momentum to a statistically significant extent!

This information is incredible and actually really forces me to rethink some of my mean reversion based strategies. This also happens to coincide with meme stock eras, early introduction of trading apps and the whole, as I call it, "democratization of trading for everyone". We can literally see the retail footprint show up and how retail has fundamentally shifted market dynamics away from mean reversion to more about momentum.

This just amazes me, I was never expecting to actually be able to physically see how dramatic retail has impacted the market! And this was never the intention of this research, it was focused mostly on looking at how the market has evolved in relation to computer algorithms and AI, but just happened to also pick up on the retail bandwagon influx in the crossfire.


💡 Trading Tip: Regime awareness matters. In reversion eras, fading strength is profitable. In momentum eras, chasing breakouts is. Today, evidence leans momentum, but in short bursts (intraday to multi-week).

Bai–Perron Structural Breaks

Oh man, this one was a nightmare.
Being a quant trader, I have some serious computing power and servers and this really gave them a run for their money.

This test essentially explores for statistically significant regime shifts. It identifies them on its own and returns the dates of the independent regimes. This took some hours to process, but essentially what it has done is identified, on SPX, independent regimens that are fundamentally different from each other.

Here is the raw table breakdown of the regimes:



And displayed overlaid with the close of the S&P:



Breaks detected:

• 1929–1933: Great Depression.
  
• 1973–1987: Oil crisis → Black Monday.
  
• 2000–2009: Dot-com → Global Financial Crisis.
  
• 2020: COVID volatility shock.
  

These align almost perfectly with historical crises.
The point of the function is essentially to just have an unbiased, algorithm validate that there are or have been independent shifts and regimes present in the market, without us imposing our own opinions (i.e. "the market has never been the same since 2008" and don't forget the million dollar "Trump market" (which by the way is disproved as significant using this analysis, there is no statistically significant difference in a "Trump market" or it would have shown up ;) ).

💡 Trading Tip: Structural breaks matter most to macro investors. Regime shifts reset correlations, volatility, and trend behaviors. After 2020, treating markets as “post-2010 continuation” is wrong — structurally, a new volatility regime has been in play.

Conclusion

So, what can we say about all of this?

The statistical fingerprint of markets has changed dramatically:

• Early 20th century: mean-reverting, inefficient.
  
• 1950–1970: calm postwar boom.
  
• 1970–2000: fat tails, trend-followers dominate.
  
• 2000–2010: crash-prone, clustered volatility.
  
• 2011–current: machine-driven randomness punctuated by bursts of momentum (often retail-driven).
  

To answer my initial question regarding whether the introduction of computing and AI fundamentally shifted the market, looking at the data, it suggests that algorithmic trading didn’t make markets “more linear.” Instead, it compressed timescales, enforced near-randomness, and amplified volatility persistence. Retail surges post-2018 added another layer: sudden, meme-like bursts of momentum.

But here are the things that surprised me the most and I think should be really taken away from this research and thought about. These are my observations:

• The market went from a true Random-walk situation from 1888 to 1950, to a more trendy and predictable version in 1950 to 1980.
  
• The era between 1888 and 1950 and the era between 1950 and 1990 are fundamentally different. These are not the same markets anymore and there aren't any visible remnants of our 1900s, 1920s, 1950s or even 1990s markets. This matters because we can't really compare this current market to say the dotcom bubble, since the factors that made up the market mechanics in that era are fundamentally different than currently. As well, those using strategies that are based on 'old regimens', such as EWT or certain pattern formations (for me, I use Bulkowski patterns who did the majority of his analyses and statistics in the 1990s) are defunct. The regimen is different, its changed and it is fundamentally different. Thus, is is unlikely that the traditional patterns from the 90s or the EWT as it was written in the 1930s, a regimen that was fundamentally different, mean reversion based, will hold up in the current market climate. Remember, 1920s to about the 1950s was a major mean reversion era, the market has now moved away from mean reversion. So these strategies built on those dynamics need to be approached with absolute caution.
  
  

In all, I am glad I spent hours doing this because I will have to look into revamping some of my own stuff to be more in line with the current era. I have noticed some of Bulkowski's patterns just don't work, and now it makes sense. I also noticed some of my old mean reversion strategies aren't that great anymore either, and now it makes sense.

Whether you are a technical trader or a quant trader, statistics can help you understand the reason and rationale and guide you in your pursuit of profitable trading, without diverging your strategy (you can remain technical based or quant based, you just can be informed about the nitty gritty of it all with stats). And I hope that this analysis/article helps you see the usefulness of stats in guiding your understanding of market mechanics.


I will leave you with some final pragmatic advice based on the analysis:

1. Trade shorter momentum bursts.
   
2. Respect volatility persistence.
   
3. Use structural break analysis to anticipate when “old rules” stop applying. (more advanced but if you are up for it!)
   

The key take away from all of this heavy stats stuff, if anything, is that we are in a momentum driven market that does not favour mean reversion and is quick to shake off downside volatility.

I hope you found this insightful, this took a bunch of time to process these analyses and then write this post, so if you enjoyed it and found it helpful, share some love with a like and/or comment!

Thanks so much everyone and as always safe trades!

Special thanks again to Tradingview for the great data!
To Grammarly for hopefully having edited errors in this post!
SORA for the cover art.
And to R for providing the means of the analysis.
As well, the biggest thanks to you all, the Tradingview community, for reading, interacting and engaging!