 Library "DataCorrelation"
Implementation of functions related to data correlation calculations. Formulas have been transformed in such a way that we avoid running loops and instead make use of time series to gradually build the data we need to perform calculation. This allows the calculations to run on unbound series, and/or higher number of samples

🎲 Simplifying Covariance

Original Formula
```//For Sample
Covₓᵧ = ∑ ((xᵢ-x̄)(yᵢ-ȳ)) / (n-1)

//For Population
Covₓᵧ = ∑ ((xᵢ-x̄)(yᵢ-ȳ)) / n```

Now, if we look at numerator, this can be simplified as follows
```∑ ((xᵢ-x̄)(yᵢ-ȳ))

=> (x₁-x̄)(y₁-ȳ) + (x₂-x̄)(y₂-ȳ) + (x₃-x̄)(y₃-ȳ) ... + (xₙ-x̄)(yₙ-ȳ)
=> (x₁y₁ + x̄ȳ - x₁ȳ - y₁x̄) + (x₂y₂ + x̄ȳ - x₂ȳ - y₂x̄) + (x₃y₃ + x̄ȳ - x₃ȳ - y₃x̄) ... + (xₙyₙ + x̄ȳ - xₙȳ - yₙx̄)
=> (x₁y₁ + x₂y₂ + x₃y₃ ... + xₙyₙ) + (x̄ȳ + x̄ȳ + x̄ȳ ... + x̄ȳ) - (x₁ȳ + x₂ȳ + x₃ȳ ... xₙȳ) - (y₁x̄ + y₂x̄ + y₃x̄ + yₙx̄)
=> ∑xᵢyᵢ + n(x̄ȳ) - ȳ∑xᵢ - x̄∑yᵢ```

So, overall formula can be simplified to be used in pine as
```//For Sample
Covₓᵧ = (∑xᵢyᵢ + n(x̄ȳ) - ȳ∑xᵢ - x̄∑yᵢ) / (n-1)

//For Population
Covₓᵧ = (∑xᵢyᵢ + n(x̄ȳ) - ȳ∑xᵢ - x̄∑yᵢ) / n```

🎲 Simplifying Standard Deviation

Original Formula
```//For Sample
σ = √(∑(xᵢ-x̄)² / (n-1))

//For Population
σ = √(∑(xᵢ-x̄)² / n)```

Now, if we look at numerator within square root
```∑(xᵢ-x̄)²

=> (x₁² + x̄² - 2x₁x̄) + (x₂² + x̄² - 2x₂x̄) + (x₃² + x̄² - 2x₃x̄) ... + (xₙ² + x̄² - 2xₙx̄)
=> (x₁² + x₂² + x₃² ... + xₙ²) + (x̄² + x̄² + x̄² ... + x̄²) - (2x₁x̄ + 2x₂x̄ + 2x₃x̄ ... + 2xₙx̄)
=> ∑xᵢ² + nx̄² - 2x̄∑xᵢ
=> ∑xᵢ² + x̄(nx̄ - 2∑xᵢ)```

So, overall formula can be simplified to be used in pine as
```//For Sample
σ = √(∑xᵢ² + x̄(nx̄ - 2∑xᵢ) / (n-1))

//For Population
σ = √(∑xᵢ² + x̄(nx̄ - 2∑xᵢ) / n)```

🎲 Using BinaryInsertionSort library

Chatterjee Correlation and Spearman Correlation functions make use of BinaryInsertionSort library to speed up sorting. The library in turn implements mechanism to insert values into sorted order so that load on sorting is reduced by higher extent allowing the functions to work on higher sample size.

🎲 Function Documentation

chatterjeeCorrelation(x, y, sampleSize, plotSize)
Calculates chatterjee correlation between two series. Formula is - ξnₓᵧ = 1 - (3 * ∑ |rᵢ₊₁ - rᵢ|)/ (n²-1)
Parameters:
x: First series for which correlation need to be calculated
y: Second series for which correlation need to be calculated
sampleSize: number of samples to be considered for calculattion of correlation. Default is 20000
plotSize: How many historical values need to be plotted on chart.
Returns: float correlation - Chatterjee correlation value if falls within plotSize, else returns na

spearmanCorrelation(x, y, sampleSize, plotSize)
Calculates spearman correlation between two series. Formula is - ρ = 1 - (6∑dᵢ²/n(n²-1))
Parameters:
x: First series for which correlation need to be calculated
y: Second series for which correlation need to be calculated
sampleSize: number of samples to be considered for calculattion of correlation. Default is 20000
plotSize: How many historical values need to be plotted on chart.
Returns: float correlation - Spearman correlation value if falls within plotSize, else returns na

covariance(x, y, include, biased)
Calculates covariance between two series of unbound length. Formula is Covₓᵧ = ∑ ((xᵢ-x̄)(yᵢ-ȳ)) / (n-1) for sample and Covₓᵧ = ∑ ((xᵢ-x̄)(yᵢ-ȳ)) / n for population
Parameters:
x: First series for which covariance need to be calculated
y: Second series for which covariance need to be calculated
include: boolean flag used for selectively including sample
biased: boolean flag representing population covariance instead of sample covariance
Returns: float covariance - covariance of selective samples of two series x, y

stddev(x, include, biased)
Calculates Standard Deviation of a series. Formula is σ = √( ∑(xᵢ-x̄)² / n ) for sample and σ = √( ∑(xᵢ-x̄)² / (n-1) ) for population
Parameters:
x: Series for which Standard Deviation need to be calculated
include: boolean flag used for selectively including sample
biased: boolean flag representing population covariance instead of sample covariance
Returns: float stddev - standard deviation of selective samples of series x

correlation(x, y, include)
Calculates pearson correlation between two series of unbound length. Formula is r = Covₓᵧ / σₓσᵧ
Parameters:
x: First series for which correlation need to be calculated
y: Second series for which correlation need to be calculated
include: boolean flag used for selectively including sample
Returns: float correlation - correlation between selective samples of two series x, y

Pine library

In true TradingView spirit, the author has published this Pine code as an open-source library so that other Pine programmers from our community can reuse it. Cheers to the author! You may use this library privately or in other open-source publications, but reuse of this code in a publication is governed by House Rules.

Disclaimer