Skip to content

Two sample Kolmogorov Smirnov test implemented in PySpark SQL

Notifications You must be signed in to change notification settings

Ben-Epstein/KS-2Samp-PySparkSQL

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 

Repository files navigation

KS-2Samp-PySparkSQL

Two-sample Kolmogorov-Smirnov test implemented in PySpark SQL

I wrote this because I needed a KS implementation that operated directly on a PySpark SQL dataframe. A Spark built-in two-sample KS test is only available through the RDD API, but the conversion to an RDD was slowing me down too much.

The KS test is only valid for continuous distributions.

About

Two sample Kolmogorov Smirnov test implemented in PySpark SQL

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%