To run the solution:
-
Navigate to the downloaded and unzipped EQWSampleProblem folder.
-
In the command line, enter:
spark-submit --deploy-mode client <Path to pysparkJob.py>
The program will create a file called output.txt that provides the significant values for POI1, POI2, and POI3.
You can find a more detailed discussion about difficulties, assumptions, and results of this assignment in the Data Problem Report.