The purpose of this project is to display working knowledge in the areas of data collection, database design, and data visualization
- Python
- Libraries:
- Pandas
- Numpy
- Requests
- SQLAlchemy
- Flask
- Flask_SQLAlchemy
- SQLAlchemy
- SQLAlchemy_utils
- OS
- Math
- Psycopg2
- PostgreSQL
- Javascript
- Libraries:
- D3.js
- Plotly.js
- Google Charts
- Leaflet.js
- FusionCharts.js
To run this code, ensure that all the above libraries are installed in your environment
-
- Sign up for Data.gov API key.
-
- Sign up for Mapbox API key.
-
- Enter your Mapbox API in
config.js
located in the static folder.
- Enter your Mapbox API in
const API_KEY = 'ENTER YOUR API';
-
- Enter your data.gov API key AND PostgreSQL info in
config.py
- Enter your data.gov API key AND PostgreSQL info in
api_key = " ENTER YOUR API"
UserName = "YOUR POSTGRESQL USERNAME"
Password = "YOUR POSTGRESQL PASSWORD"
-
- Run the
RUNME.ipynb
Python Notebook
.
- Run the
-
- Run the
Flask
server by running theapp.py
file.
- Run the
-
- Run the IP address created by the
Flask
server which should look like http://127.0.0.1:5000/ or similar. This renders the index.html file containing the visualizations.
- Run the IP address created by the
Important Note:
It is only necessary to run the RUNME.ipynb
Python Notebook
.
This folder contains files that:
- Create the
Postgres
Database and theTables
that will store the information from the queries. - Query API from the Department of Education: College Scorecard
- Upload the API's queried information to the tables created in step 1.
- Upload auxiliar tables created as CSV files (from the API's documentation) to the database and tables created in step 1.
0_RUN_ME.ipynb
is the master notebook that runs all the other notebooks.1_ Entity Relation Diagram.png
shows the Diagram used to create the database and tables.1_create_tables.ipynb
contains the necessary code to create the Database and Tables.2_education_api.ipynb
contains the necessary code to query the Department of Education API: College Scorecard.3_upload_to_db_main_tables.ipynb
contains the necessary code to upload the information obtained in the API to the database.4_upload_to_db_auxiliar_tables.ipynb
contains the necessary code to load the CSV files in thedata
folder to the database.5_query_db.ipynb
contains the necessary code to query the database and creates test outputs to theoutput
folder, in CSV, JSON, and TXT formats.data
This folder contains the auxiliar information collected using the API's documentation. These additional tables are used to add descriptions to the codes extracted through the API.Output
This folder contains files in CSV, JSON, and TXT formats obtained in the query to the database described in file 5.queries
This folder contains files withSQL
code used to create the database and tables described in files 1. It also contains theSQL
code used to query the database as described in files 5.CollegeScorecardDataDictionary.xlsx
This is a file downloaded the Department-of-Education API's Documentation: College Scorecard. This contains the metadata explanation used in the visualizations and the information used to create the CSV files int thedata
folder.
This file contains the Python
code necessary to create a Flask
development server to deploy the visualizations created using Javascript
libraries.
This folder contains all the Javascript
code used to create the visualizations displayed through the HTMl files.
CSS
Folder containing CSS for html files.dashboard.js
file that populates metrics onindex.html
.metrics.js
loads data for College Financial Info visualization.state_centers.js
contains coordinates for the center of each state and US territory.slidingmap.js
loads data for in-state tuition average by state for sliding scale map.slidingmap_out_of_state.js
loads data for out-of-state tuition average by state for sliding scale map.fusioncharts-suite-xt
contains FusionCharts.js library documentation.fusioncharts-xt-definition
contains FusionCharts.js maps definitions.
This folder contains all the HTMl
code used to display all the visualizations generated by using the Javascript
code and libraries
.
template.html
Flask template file that provides framework for all otherHTML
files.index.html
landing page of websitemetrics.html
webpage to display College Financial Info.school_state.html
webpage to display school locations by state.references.html
webpage displaying libraries used in this project.sliding_scale.html
webpage to display schools by sliding scale.