A streamlined and efficient ETL pipeline for online rental listing data.

This project gathers rental listing data from the Rentfaster.ca API. A custom data engineering process cleans the data and stores it in the appropriate format for analytical queries.

The data collected by this service forms the foundation for The Calgary Project.

Dependencies can be installed via:

pip install requirements.txt

Testing for this project is handled by pytest.

In the project directory, type:

$ pytest tests/test.py

All tests should pass before running app.py.

• Requests - HTTP library
• Pandas - Python data analysis library
• mysql-connector-python - MySQL driver written in python
• Schedule - Python job scheduling for humans.
• pytest - Testing library

• Microsoft Azure - Cloud service
• Docker - Virtualization

Feedback and constructive criticism is more than welcome!

• Andrew Moss - Creator - agmoss

This project is licensed under the MIT License - see the LICENSE.md file for details

• Rentfaster.ca