Scrape and localise kijiji ads out of a map.

This tool has been inspired by the scrapy crawler kijiji_scraper. It is not intended to be generic but can be easily adapted to fit your needs.

The scraper is based on the scrapy framework to retrieve attributes like the title, address and price of an ad. The kijiji start url should use the english locale setting to be scraped.

The geocoding is based on the geocoder module and uses the following geocoding services: google, osm, mapquest. Take a look at the geocoder providers informations to know limits of each services (for example mapquest requires an API key).

The map is generated using the folium module with the default setting to use the openstreetmap tile.

This script works on Python 3.

Packages requirements:

• scrapy v1.1.0
• geocoder v1.15.0
• folium v0.2.1

Each packages can be install using pip:

$ pip install <package>

Enter the following to generate the map, and let it run until the end otherwise the map won't be generated. note: it could take few minutes depending the number of pages you want to scrap.

$ ./scrapyjiji

Open the generate map file in your browser:

$ firefox map.html

Released under the MIT License.