Skip to content

A web scraper tool that scrapes the AirAsia booking website

Notifications You must be signed in to change notification settings

FeelingsLw/AAScraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AAScraper

Python 3.2

AAScraper currently does not work with the latest AirAsia API

AAScraper is a web scraper tool that scrapes the AirAsia booking website to organically obtain the prices of all and the cheapest flights for a given date range, inspired by SkyScanner.

Installation

AAScraper uses a number of open source Python libraries to work properly.

  • bs4 - Beautiful Soup 4 to parse the HTML source(s)
  • requests and urllib to perform the requests

For installation run the command:

$ pip3 install requirements.txt

Usage

Included with the source is the file scrape.py that gives an example on how these functions are meant to be used. The function scrape() creates a file with the cheapest outgoing and return flights for every single day in a given date range.

Example output:

{
    "2017-12-12": {
        "outgoing": [
            {
                "flights": [
                    {
                        "destination": "14:50 (ICN)",
                        "origin": "07:20 (KUL)"
                    }
                ],
                "price": "699.00 MYR"
            },
            {
                "flights": [
                    {
                        "destination": "08:05 (DMK)",
                        "origin": "06:55 (KUL)"
                    },
                    {
                        "destination": "23:05 (ICN)",
                        "origin": "15:40 (DMK)"
                    }
                ],
                "price": "1,953.72 MYR"
            },
...
            {
                "flights": [
                    {
                        "destination": "15:10 (DMK)",
                        "origin": "11:20 (ICN)"
                    },
                    {
                        "destination": "00:15 (KUL) +1",
                        "origin": "21:00 (DMK)"
                    }
                ],
                "price": "1,237.75 MYR"
            }
        ]
    }
}

To-do

  • Identify cookie generation method or pre-load cookies to be used in API queries
  • Develop a front-end for simple data visualisation
  • Add proxy support

About

A web scraper tool that scrapes the AirAsia booking website

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages