Skip to content

Scrape Southwest Airlines' site for flight data.

Notifications You must be signed in to change notification settings

etucker/swa-scraper

 
 

Repository files navigation

This is a spider for use with Scrapy that crawls for and parses fares for one-way flights on Southwest's website and puts the fares in an AWS Dynamo DB

Setup after cloning repo on linux EC2 instance: sudo yum install git sudo yum install emacs sudo yum install make sudo yum install libffi-devel sudo yum install gcc sudo yum install openssl-devel sudo yum install -y gcc libxml2 libxml2-devel libxslt libxslt-devel sudo pip install scrapy sudo pip install boto3

copy aws credentials into /home/ec2-user/.aws/credentials

create privateConstants.py: author = '' password = ''

About

Scrape Southwest Airlines' site for flight data.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%