BestBnB is a web app based on a multi-step recommendation algorithm that allows you to tailor your travel experience to match your lifestyle and wallet. It is written mostly Python and Javascript using Flask, postgres, sklearn, xgboost, pandas, bokeh, and folium(leaflet.js). The app was developed during my time at Insight as a Data Science fellow.

The bulk of functions are in the airbnb_pipeline.py file. There are also a few jupyter notebooks that can be viewed on ghitub in airbnb/Exploratory_Notebooks/

The data is comprised mostly of AirBnB listings and reviews that have been scraped by Murray Cox and are publicly available here.

Here's a very high-level diagram of the process. There are two basic models: The Price model and the Neighborhood/Map model.

The Neighborhood Model is NLP pipeline in sklearn:

model = make_pipeline(TfidfVectorizer(stop_words = "english", min_df = 5,
                                      ngram_range = (1,2)),
                      TruncatedSVD(100),
                      Normalizer())

knn = NearestNeighbors(500, metric = "cosine", algorithm = "brute")

Every descrption is vectorized using bigram TF-IDF and then projected onto a 100-dimensioal latent space. When a new query is sent through the pipeline the Nearest Neighbors model finds the closest listings in the latent space using cosine similarity and uses the latitude and longitude to build the heat map and neighborhood scores.

For the price model I went with gradient boosting regression with xgboost since it performed significantly better than Ridge Regression and handles missing values gracefully. The improvement is, from what I can tell, mostly due to the fact that some features like latitude and longitude vary nonlinearly with respect to price.

model = xgb.XGBRegressor(learning_rate = 0.1, max_depth = 6, n_estimators = 150)

Ensembling with a neural net helps a bit but not enough to warrant the hassle in my opinion.

###Acknowledgements:

A big thank you to everyone at Insight especially the program directors and my fellow fellows for being so cool and helpful. Also thaks to Murray Cox for scraping the AirBnB data and to all the great people who contributed to all the open source packages I used for this project.

Make sure to have miniconda(wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh) and postgres installed. Here's a good tutorial on setting postgres up with ubuntu.

• clone the repo: git clone https://github.com/apapiu/airbnb_app.git

• create conda environment:

  conda create --name airbnb_app python=3 pandas scikit-learn folium sqlalchemy psycopg2 flask bokeh

• activate the environment: source activate airbnb_app

• pip install some stuff not on conda: pip install sqlalchemy-utils

• Create a username and database in postgres and add the needed enviornment variables in your .bashrc (or .bash_profile if on MacOS):

    export home_folder=$'/the/folder/you/cloned/the/repo/in'
    export dbname=$'your_database_name'
    export username=$'your_username' 
    export password=$'*****' #only needed for linux
  

• run: python /airbnb/get_data.py

• run: python /airbnb/models.py

• cd airbnb/web_app

• ./run.py