Data Warehousing and Mining Assignment

Introduction

Airbnb has successfully disrupted the traditional hospitality industry as more and more travellers decide to use Airbnb as one of the primary accommodation providers. From the Barwon South West data set provided in inside Airbnb, we hope something valuable to potential investors and hosts could be found out with data mining and machine learning.

Task 1: Data Pre-processing

Pre-processing is designed to select the proper columns data to work with and clean the dataset like removing the Nan values and dealing with the data format.

Task 2: Exploratory Data Analysis (EDA) with Data Visualization

As we are focusing on predicting the prices for accommodations and finding out features that contribute to high prices. We first see the price distribution through boxplot and real street map.

Task 3: Building the Accommodation Prediction Model

Now, we are trying to build a model to predict the price. The samples are divided into a training set (80?ta samples ) and a testing set (20?ta samples).

Task 4 ?Advanced tasks??Sentiment analysis

1. Deciding which columns to work with
   We want to keep the information from the dataset as much as possible while removing those irrelevant columns. Removing the irrelevant information could effectively reduce the unnecessary information and avoid the curse of dimensionality, thus to increase the model’s performance.
2. Cleaning prices and dealing with missing values Operations to change the currency to float values and drop the rows with Nan values
3. Price column visualization Vilsulize the price distribution of accommodations with boxplot
4. Accommodation distribution on maps You could use the opensource leaflet (python interface), google maps or any other tools to visualize the accommodation distribution based on the latitude and longitude of each accommodation.
5. Summarize the number of accommodations in each market/ each region
6. Summarize the mean price of accommodations in each market/ each region
7. Choose a supervised model such as Xgboost, ANNs and other models to implement your price predictor
8. Perform an analysis to discuss what kinds features are most related to the accommodation price
9. Perform the sentiment analysis for the review comments
10. Analysis of the reasons why people like and dislike the accommodations (open question) The analysis could be performed in the following aspects: The hotel view, the location, staff attitude and