Webscraping, Feature Engineering, Dataset Creation Progress Assignment

DATASETS:

trancolist (Legitimate URLs) and phishtank (Phishing URLs) datasets were used.

LIBRARIES INSTALLED (only till review 2):

• beautifulsoup4
• Matplotlib
• Pandas
• Requests
• urllib3

CONCEPTS (only till review 2) – WEBSCRAPING, FEATURE ENGINEERING, DATASET CREATION PROGRESS TILL REVIEW-2:

• Using BeautifulSoup module, soup object was created and a vector was created by calling all the functions for the soup object. Feature Extraction was defined and the data was saved in a directory.
• The .csv datasets were converted to dataframe using pandas library. URLs were retrieved from both datasets and the above-mentioned features were extracted. We extracted data from only 1000 URLs from both datasets for now and created structured data.
• The new dataframes with the values of these features(structured data), were created for both Legitimate and Phishing datasets.
• These dataframes were converted to .csv files our new datasets upon which we will work using various Machine Learning algorithms.

RESULTS:

Successfully created datasets for Legitimate and Phishing websites by webscraping, feature engineering, dataframes, datasets concepts.

Are you struggling to keep up with the demands of your academic journey? Don't worry, we've got your back! Exam Question Bank is your trusted partner in achieving academic excellence for all kind of technical and non-technical subjects.

Our comprehensive range of academic services is designed to cater to students at every level. Whether you're a high school student, a college undergraduate, or pursuing advanced studies, we have the expertise and resources to support you.

To connect with expert and ask your query click here Exam Question Bank