This page is dedicated to an AI Internship experiment. This page shows the main page and the sub page of a given website interns should scrape.

Task Description: We are going to go back to the scraping tool we built in Week 1. Remember that we scraped job title, employer, job location and url information from this URL (https://www.builtinnyc.com/jobs ). This time, we are going to build a comprehensive web scraping tool that scrapes a list of URL (https://www.builtinnyc.com/jobs ) that not only collects the same information (Job Title, Employer, Job Location and URL of the link to the job posting description page) but also access that  job posting description page, and scrape any text block that contains information about required skill sets/ general requirements for the job. Again, this job posting description page can be accessed by clicking on the ‘block’ of the job, or the job title name. Note that clicking on the employer information will NOT take you to the job posting description page but rather an information page about the employer.

The code should be able to run at any given point in time. This job posting website updates frequently throughout the day, and the code should be generating different outputs as it should scrape the current website.

The scraped information should be saved in a table format as a csv file under the title ‘job_req_builtin.csv’.