Acquisition

Tropical storms and hurricanes are highly monitored and studied weather phenomena. As such, there is a great deal of data collected about every storm system that has the potential or achieves the status of tropical storm or hurricane. This data includes a unique identifier for tracking, storm position, and assorted meteorological measurements and characterizations. These measurements and characterizations include date and time of measurement, wind speed, system pressure, position (latitude and longitude). The measurements are recorded approximately every three hours when a system is being monitored. 

Weather monitoring is an international collaborative effort. Tropical systems can be enormous and have the potential to impact multiple countries in a short amount of time. Thus, the complete data set records can include observations from multiple weather organizations. 

NOAA Historical Hurricane data is posted as a static data set on an official government website. The team did investigate the availability of an API for this data and while there are NOAA API resources, the team was not successful in identifying one that provided historical tropical weather data. As this data set is historical and the period of execution of this work is outside hurricane season, there is no requirement for continuously updated or dynamic data that would be provided by an API. The research objectives are focused on leveraging historical data to identify patterns and relationships with other historical data, therefore the data source is appropriate and relevant for the intended application.

Data Source

Download data .csv file

Gas prices change daily and differ based on location, meaning there is an outstanding amount of data available. This data is broken down by metro with its average price and price by fuel grade. 

The gas analysis is supported by data from the AAA Fuel prices database. This is a reputable, open source, online dashboard that contains information pertaining to all 50 United States. This can be drilled up to show nationwide averages, or drilled down to metro level by state. AAA provides real time, daily updates to the dashboard. To access data from previous years on that observational level, we utilized the Wayback Machine from Internet Archive. We scraped historic data from the Wayback machine using the Florida AAA gas prices website. This resulted in a .csv file with each row being that observation for a certain day in a certain metro.

On the nationwide level, statewide, and metro-level, the observations are formatted in a simple table with rows being the date of record along with "current avg", "yesterday avg", "week ago avg", "month ago avg", and "year ago avg" for each column, which represents types of gas. 

There were no applicable APIs that would allow us to get this rich dataset. The AAA data set through the Wayback Machine was the optimal choice for the data collection as no other online source would have provided means to access metro level data at the required observational level.

• Per discussion with Dr. Alfonso Bastias, a web scraping algorithm curated partially by ChatGPT was implemented only to collect the data 

Data Source

Method of Historical Acquisition