Exploring the Opioid Crisis

Dataset #1: Analyzing Opioid Prescription Rates by State and Provider Type

Source: Centers for Medicare and Medicaid Services – Medicaid Opioid Prescribing Rates by Geography
Summary of data and research questions it may address: The dataset contains opioid prescription rate data at the national level and state levels. It can help us address our research questions: What states have the highest rates of opioid prescriptions? How are prescription rates changing over time?
Collection method: Data was requested using the healthdata.gov API and loaded directly into the code file using the dataset's unique identifier. JSON data was converted into a data frame and CSV file for analysis.
Cleaning and preprocessing:
- Link to cleaning
- Handling issues and noise: Redundant columns were dropped, columns were renamed for better understanding, and data was split into national and state-specific data. Some columns contained NA values; those were dropped because they were not relevant to the research questions. For the national data, rows referring to subgroup insurance types were removed because only total reports will be used in the analysis. At the state level, rows containing data from 2019 were selected to perform a comparison between states in the most recent year of this dataset.

Before Cleaning

Note redundant columns and unclear plan type abbreviations refer to full code to see NA values and other noise in data.

After Cleaning: National

Condensed and cleaned data to analyze opioid prescription trends at the national level from 2013-2022.

After Cleaning: State

Condensed and cleaned data to analyze opioid prescription trends at the state level in 2019.

- Understanding the data: The data is split into two data frames to perform analysis on the national level and the state level. The opioid prescription rate and the long-acting prescription rate data were both converted to floating-point values for analysis. The opioid prescription rate = (# opioid prescriptions/total prescriptions) * 100. The long-acting opioid prescription rate = (# long-acting opioid prescriptions/ total opioid prescriptions) * 100. These transformations are useful, especially at the state level, because state populations vary vastly, and accounting for total prescriptions allows for more direct comparison.
- Statistical analysis: The mean opioid prescription rate across all 50 states is 3.65%, and the median is 3.3%. The maximum value of 10.45% is much larger than the central tendencies, indicating potential outliers. The mean long-acting opioid prescription rate across all 50 states is 11.24%, and the median is 7.45%. The maximum value of 79.56% is much larger than the central tendencies, so the data is likely right skewed. This is investigated below with boxplots.

Figure 1. Boxplots for statistical analysis of total opioid prescription rate and long-acting opioid prescription rate by state. Outliers on both graphs reveal the data is right-skewed, but the vast majority of states fall within 1.5x of the IQR.

- Data quality assessment: This data is complete, consistent, and usable. For time series analysis, data for each year is observed, and for state comparisons, data is reported for every state. Columns that were missing values were not relevant to the analysis, and once removed, the data was complete for our purposes.
- Advanced data understanding and visualizations:

Figure 2. Q-Q plots reveal that the data is not normal, as there is some structured deviation from the theoretical normal line. However, these are small datasets containing one data point for each year between 2013 and 2022, so normality cannot be assumed.

Figure 3. Trends in nationwide opioid prescription rates from 2013 to 2022. The overall opioid prescription rate (blue line, the percentage of opioid prescriptions out of all prescriptions) has been steadily and significantly decreasing over time. However, the proportion of long-acting opioid prescriptions out of all opioid prescriptions (green line) began significantly increasing in 2017. Long-acting opioids are more likely to lead to overdose (NIH), so this change in long-acting prescription rate is important to be aware of.

Figure 4. Color-coded map of the United States and corresponding bar chart to reveal opioid prescription rates by state. The map reveals Montana and Virginia to have significantly higher opioid prescription rates compared to other states. The bar chart supplements this by revealing the next eight states with the highest rates. This visualization is particularly useful at the state level to investigate the causes of higher and perhaps unnecessary opioid prescriptions.

Dataset #2: Analyzing Drug Overdose Rates by Location and Drug Type in 2019

Source: https://www.cdc.gov/nchs/nvss/vsrr/prov-drug-involved-mortality.htm
Summary of data and research questions it may address: The dataset contains monthly drug overdose death counts from 2019 onward, categorized by location and drug type. It can help us address our research questions: (1) What states have the highest rates of opioid overdoses? (2) What drug classes are experiencing higher overdose rates?
Collection method: Downloaded CSV from the government website
Cleaning and preprocessing:
- Link to cleaning
- Handling issues and noise: Rows where the 'jurisdiction_occurrence' column was labeled as 'United States' were removed, along with unwanted columns, including 'data_as_of,' 'time_period,' 'month_ending_date,' and 'footnote.' Column names were updated as follows: 'death_year' to 'Year,' 'death_month' to 'Month,' 'jurisdiction_occurrence' to 'Location,' 'drug_involved' to 'Drug,' and 'drug_overdose_deaths' to 'Deaths.' Additionally, rows with empty values in the 'Deaths' column were removed, and commas were removed from the 'Deaths' values before converting them to an integer data type. The dataset did not include duplicates.

Before Cleaning

The dataset contained multiple columns that did not pertain to our research questions. 462 rows listed 'United States' as the 'jurisdiction_occurrance' while the majority of the data was listed by region, and there were missing values in 'drug_overdose_deaths.' Additionally, 'death_year,' 'death_month,' and 'drug_over_deaths' were objected data types and contained commas, which made them unusable for statistical analysis.

After Cleaning

Columns that were unrelated to our research questions were removed, rows with missing values and 'United States' in the 'jurisdiction_occurrance' column were removed, column names were renamed for ease of handling, and the dataset was refined to include 'Year' and 'Month' as integers, 'Location' and 'Drug' as objects, and 'Deaths' converted to an integer data type for better numerical analysis.

- Understanding the data: Before cleaning, the dataset contained various data types. Columns such as 'data_as_of,' 'jurisdiction_occurrence,' 'drug_involved,' 'time_period,' 'month_ending_date,' 'drug_overdose_deaths,' and 'footnote' were stored as objects, while 'death_year' and 'death_month' were integers. After cleaning, the dataset was refined to include 'Year' and 'Month' as integers, 'Location' and 'Drug' as objects, and 'Deaths' converted to an integer data type for better numerical analysis.
- Statistical analysis: The summary statistics for drug-related death counts indicate that Fentanyl has the highest average number of deaths, with a mean of 5,888 and a median of 4,805, followed by Methamphetamine (mean: 2,601, median: 1,726.5) and Cocaine (mean: 2,420, median: 2,073.5). Heroin has a lower mean of 998.4 deaths, while Gabapentin (mean: 366.99), Oxycodone (mean: 469.12), and Xylazine (mean: 346.11) report the lowest averages. The highest single recorded death count is associated with Fentanyl (18,317 deaths), while Xylazine has the lowest recorded count (10 deaths). In terms of variability, Fentanyl exhibits the highest variance (2.07 × 10⁷) and standard deviation (4,554.27), suggesting a wide range of reported deaths. Conversely, Gabapentin has the lowest variance (8.89 × 10⁴) and standard deviation (298.19), indicating more consistency in reported deaths. The skewness values suggest that all drug-related deaths are right-skewed, meaning there are a few extremely high values influencing the distribution. Oxycodone (1.87 skewness) and Gabapentin (1.62 skewness) show particularly strong right-skewed distributions, while Cocaine (0.65 skewness) and Fentanyl (0.90 skewness) have comparatively lower skewness, indicating a somewhat more balanced distribution.
- Data quality assessment: The dataset aligns with the research project's objectives by looking at where opioid overdose death between 2019-2024 was most common in the United States. It also helps us by looking at what specific drugs led most to overdose deaths in the United States between 2019 and 2024. The dataset is very usable. The only inconsistencies were that 462 rows listed 'United States' as the 'jurisdiction_occurance' while the rest of the data was listed by region, and there were missing values in 'drug_overdose_deaths.' Both of these rows were removed in the data-cleaning process for completeness and consistency.
- Advanced data understanding and visualizations:

Figure 5. Q-Q Plots examining normality of data by drug type. Fentanyl and Methamphetamine–These drugs exhibit particularly strong right-skewness and heavy tails, indicating a wide range of reported deaths with some extreme cases. Oxycodone and Gabapentin–These also display significant right-skewness but appear to have more pronounced deviations from normality in the middle quantiles. Cocaine and Heroin – While still right-skewed, these drugs exhibit slightly less extreme departures from normality compared to Fentanyl and Methamphetamine.

Figure 6. This visualization displays the total overdose deaths per drug in the U.S. from 2019 to 2024. The data indicates that Fentanyl has the highest number of overdose deaths, significantly exceeding other drugs. Methamphetamine and Cocaine also show large numbers of fatalities, though considerably lower than Fentanyl. Other substances, including Heroin, Oxycodone, Gabapentin, and Xylazine, have lower but still notable overdose death counts. This chart highlights the severity of the opioid crisis, particularly the dominant role of Fentanyl in overdose-related fatalities.

Figure 7. This visualization presents the total opioid overdose deaths per region in the U.S. from 2019 to 2024. The data is broken down by opioid type: Fentanyl (blue), Heroin (orange), and Oxycodone (green). From the chart, it is evident that Fentanyl is the leading cause of opioid overdose deaths across all regions. Regions 4, 5, and 3 have the highest total deaths, with Region 4 (Southeastern U.S.) showing the most fatalities overall. This chart underscores the regional disparities in opioid-related fatalities, highlighting the devastating impact of Fentanyl in every region and the presence of Heroin and Oxycodone as additional contributors in some areas. The regions are defined as follows:

Region 1: Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, and Vermont

Region 2: New Jersey and New York

Region 3: Delaware, District of Columbia, Maryland, Pennsylvania, Virginia, and West Virginia

Region 4: Alabama, Florida, Georgia, Kentucky, Mississippi, North Carolina, South Carolina, and Tennessee

Region 5: Illinois, Indiana, Michigan, Minnesota, Ohio, and Wisconsin

Region 6: Arkansas, Louisiana, New Mexico, Oklahoma, and Texas

Region 7: Iowa, Kansas, Missouri, and Nebraska

Region 8: Colorado, Montana, North Dakota, South Dakota, Utah, and Wyoming

Region 9: Arizona, California, Hawaii, and Nevada

Region 10: Alaska, Idaho, Oregon, and Washington

Dataset #3: Exploring the Correlation of Opioid Use to Patient Demographics

Source: https://www.samhsa.gov/data/data-we-collect/nsduh-national-survey-drug-use-and-health/state-releases
Summary of data and research questions it may address: This data explores patient demographics, including mental health status and substance abuse status. Is opioid use correlated to patient demographics such as income, mental health status, and education?
Collection method: Web export contained 26 different HTML files, each with 40+ tables on each page detailing different demographic information for each drug/condition/etc. Each table of interest had to be copied into an Excel file for preprocessing, and then cleaning could take place.
Cleaning and preprocessing:
- Link to Cleaning

Before Cleaning

Lots of unnamed columns & missing values

After Cleaning

Removed unnecessary columns, renamed columns to be descriptive.

Before Cleaning

Lots of unnamed columns, missing values, the rows were not helpful in the way they were sorted.

After cleaning

Selected rows only related to demographic information, renamed all columns to be descriptive, converted all columns to be numeric type.

Handling issues and noise: Removed rows that did not contain relevant information for demographics. If no values were supplied, a 0 would be filled in its place. There were only 2 instances of this needing to happen. It was small and did not impact the data analysis portion.

Understanding the data: This data was a piecemeal approach to pulling out the exact statistics our project is interested in. It required some massaging to get it into the correct data frame format and required some changing of data types; all items were data typed as objects to start. This data was from a 2022 and 2023 national survey about mental health and substance abuse across the United States to gather a full picture of how those in the US are struggling with addiction.
Statistical analysis: Demographic data will be used for this portion of the analysis. From the summary statistics, the mean for Mental problems in 2022 was 19.1, and in 2023, it was 21.59. Respectively, the mean substance problems for 2022 are 11.12, and for 2023, they are 10.94.

Figure 8. From the box and whisker plot summaries of the data, it is pretty evident just how skewed mental problems in 2023 are. The 50% quartile is significantly higher than the mean, indicating a negatively skewed data set. Additionally, the 50% quartiles for substance problems are 12.4 and 12.9. They're almost identical between the two years. Slightly higher than the mean, also meaning a skewed negative data set, but not nearly as bad as the mental problems.

- Data quality assessment: Overall, the quality of the data was strong! There were not a lot of null values, and there was plenty of documentation supporting what each table or source showed in each variation of the tables. The one gripe about this data set is that there are multiple categories of data for demographics in rows and columns; it was a bit of a pain to untangle. The data is skewed negatively for all years in the data set.
- Visualizations:

Figure 9. This visualization shows the percentage of mental health problems as it changed over time between 2022 and 2023 between demographic groups. From this visualization, it is apparent that all but Asian demographic groups saw an increase in mental health problems between 2022 and 2023. The highest spike in mental health problems was in American Indians and Alaska Natives.

Figure 10. This visualization shows the percentage of substance abuse problems as it changed over time between 2022 and 2023 between demographic groups. From this visualization, it is apparent that all but Native Hawaiian and Other Pacific Islanders demographic groups saw an increase in mental health problems between 2022 and 2023. The highest spike in substance abuse problems was in the Two or More Races demographic group.

Figure 11. This visualization details the reasons individuals would not seek mental health treatment during the year 2023. The highest reason is that they thought they should have been able to handle their health, emotions, and behaviors on their own. This is followed by worrying about what other people would think or say if they got treatment. This is a very common stigma in America that going to get help is frowned upon or you are seen as less than. It is not at all surprising to see these as the top two reasons for not seeking treatment.

Dataset #4: Exploring the Correlation between Overdose Rates and Access to Healthcare

Source: https://www.cdc.gov/mmwr/volumes/70/wr/mm7015a1.htm
Summary of data and research questions it may address: This data explores the correlation between overdose rates and access to healthcare.
Collection method: Web scrapping CDC government data using BeautifulSoup
Cleaning and preprocessing:
- Link to cleaning

Before Cleaning

Note NA values in first row

After Cleaning

Dropped first row

Understanding the data: Once scraped using BeautifulSoup, the appropriate columns were transformed from objects to floating points for analysis.

Statistical analysis:

Visualizations:

Figure 12. This is a bubble plot that helps visualize values in a 2D type of scatter plot. This plot shows us the jurisdiction (the state), the value of fatal opioid overdoses costs in healthcare, and the value of statistical life lost. This graph demonstrates a trend that with more health care spent (higher on the y-axis), the state tends to have a higher value of life lost. This shows a great proportion between these and a correlation to these two variables in the spending and the importance of the people who overdose due to opioids and were fatal.

Figure 13. The above visualization is a horizontal bar chart showing the proportion of each state's contribution to the total amount of cases of Opioid Use Disorder. We can see that California, Texas, and Florida are the most significant contributors to the issue. This is interesting, as the populations differ. Still, they all appear to contribute similar amounts to the total, even with states with a ton of cities where people tend to associate drug use issues with lots of urban populations or blue states such as Illinois or California.

Page updated

Google Sites

Report abuse