Exploring the Opioid Crisis

Dataset 1: Analyzing Opioid Prescription Rates by State and Provider Type

- Link to code
- Model 1: K-Means Clustering
  - Why the model was chosen: K-Means was chosen to explore the relationship between total opioid claims, opioid prescription rates (percentage of opioid prescriptions out of total prescriptions), and total opioid costs. This unsupervised method allows for an understanding of patterns in provider behavior and can reveal different risk profiles or areas in need of intervention. For example, providers who have high costs, high claims, and high rate points should be identified. On the other hand, some prescribers may have a low rate but high cost, indicating differences in the opioid market and how this may be affecting patients.
  - Model assumptions: K-Means requires scaled, quantitative data because it is based on Euclidean distance. The model is inherently exploratory, with the assumption or hope that the data is naturally clustered. K, the number of clusters, is also an informed assumption based on the elbow plot.
  - Data preparation:

Data Before Transformation for K-Means

Raw data containing many mixed feautures

Data Before Transformation for K-Means

Scaled data, narrowed to 3 features

- - Hyperparameter tuning: An elbow plot is used to determine the optimal number of clusters, revealing that 3 clusters are optimal. A random state is defined to allow for the reproducibility of results. Finally, n_init is set to 10, instructing the model to randomly initialize centroids 10 times and choose the optimal result.

Challenges faced and solutions: Initial attempts at this model revealed outliers that skewed centroids and created uninterpretable clusters. Eliminating data points outside of 1.5 x IQR helped significantly.

Figure 1. A) Elbow plot investigating the optimal number of clusters for K-Means to explore patterns in opioid prescription rates by provider. B) K-Means clustering with three scaled features: total opioid claims, total opioid cost, and opioid prescription rates of a random sample of 58,000 providers in the US.

Performance Evaluation: K-Means clustering of this dataset did not reveal any clear patterns. While there are 3 clusters that correlate to varying levels of opioid claims, cost, and rates, there is no clear separation between clusters. This is a common outcome of K-Means, but this exercise was helpful nonetheless to explore the distribution of provider behaviors. For example, there are providers in cluster 1 (blue) that are clearly frequent prescribers with high claims, costs, and a high percentage of opioid prescriptions. Classification will be performed next to determine if the provider's state and specialty can be predicted based on their prescribing patterns.

- Model 2: Decision Tree
  - Why the model was chosen: This model's goal is to predict a prescriber's credentials (MD, DO, PA, etc) based on their opioid prescription patterns and geographic location. A decision tree is a suitable classification technique for this because decision trees can handle categorical features, in this case, the state code and rural-urban area code. Classification of a prescriber's credentials is interesting because it may reveal patterns in prescribing techniques that are rooted in a prescriber's education. It is helpful to understand how different types of credentials are educated and how their views on opioids may impact patients.
  - Data preparation: Categorical features were encoded, and data was split into training and testing data. The label, the prescriber credential, was removed and saved for accurate measurements.

Data Before Transformation for Decision Tree

Categorical features, varying formats of provider credentials

Data After Transformation for Decision Tree

Encoded features, all numeric data

- - Hyperparameter tuning: Model accuracy was improved by 2% by setting the maximum depth to 15 and the minimum sample split to 10. This prevented the tree from growing too deep and reduced the number of splits to smooth the model. The data was also thoroughly prepared and balanced to optimize model performance by ensuring consistent labeling and balancing.

Challenges faced and solutions: Initial inspection of the data revealed a wide variety in the format of credential data input. For example, MDs were labeled MD, M.D., physician, and many more. Thus, it was necessary to remap all credentials to specific formats to allow for proper labeling and modelings. The data was also imbalanced, with a large majority of providers being MDs and some very niche provider types. Only the top 12 most popular provider types were kept, and this data was balanced by oversampling and undersampling.

Figure 3. Confusion matrix evaluating the accuracy of a decision tree that predicts the credentials of a provider based on opioid prescribing habits and location. Labels are: 0: 'APRN', 1: 'ARNP', 2: 'CRNP', 3: 'DDS', 4: 'DMD', 5: 'DO', 6: 'DPM', 7: 'FNP', 8: 'MD', 9: 'NP', 10: 'OD', 11: 'PA-C', 12: 'PharmD'.

Performance Evaluation: The accuracy of this decision tree is 76%. In the context of this research question, this is a strong performance. Very high accuracy is not needed, but a guideline to understand general prescribing behaviors within each credential is useful. Furthermore, many of the errors come from closely related specialties such as DMD and DDS, the two types of dental degrees. Therefore, there may not be major differences between DMD and DDS programs, but dentists, in general, have a distinct prescribing pattern. This information can be used to inform how prescribers are behaving and being educated about opioids.

Dataset #2: Analyzing Drug Overdose Rates by Location and Drug Type in 2019

- Link to code
- Model 1: K-Means Clustering
  - Why the model was chosen: K-means clustering was chosen for its simplicity in identifying groups of regions based on drug overdose patterns. The dataset contains overdose death counts by region and drug type, which is fairly high-dimensional. Since there aren't any labels or categories to predict, unsupervised learning was used to identify patterns in the data.
  - Model assumptions: This model assumes clusters are equal-sized. It is sensitive to feature scale, so PCA was used to reduce dimensions and reveal groupings.
  - Hyperparameter tuning: The number of clusters, k=4, was determined by using silhouette score optimization and gave the best clustering quality.
  - Challenges: Some regions had very few overdose entries, so these regions were filtered out to create meaningful clustering.
  - Performance evaluation: The silhouette score for this model is 0.329, and the Davies-Bouldin Index is 0.841. These scores suggest the clusters are fairly compact and separated, but the results aren't perfect. However, they are strong enough to justify further exploration.

Figure 4. K-Means Clustering of Regions Based on Drug Overdose Patterns: This scatter plot shows how different regions group together based on their overdose profiles. The plot reveals clear separations between some clusters, suggesting that certain regions share similar patterns in drug overdose activity while others are more distinct.

- Model 2: Apriori Frequent Pattern Mining
  - Why the model was chosen: Apriori was chosen to discover common drugs involved in overdose deaths across different regions and timeframes. The dataset includes lists of drugs involved in overdoses grouped by categorical values like region, year, and month, which works best with frequent pattern mining.
  - Model assumptions: This model assumes that the data is set up like transactions, which is why the drug lists were converted into a format where each row acts like a basket of drugs. A higher threshold was used to make sure the patterns were not noisy.
  - Hyperparameter tuning: Minimum support was changed to 0.1 after seeing the lack of results at higher thresholds. This gave us enough frequent item sets to work with.
  - Challenges: There was high dimensionality in the dataset, so PCA was used to reduce dimensionality and create meaningful patterns.
  - Performance evaluation: The model worked very well with support, confidence, and lift all around 1. This suggests there are strong co-occurrences of drugs, but the results revealed obvious patterns.

Figure 5. Association Rules from Apriori Frequent Pattern Mining: This table shows the top association rules generated from the Apriori algorithm, showing strong co-occurrence patterns among drugs found in overdose cases. The antecedent is a drug, and the consequent is the drug(s) also likely present.

Dataset #3: Cost components of opioid use disorder and fatal opioid overdose, by jurisdiction

- Link to code
- Model 1: Multiple Linear Regression Model
  - Why was the model chosen: Multiple Linear Regression is significant in predicting values with many coefficients. This model is used to indicate how many deaths would occur because numerous factors lead to increasing death rates, from spending on health care to population. This was an excellent choice for a model, as it can also indicate which variables are more important based on their coefficient values. It is excellent for its simplicity and time complexity, which are ideal for a large dataset like this.
  - Model assumptions: The model assumes a linear relationship between the variables, that they are independent, and that there is no high correlation between the variables in predicting the model.
  - Hyperparameter tuning: Coefficients were tuned throughout model preparation to minimize the root mean squared error (RMSE). The model didn't improve with fewer coefficients and didn't seem to overfit when testing. Thus, all variables were kept. Other hyperparameter tuning was performed during data preprocessing in Data Exploration for the dataset used.
  - Challenges: One challenge was determining the best number of coefficients due to the similarity in some of the models' RMSEs. All variables were used as the RMSE didn't improve with fewer. There were some outliers with costs and null values in the data, but these were filtered out during the Data Exploration phase.
  - Model Evaluation: For this model, performance evaluation was based on the root mean squared error (RMSE) of the model. The RMSE was rounded to 0.142. For this data set, the model performs quite well, as its prediction range spans from 2000 to 165000, which is a substantial range with a relatively small error.

Figure 6. The above is the Multiple Linear Regression model using the highest value coefficients to demonstrate the variables that have the highest effect on the prediction of estimated cases of Opioid Use Disorder (OUD). This does show a strong linear correlation between the two with the highest coefficient, which is what the model assumes.

Model 2: K-Means Clustering

- Why was the model chosen: This model was chosen to group states and identify which states are similar, given the millions of dollars spent on addressing this crisis. K-means clustering is effective for grouping data, as it is based on a centroid, which is the mean of the cluster. It is also a low complexity, which saves time as the data adjusts. It is a great effort to group similar states, which is excellent in categorizing states by their similarity on a major issue, to identify which states have more significant problems than others. This can help with the classification of states.
- Model assumptions: This model assumes that all clusters have a similar variance, circular clusters with similar radius, and similar cluster sizes.
- Hyperparameter tuning: For this dataset, outliers were removed during data preprocessing by the Data Exploration team. Additionally, this model is less complex with two variables, so the data frame was edited to facilitate the calculation of the sum of the cost to cluster these more easily. Without the sum and using some variables, the silhouette score wasn't as strong. Other hyperparameter tuning was performed during data preprocessing in Data Exploration for the dataset used.
- Challenges: This model was complex to create due to the large number of variables. To navigate this, variables for the sums of the Opioid Use Disorder spending and the Fatal Opioid Overdose spending for their respective columns. Also, finding the right k for this wasn't easy, as the silhouette score wasn't changing much. However, it is usually a good sign when it is .5 or higher, which started after six, and six was the "elbow" of the k-means plot by its score.
- Model Evaluation: The silhouette score for this model was rounded to .523. It is good practice to classify a clustering model as having good clustering if it is above .5. Since this is above .5, this was good clustering. However, if there were all of the 50 states instead of 37 with the District of Columbia, fewer outliers would've been removed during preprocessing, which would've improved the clustering below. Although these outliers affect this model, this model gives a good overall representation of certain states and groups they should fall into.

Figure 7. Above is the clustering of the data by its total costs of OUDs and FOOs. It has three bigger clusters with three smaller ones. These smaller ones were close to outliers but were not removed during preprocessing. This is why six was a good number, as it took into account these outliers for better clusters. The larger clusters show a good correlation despite what the almost outliers show,

Dataset #4: Per capita cost of opioid use and overdose

- Link to code
- Model 1: SVM
  - Why was the model chosen: Suited for high-dimensional data where the separation between classes is not clear. The goal of SVM is to find a hyperplane that separates the classes by maximizing the distance between the hyperplane and the data nearest to it. It is a versatile choice for non-linearly related data.
  - Model assumptions: SVM assumes that data is linearly separable, meaning there is a hyperplane that can separate the classes. It aims to maximize the margin between classes for improved performance.
  - Hyperparameter tuning: There are different kernel types that can be used for SVM, linear, polynomial, and radial basis functions (RBF). For this, all three kernels will be run to see which performs the best.
  - Challenges: It is very sensitive to outliers and can drastically change the decision boundary if not treated with care. It struggles to handle large datasets due to its computationally expensive nature; additionally, kernel choice may affect performance. The data set was pretty small, so it would be interesting to see how the hyperplane would change with more records for each state over time.
  - Model Evaluation: Overall, this model did shockingly well! From the initial view of the scatter plot, it looked like it would be a tight squeeze to fit a hyperplane in between; however, it was able to fit! All three models were calculated rather quickly; the only model that didn't achieve peak performance in this case was the polynomial version.

Data Before

Overall, the data was in pretty decent shape to start. Columns and data were in a desirable format.

Data After

Removed all data unless it pertained to opioid overdose. Added a boolean column if opioid fatalities > 500

Linear Kernel

Accuracy: 100%

Precision: 100%

Recall: 100%

F1: 100%

Polynomial Kernel

Accuracy: 87.5%

Precision: 100%

Recall: 80%

F1: 88.9%

RBF Kernel

Accuracy: 100%

Precision: 100%

Recall: 100%

F1: 100%

Before Decision Boundary

After Decision Boundary

From the above figures, it is clear that there exists a linear separation between fatalities below 500 and fatalities per capita. Most likely due to the exponentially increasing medical costs, the more overdoses happen.

Model 2: Hierarchical Clustering
- Why the model was chosen: The model can handle various types of data attributes. Clusters do not need to have a distinct shape. It can handle noisy data. With this data set, there were some outliers, and it felt like a great candidate for hierarchical clustering.
- Model assumptions: It assumes the data originates from a specific model, and it is a hypothesized cluster model of best fit.
- Hyperparameter tuning: I tried for k=3, 4, and 5 for cluster assignment to see how the clusters changed visually. I used an agglomerative approach to start with all points individually merging until one final point was achieved.
- Challenges: Overall, the model performed very well with each clustering assignment; the clusters are very distinct and easy to see. The one challenge is that the data set is relatively small, so it would be interesting to see how adding additional data would change the assignments.
- Model evaluation: Overall, the k=5 visually and logically makes the most sense as the cluster assignment that should be used. Each cluster is pretty contained vertically in intervals of 10000, so having 5 intervals from each range of 10000 makes the most sense to capture all of the data for that cluster.

Data Before

Overall, the data was in pretty decent shape to start. Columns and data were in a desirable format.

Data After

Removed all data unless it pertained to opioid overdose.

K = 3 Clusters

K = 4 Clusters

K = 5 Clusters

Page updated

Google Sites

Report abuse