The Receiver Operating Characteristic (ROC) curve is a graphical tool used to evaluate the performance of a diagnostic test or classification model. It is constructed using the sensitivity (true positive rate) and specificity (true negative rate) calculated from a dataset containing both individuals with and without the disease (or positive and negative cases). The AUC systematically counts how many 'duels' between a sick patient and a healthy patient the test manages to score higher for the sick patient — and expresses this as a proportion from 0 to 1. It means, the first step will be calculate and order the predict probabilities. For each cut off possible, there will be calculate the sensitivity and specificity. Then, the probabilities will be compared against the real results. The ROC curve is also commonly used to compare the performance of two or more classification models. 

The graph plots:

• Y-axis: Sensitivity (True Positive Rate)

• X-axis: 1 − Specificity (False Positive Rate)

Both axes range from 0 to 1.

Each point on the ROC curve represents a different decision threshold (cutoff value) used to classify observations as positive or negative. As the threshold changes, the sensitivity and specificity of the model also change.

The diagonal line represents the performance of a model that classifies observations randomly:

• Above the diagonal: Performance is better than random chance.

• Below the diagonal: Performance is worse than random chance.

An ideal model has a ROC curve that rises quickly toward the upper-left corner, indicating high sensitivity and high specificity. Therefore, when comparing models, the one with the curve closest to the upper-left corner generally provides the best overall classification performance.

The Area Under the Curve (AUC) is the area beneath the ROC curve and provides a single measure of a model's overall ability to discriminate between positive and negative cases. Like the ROC curve, the AUC ranges from 0 to 1.

• AUC = 0.5: The model performs no better than random chance.

• AUC close to 1.0: The model has excellent discriminatory ability, approaching perfect classification.

• AUC = 1.0: The model perfectly distinguishes between positive and negative cases.

A practical interpretation of the AUC is that it represents the probability that the model will assign a higher predicted score to a randomly selected positive case than to a randomly selected negative case. For example, an AUC of 0.73 indicates a 73% probability that the model will correctly rank a positive case above a negative one.

In this example (image), the Youden index cutoff helps classify a test result as either positive or negative for the condition. In a screening scenario, the priority is high sensitivity to avoid missing cases, whereas diagnostic confirmation requires higher specificity to ensure accurate rule-in.

Data rarely arrives in a format that is ready for analysis. As Hadley Wickham aptly notes, “Data is often organized to facilitate some goal other than analysis. For example, it’s common for data to be structured to make data entry, not analysis, easy.” This reality explains why data preparation is not just a preliminary step, but a core part of any analytical workflow.

Preparing data is essential and it is often the most time-consuming stage of an analysis. Yet, the quality of insights you generate depends directly on how well this step is done. Clean, well-structured data allows analysts to explore patterns, build reliable models, and communicate results effectively.

Below are key concepts that every data professional should keep in mind when preparing data, especially when working in R with the tidyverse.

Choosing the Right Data Shape: Long vs. Wide

One of the most common data preparation tasks is reshaping datasets.

• pivot_wider(names_from = , values_from = )
  Converts data from long format to wide format. This is useful for reporting, tables, or when variables need to appear as separate columns.

• pivot_longer(names_to = , values_to = )
  Converts data from wide format to long format. This format is preferred for most statistical analyses and is especially important for visualization.

👉 Best practice: When creating plots, always work with long-format data. Visualization libraries are designed to work more naturally and consistently with this structure.

Cleaning Values Embedded in Text

Real-world datasets often mix numbers and text in the same column. When this happens:

• parse_number()
  Use this function inside mutate() to extract numeric values from character strings.
  This ensures that values intended for calculations are correctly interpreted as numbers, not text.

Understanding the Power of .value

A more advanced, but extremely powerful, concept in data reshaping is .value.

• .value is a special sentinel used inside the names_to argument of pivot_longer().

• It is particularly useful when column names contain multiple pieces of information.

• Instead of storing all values in a single column, .value allows part of the column name to remain as actual column headers in the long dataset.

In practice, this means you can reshape complex “wide” datasets while preserving meaningful structure without unnecessary post-processing.

Final Thoughts

Data preparation is not glamorous, but it is foundational. Understanding how to pivot data correctly, extract clean numeric values, and use advanced tools like .value can dramatically improve both efficiency and analytical clarity.

In a world increasingly driven by data, knowing how to prepare data well is just as important as knowing how to analyze it.

📚 Source: R for Data Science