The average treatment effect is equal to the difference in potential outcomes for the average person or unit being considered.
Due to something of a fluke of mathematics, we may be able to observe the average treatment effect. While we cannot, in general, observe the difference in potential outcomes for any particular person, we may be able to observe the difference in potential outcome for the average person. That is we may be able to observe the difference for the person who has the average difference in potential outcomes.
It turns out that the average of potential outcomes for treatment A less the average potential of potential outcomes for treatment B is equal to the average of potential outcomes for treatment A less than potential outcomes for treatment B. The difference in averages is equal to the average of the differences.
Note however, it is the average of potential outcomes for treatment A, not necessarily the average of observed outcomes for treatment A. What is the difference? The average of potential outcomes is the average of outcomes if EVERYONE received treatment A. The average of the observed outcomes is just that. It is the average for those people who happen to get treatment A. Unless I know why someone was assigned to treatment A, I cannot determine whether the two are the same.
One way to measure the average treatment effect is to have a randomized controlled trial. Such a trial randomly selects patients from the population of interest and assigns them to a particular treatment. If we have a large enough set of patients assigned to a particular treatment and we observe the outcomes for those patients then we have a reasonable estimate of the average potential outcomes for that treatment. If we have reasonable estimates of the average of the potential outcomes for the different treatments we have an estimate of the average treatment effect.