During production petroleum oil is used to form the plastic fibers that are weaved into polyester clothing. This process uses a lot of energy from thermal energy and mechanical energy needed to form the garments themselves.

This hot plastic releases volatile organic chemicals (VOCs) into the atmosphere which are responsible for expediting the greenhouse affects and leading to more drastic global climate change.

At the end of polyester clothing's short life, it is sent to a landfill where it does not decompose measurably. What it does to however is leech VOCs into the atmosphere and hydrosphere, in addition to leeching microplastics into the watershed. These things directly impact the health of your local community. 

Locally, the Millersville landfill does a good job at preventing excess leeching and release of gas, but it is not completely avoidable.