Joe is an ex-boyfriend of Taylors from the Fearless era. Joe is the first well known and famous man/boy she dated when starting out her career. During the Fearless time and era, taylor was very open on her relationships and specifically hers with Joe as well- good and bad. Often, after they broke up, Taylor would diss him publically. They are now friends many years later, which is concluded through the song invisible string from folklore. "Cold was the steel of my ax to grind for the boys who broke my heart; now I send their babies presents." For context, Joe and his now wife had a child sometime before folklore was dropped, and it is said that Taylor sent them baby shower gifts.
Andrea Swift (Taylor’s mother) once said in an award show speech that Taylor wrote this song after she was told by her parents they didn’t like the guy she was dating/liked at the time
This song was a last minute addition to the album right before it’s release - and Taylor and Joe’s very public breakup also happened right before the release of the album
Lyrics in Mr. Perfectly Fine are parallels to lyrics in Forever & Always, making it appear like the songs are about the same person
Lyrics about Joe always having his hands in his pockets are in both this song and Last Kiss, from Speak Now, which is also allegedly about Joe
She refers to an ex’s new girl as an ‘actress’ in this song and we can refer that to Joe Jonas’ new girlfriend after Taylor, Camilla Belle. It seems that Taylor had quite the resentment for her because of her nasty and public breakup with Joe
This song has lyric parallels to songs in Fearless like You All Over Me, which both talk about a man always seeming to have his hands in his pockets
Holy Ground is part of the triangle of songs from her 2-4th album (Forever & Always, Last Kiss, and Holy Ground). This is a only just a theory