The film is set in 1931 during the Great Depression with a voiceover from Michael Sullivan, Jr., speaking about his father. Michael Sullivan, Sr. was orphaned and then raised by Irish Mob boss John Rooney in Rock Island, Illinois; he is now Rooney's most fearsome enforcer, unbeknownst to his own children. Rooney has come to love Sullivan more than his own biological son, the rash and unpredictable Connor.
Speaking at his brother's wake, Rooney's associate Finn McGovern insinuates that Rooney is responsible for his brother's death. Rooney sends Connor and Sullivan to talk with McGovern, while 12-year-old Michael watches through a hole in the wall, after hiding in the back of the family car. McGovern steadfastly denies that his brother stole anything from the mob before implying Connor was responsible, and Connor shoots him on the spot before Sullivan shoots the other witnesses. Michael is detected and sworn to secrecy.
The next day, Rooney visits the house to intimidate the boy, and Michael soon begins misbehaving at school. At a meeting with his mob associates, Rooney pointedly humiliates Connor after he halfheartedly apologizes for McGovern's murder. Then he sends Sullivan to collect a debt from a speakeasy owner Tony Calvino. Connor, jealous and afraid, sends a letter with Sullivan for Calvino. Calvino reads it then reaches for his revolver, but Sullivan kills both Calvino and his bodyguard. The letter reads "Kill Sullivan and all debts are paid," and he rushes home. When he arrives he discovers Connor has already murdered his wife, Annie, and their younger son, Peter, but not Michael, who had arrived home late from detention.
Sullivan and Michael flee Rock Island and head to Chicago to meet Frank Nitti. He offers to work for the Chicago Outfit in exchange for permission to kill Connor, but Nitti rejects the offer. Rooney, listening next door with Connor, reluctantly allows Nitti to dispatch freelance killer Harlen Maguire, who doubles as a crime scene photographer, to kill Sullivan. Nitti suggests Michael must be killed as well to prevent him seeking revenge in the future, but Rooney refuses. However, Nitti secretly orders Maguire to kill Michael as well. Maguire tracks Sullivan to a roadside diner. The two meet and have a seemingly casual conversation while Michael hides in the car. Sensing the danger, Sullivan escapes the diner through the bathroom window and slashes Maguire's tires before driving away.
Sullivan now plots to force the Outfit to give up Connor by robbing the banks that hold its money. He teaches Michael to drive in order to become their getaway driver. Sullivan is impeded when the mob withdraws its money, so he visits Rooney's accountant Alexander Rance. Rance stalls Sullivan until Maguire enters with a shotgun. Rance is killed in the crossfire before Sullivan wounds Maguire and escapes with Rooney's ledgers. However, during the escape, Maguire shoots Sullivan in the shoulder. Michael drives them to a farm after Sullivan collapses from his wound, where an elderly childless couple helps him recover. Sullivan's bond with his son grows and Michael comes to realize his father loves him.
The ledgers reveal that Connor has been embezzling from his father for years, using the names of dead men including McGovern. Believing Rooney will call off the hit on him if he knows the truth, Sullivan gifts the couple a chunk of the stolen cash and heads back to Chicago. Sullivan confronts Rooney at Mass, learning Rooney already knows about Connor and expects he will be killed – if not by Sullivan, then by Nitti's men once Rooney is dead. He still refuses to give up his son and urges Sullivan to flee with Michael and ensure he becomes a better man than either of them.
Later one night, cloaked by darkness and rain, Sullivan ambushes and kills Rooney's bodyguards with his Tommy Gun and walks up to Rooney. As Rooney mutters "I'm glad it's you", Sullivan reluctantly pulls the trigger. Having no further reasons to protect Connor now that Rooney is dead, Nitti reveals Connor's location after Sullivan promises the feud ends with his death. Sullivan goes to the hotel where Connor is hiding and kills him in the bathtub.
Sullivan takes his son to his Aunt Sarah's beach house in Perdition, on the shore of Lake Michigan, where he is ambushed and shot by Maguire inside the house as his son Michael Jr. stands on the beach. As Maguire photographs the dying Sullivan, Michael Jr. appears and points a gun at Maguire but cannot bring himself to pull the trigger. As Maguire beckons to Michael Jr. to give him the gun, Sullivan fatally shoots Maguire in the back. Michael Jr. tells his father that he couldn't pull the trigger, and Michael Sr. nods and smiles as he dies in his arms.
Michael says his father's fear was that he would follow the same road, and that he has not held a gun since his father died. Michael drives the car back to the farm, saying he grew up there, and now when he is asked if his father was a good man, he just tells them, "He was my father."
Tom Hanks as Michael Sullivan
Tyler Hoechlin as Michael Sullivan Jr.
Paul Newman as John Rooney (based on John Patrick Looney)
Jude Law as Harlen Maguire
Daniel Craig as Connor Rooney
Stanley Tucci as Frank Nitti
Jennifer Jason Leigh as Annie Sullivan
Liam Aiken as Peter Sullivan
Dylan Baker as Alexander Rance
Ciarán Hinds as Finn McGovern
Doug Spinuzza as Calvino
David Darlow as Jack Kelly
Kerry Rossall as Rooney's Henchman (uncredited)
Anthony LaPaglia as Al Capone (uncredited)
Kevin Chamberlin as Frank the Bouncer
Harry Groener as Mr. McDougal
JoBe Cerny as Banker