Daugherty played basketball at Charles D. Owen High School in Black Mountain, North Carolina, where he led the Warhorses to the 1982 state finals. Daugherty accepted a scholarship to play at the University of North Carolina under legendary college basketball coach Dean Smith.
Daugherty was one of the greatest big men ever to play at the University of North Carolina. He entered college as a 16-year-old freshman and was a two-time All-ACC first team selection, and a first team All-American in 1986. He was named to the ACC 50th Anniversary men's basketball team in 2002 and was inducted into the North Carolina Sports Hall of Fame. Daugherty averaged more than twenty points per game in his senior season.
Daugherty was taken as the first overall pick by the Cleveland Cavaliers in the 1986 NBA draft. Cleveland had obtained the rights to the first pick in a trade with the Philadelphia 76ers for Roy Hinson and cash.