It is said in family histories that the Harris family, while the Harris boys were young, lived on a plantation that had slaves. The Harris family was converted to the Church of Jesus Christ of Latter-day Saints in 1848. The family left Tennessee and went to Nauvoo, Illinois before joining the Brigham Young Company of 1848. They departed on June 5 and arrived in the Salt Lake Valley on Sep 24.
Alexander Harris married Harriet Ann Craner on June 21, 1855 in Salt Lake City, Utah They soon moved to Harriman, Utah. In 1860, William and his brothers Alex and Eli were in conversation when Alex suggested the three of them move to Cache Valley. "Boys, let's go to Cache Valley," said Alex, with the other boys saying "All right!" They moved to Richmond in the spring of 1860.