Wearing heavy armor lowers movement speed by 2% per piece.Â
Wearing a full set of heavy armor will lower incoming ranged damage by 15%.
Wearing any amount of heavy armor increases magicka costs for spells depending on the spells level, 20/40/60/80/100% extra cost for novice-master respectively. This penalty can be negated by having the appropriate perk for the spell level in the Heavy Armor tree, though costs will still be increased by an amount proportional to your effective mass.