Capitalize in stratigraphic terms and as part of a proper name:
Upper Silurian
Upper Egypt
Upper Peninsula (Michigan)
Upper West Side (New York City)
Generally lowercase for rivers and states:
upper Mississippi River
upper New York State
See also lower, middle.