Einstein said: Time is relative to the observer (which hardly anybody can comprehend). But, even before Einstein, time was relative to the position of the Sun. And noon was defined as when the Sun was the highest in the sky. This was just fine when people were walking, riding horses, or traveling on slow moving boats. That all changed when the railroads invented speed and precise schedules.
Prior to the railroad, the exact time was irrelevant. However, railroads which operate more than one train, need precise schedules based on a common time for all the trains operating on a particular section of track. So, the early railroads began issuing schedules based on a single location which was oftentimes the railroad's headquarter city (meaning: each railroad company had their own time). This time became known as "Railroad Time", and every city east or west of the railroad's headquarter city would have a slightly different "Sun relative time" (aka Solar Time); which eventually became known as "Local Time". This was just fine until another railroad came to town; and now you've got two Railroad Times (and the Local Time). And, this wasn't so bad until the railroads started sharing track. Railroads quickly realized it was best if the operating crews of heavy trains - operating at high speeds - traveling in opposite directions - usually on single track - all could agree on exactly what time it was.
From this realization, came regional standardized times for all schedules, and then the concept of chopping the country up into zones, which became "Time Zones". The English railroads, having a twenty-year head-start - and a much higher density of trains, adopted this idea first. The US and Canadian railroads converted to a Standard Railroad Time on Nov 18, 1883. The Canadian Government adopted the concept two years later. But, in the USA, the usual insane factions fought and argued about it for decades until everybody finally agreed (or they started fighting about something else and forgot what the "time argument was about exactly") and it was adopted formally in 1918, 35 years later.