Kolej Mohamad Rashid was built on the former Serdang estate land by Macmillan and Company, and was initially known as the First Residential College. It consisted of five residential blocks named after several types of hardwood trees: Resak, Kempas, Merbau, Cengal, and Kemuning.

As the oldest college on campus and one that developed alongside the College of Agriculture and later Universiti Pertanian Malaysia, it is no surprise that this college is often associated with several notable figures. Among them are Dr. Haji Mohamad Rashid Ahmad (Cikgu Rashid), Tan Sri Dato' Dr. Nayan Ariffin, Tan Sri Ani Arope, Tan Sri Rashdan Baba, Prof. Dato' Dr. Ir. Mohd Zohardie Bardaie, Prof. Dr. Rahim Md. Sail, Assoc. Prof. Dato' Dr. Idris Abdol, Prof. Dr. Ghizan Salleh, and many other prominent personalities from the First Residential College during that time.

In 1975, part of the college was repurposed to house a clinic, the Agricultural College Students' Association Office, and the Student Union Office (CAMSU). Initially, the majority of residents at KMR were matriculation students, making up about 95 percent of its occupants.

On 14 August 1988, the First Residential College was officially renamed Cikgu Rashid Residential College. When the Collegiate System was introduced at UPM in early 1995, the college name was updated to Kolej Cikgu Rashid.

In May 1995, in line with the university’s corporate image, Kolej Cikgu Rashid was renamed Kolej Mohamad Rashid, and has since been popularly known as KMR among the university community.