Once the temperature drops to the dew point, the temperature will continue falling, but more slowly than before.
As the temperature drops below the original dew point temperature, some of the water vapor begins to condense out of the atmosphere and form dew or fog. This reduces the amount of water vapor in the air, causing the dew point to drop. As the night continues, the temperature and dew point will continue to drop in tandem so that the air remains saturated and the amount of dew or fog continues to increase.
But the transformation of water vapor into liquid water releases energy. Before, the loss of energy into space simply caused the land and air to cool. This cooling represents a loss of sensible heat, because the temperature changed can be felt, or sensed. Once condensation begins, though, the change from gas to liquid is connected to part of that energy loss. The energy stored in gas form and lost when the gas became liquid is called latent heat, because it was latent, or hidden, within the gas phase of the water. The more latent heat is released by condensation, the less sensible heat is lost, so the temperature doesn't cool down as much.
The same sort of thing happens when clouds form. As an air parcel ascends, it expands and cools. Eventually, it reaches saturation, whereupon cloud starts forming. What happens to the air parcel as it continues to rise?
By the way, the temperature of the air parcel doesn't depend much on how fast it's rising. When it reaches a given altitude, it will have a particular temperature no matter how long it took to get there.