A cave (or cavern) is a naturally occurring area or space under the surface of the Earth, or underground. Caves are often a system of interconnected passageways created by the weathering of rock. If you are exploring caves, either for fun or for actual scientific exploration of a cave system is called caving, potholing or spelunking. The scientific study of caves is called Speleology and the formation and development of caves called speleogenesis.
Caves can be formed by many geological processes including chemical actions, erosion from water, tectonic and volcanic forces, microorganisms, and pressure. (Those are some big words, huh?!?)
The most common type of cave formation are solutional caves which are formed by natural acids in groundwater dissolving soluble rock such as limestone (or sometimes chalk, dolomite, marble or gypsum).
The maximum depth a cave system can reach underground before the pressure of overlying rocks becomes too great is estimated to be about 3,000 m (9,800 ft).