As the use of machine learning (ML) becomes more prevalent in many practical software systems, maintaining the long-term availability of ML systems has emerged as a new critical challenge. In ML system operations, ML models continuously confront the risk of performance degradation due to dataset shifts or security attacks. As the system performance and quality may deteriorate over time, implementing proper mitigation methods to ensure the high availability of ML systems is essential. This talk introduces recent approaches to enhance the reliability and availability of ML systems through N-version configuration, software rejuvenation, and ML model updates. While these methods are based on well-established techniques, several unique perspectives, specifically tailored for ML-based systems, are discussed. The talk will also highlight future challenges for high-availability ML systems, which may consist of multiple dependent ML and AI models.