Research Highlights
- Video Compression and Reinforcement Learning (RL)
- Propose an RL-based coding unit split decision algorithm for HEVC/H.265.
- Formulate the coding unit split decision as an RL problem.
- Train an RL agent to predict the RD cost reduction associated with a CU split decision
- Results: The model shows the same level of RD cost prediction accuracy as the exhaustive search scheme.
- Propose an RL-based intra frame rate control algorithm for HEVC/H.265
- Formulate the Frame-level rate control as a sequential QP determination problem.
- Train an RL agent to determine QP for each CTU based on long-term RD benefits.
- Results: The model is able to perform comparably with the rate control scheme in HEVC/H.265 when trained only on few sequences and shows better subjective quality.
- Propose an RL-based inter frame rate control algorithm for HEVC/H.265.
- Extend the notion of the previous work to inter-frame rate control.
- Introduce the actor-critic method (DDPG) for fine-grained QP determination for each CTU.
- Results: The model shows 1-2% BD-rate savings relative to HEVC/H.265 rate control, with less bit rate errors.
- Propose an RL-based frame-level bit allocation algorithm for HEVC/H.265
- Formulate the frame-level bit allocation problem as an RL problem.
- Perform frame-level bit allocation by considering long-term impacts on GOP-level distortion.
- Implement actor-critic method for frame-level bit allocation.
- Results: The model shows 1-2% BD-rate savings relative to HEVC/H.265 rate control.
Significance: these represent pioneer attempts at applying RL to video encoder control.
- Domain Transfer
- Dash-cam Color Transfer (https://goo.gl/zfFMqe )
- Use deep learning tool for dash-cam training data augmentation.
- Implement cycle-GAN to deal with style transfer problem.
- 3D Reconstruction
- Interior Design (Independent Research/Undergraduate Research)
- Implement a vanishing point based 3D reconstruction
- Use extrapolation to calculate the each depth of corner in 3D space
