Portrait Mode - As named, Portrait mode works best for smartphone displays with social media and communication platforms where the focus is on the face. This orientation allows the face, or full body to occupy as much of the frame as possible, ensuring efficient communication. It's also effective for compositions where the interest in the scene is vertical or along the Z axis.
Aesthetically we associate the look with social media or video calls, so if this is the intended look, if you are producing the video for a platform which is optimised for Portrait mode, or if you intend your audience to primarily view your video on mobile devices, then this may be the way to go.
Portrait footage will look both very small and very informal on traditional screens, so if you're intended method of distribution includes these, or you're doing a more formal presentation, shoot your videos in landscape.
Landscape Mode - As named, Landscape mode is optimised to display environments, due to the way our eyes perceive the space around us. Footage shot in landscape tends to subconsciously immerse your audience in the place, as well as connecting to any subjects. It is also optimised for displaying on TV screens, Monitors, Computers, Projection, etc which use this orientation and allows for the most amount of space to be filled.
It is the standard for almost all television and film production and so is advisable for the majority of video projects.
It can however look very small on mobile devices if viewers don't rotate their screen, so if this is your primary method of video distribution, consider portrait.