How Mail Servers Work?

Mail Server is a software running on server machine. It constantly listens to mail requests through the ports. Normally there are two servers SMTP Server and POP3/IMAP server. SMTP server manages delivery of emails whereas POP3/IMAP server manages receiving of emails.

SMTP (Simple Mail Transfer Protocol)
Whenever you send a piece of e-mail, your e-mail client interacts with the SMTP server to handle the sending. The SMTP server on your host may have conversations with other SMTP servers to deliver the e-mail. The SMTP server listens on well-known port number 25.

The POP3 and IMAP Servers
In the simplest implementations of POP3, the server really does maintain a collection of text files -- one for each e-mail account. When a message arrives, the POP3 server simply appends it to the bottom of the recipient's file. When email clients checks for new email messages, it actually connects to POP3 or IMAP server on the specified port (usually 110)

The main difference between POP3 and IMAP is that pop3 deletes mail on server after it sends to the email client and IMAP doesn't.