Most of your travel will probably be out of Taoyuan International Airport (TPE), though also see below for travel out of Taipei Songshan (TSA).
There are a few ways to get to and from TPE. Much of the relevant information can be found on the TPE airport webpage.
MRT: This is the most efficient way to get to and from the airport. Both terminals of TPE are accessible via an express service on the Purple line, accessible from Taipei Main Station (Blue/Red line) or Beimen (Green line). A one way trip will take about 45 minutes and cost around 150 NTD.
Taxi: Also a quick (modulo traffic) and easy way to get to and from TPE. Taxis will run between 1000–1200 NTD depending on where you're going to / coming from in Taipei. Catching a cab to the airport you can ask how much when you get in the cab and they're usually happy to give a fixed price, though going on the meter doesn't seem to come out much different.
Bus: Bus is the cheapest (~140 NTD) way to go and takes something like an hour from Taipei Main Station to TPE depending on traffic.
HSR + Shuttle: A good way to go can be high-speed rail (HSR) to Taoyuan station (18 min ride from Taipei Main, leaves every 30 min, 160NTD), then a shuttle bus to the airport (~15 min ride, leaving every ~10 min, 30 NTD). Particularly convenient if you are flying China Airlines or EVA since you can check bags and get your boarding pass at the HSR station in Taoyuan.
Note however you are arriving to TPE: Know which terminal you are flying out of (1 or 2) as it can be less than obvious where the various airlines are located as you are arriving by bus / taxi.
Taipei Songshan (TSA): Many flights to relatively nearby locations in Asia go from TSA. Definitely preferable to TPE when possible since it is small, located in north central Taipei, and the MRT (Brown line) goes there directly.