Taiwan is possessed of enormous natural beauty, fabulous cultural depth, and historical sites in abundance. If you're looking for things to see and do but you're short on ideas (or haven't purchased the guide book yet), there are various travel blogs that provide run downs on local and national sites (such as here and here) along with trail guides for hiking (e.g. here). You'll (probably) want to see some or all of this while you live and work here, but the only way to do that is to get out and about! Within Taipei itself, a dense network of buses, comprehensive MRT (underground), and nearly ubiquitous YouBikes provide ample ways to explore the city and its surroundings. Travelling further afield is just as simple; here's some advice on doing just that.
For the ultimate in authentic Taiwanese driving experience nothing beats a scooter. You'll need a local license to drive one, but luckily it's not too difficult to get. Adam Iaizzi, a postdoc at NTU, has kindly provided a summary of his experience getting a scooter license, and some tips and tricks, here.
Intercity coaches are an inexpensive option to get where you want to go if timing isn't an issue. Long distance coaches are well appointed with comfortable reclining seats, air conditioning, and in some cases wi-fi.
The local rail network offers access to much of the region adjoining Taipei, serving many places that are otherwise difficult to get to by bus. Trains are often crowded on weekend mornings with hikers heading out for their excursions. Tickets can be purchased from machines at the station, or pre-booked online. You can also use an Easy Card to tap on (and tap off!) at all railway stations. Seat reservations are essential for comfort on long distance journeys.
Taiwan's HighSpeed Rail (高鐵) offers relatively inexpensive and rapid travel down the West coast of the island stopping at, or near to, most major cities along the way. Travel to Kaosiung from Taipei takes ~ 2.5 hrs at a cost of 1,200 NTD. Tickets can be ordered online, or purchased using an Ibon machine or at the station.