At this point, your villagers have been eating, sleeping, and working, now is the time to change that. By purchasing a Library at this point your villagers will begin to gain intelligence, just like Children do in a School. High intelligence increases the chance that a villager will skill up. Investing in one, although pricey, is worthwhile in the long run. Since you already have a library you might as well get an Enchanter as well. The Enchanter will craft paper and books to use and trade. He will also enchant armor, tools, and weapons for your villagers to use.