Genetic dissection of rare and complex phenotypes