In September/October both Google and Yahoo posted some Usability Research on Federated Login:

Google Presentation at OpenID Foundation on September 18, 2008

Yahoo UX Research on their IDP endpoint

Both companies have been asked for suggestions on how to merge the feedback from these two sets of research.  Here is one way to merge the set of conclusions:

1. Based on the test Yahoo & Gmail have done earlier in the year, we already both believe that any "login" buttons have to include the brand of the IDP and must be right next to the login box.  That does not scale, nor does it promote the OpenID technology brand, so far that we use the other features in this lsit.
2. For RPs who currently use E-mail based logins, they can maximize adoption by using either the Google or Yahoo UX suggestions along with education/re-education screens.  However this requires that the Google/Yahoo IDP also verifies the user's email.
3. The Google UX option will enable RPs to trust the most IDPs, but the Yahoo UX suggestion will make the process the simplest for users of the few IDPs that are listed below the login box.  So the best solution may be for RPs to combine our two UX suggestions by using the Google UX for the login box, but still include buttons for a very small number of IDPs under the login box.  Note that even if the RP displays buttons, users do not tend to notice them, however the Google UX option allows the RP to promote the federated login option, and if the user chose an IDP who has a button on the RP webpage, then the RP can tell users that they can use the button in the future.
   
4. From the data we have gathered, buttons work best if they are (1) just below the login box (either the password or sign-in button), (2) contain the full name of the E-mail provider (not just logo), and (3) the set of buttons is no wider then the login box.  That generally means a max of 2 buttons for a regular username/password login box, and a max of 3 buttons for the buy.com style login box Google suggests.  For example, the RP might show buttons for the 2-3 biggest social networks in their country.  One of those buttons could be an OpenID branded button if the RP was okay with the usability of that approach, and the fact that most IDPs will not be able to provide an E-mail address for which that IDP is authoratative.
   
5. An RP can use one-off protocols for a few big IDPs (like @hotmail.com), but OpenID is a good fit for RPs that want to support a large number of IDPs

This suggested combination was based off the two publicized research reports, along with one other old study Google had done.  In that study I took Netflix's login page and added a Gmail & Yahoo signin button below it.  For that study, I used a modified Yahoo BBAuth confirmation page that indicated the user's E-mail would also be shared.  As you would expect based on Yahoo's research, none of them clicked the Gmail or Yahoo buttons.  However, what I did was create a fake "upgrade to federated login" prompt that they were shown after signing in.  Four of the six testers chose to go through the wizard, and the other two said they might skip it if they were trying to do something quick, but they would want to come back later and do it.  The "upgrade wizard" took them through the flow of federated login.  Once users were sent back to Netflix, I then showed an "education" screen where I told users that in the future they should click the yahoo or gmail buttons below the login box.  I then distracted them for a while pretending I was done with the test, and then said "oh, i forgot to test one more thing, can you just sign in again one more time?"  Of the 6 users, 4 typed their Gmail/Yahoo E-mail and password into the Netflix login boxes (just like in Yahoo's UX research), and only 2 remembered the "education" screen they had just seen a few minutes ago.  For the 4 who forgot, I then reshowed them a modified version of the education screen that asked them to please use this method in the future, with a button to go to Yahoo/Gmail to finish their current login attempt.  Those 4 were all slightly embarrassed, but all said the re-education screen was good, and that they liked the fact that once they got trained, they would no longer see it.  One thing I had planned to test was to ask people to sign up for Netflix instead of sign-in, and then on the signup screen once they typed a Yahoo or Gmail address, I would use JavaScript to suggest that they "upgrade" to federated login instead of creating yet another password.  I never did that in this scenario, but we did something very similar in the Google research we publicized.

After reading the details of Yahoo's research, I realized that this old study was half-way between the suggested UX of Google and Yahoo's published reports.  The only difference between this old study and Google's suggested UI is that we changed the login box to the new style described in our research.  However that technique still uses an education/re-education screen, and it had the same % of users who forgot the education the first time.  That technique also uses the idea of prompting existing yahoo/gmail users to "upgrade" to federated login, as well as looking for new accounts that users try to create for yahoo/gmail addresses, and redirect them to the federated login flow.  So the list of conclusions above suggests how these different user studies could be merged.

Eric Sachs
Product Manager, Google Security