Popups can be quite troublesome when using Selenium, often it is due to timing issues, the popup needs to be rendered first before you can take action. Here is a sample:

  <tr>
	<td>waitForPopUp</td>
	<td>{name of the popup window}</td> <!-- when none use _blank -->
	<td>30000</td>
  </tr>
  <tr>
	<td>selectWindow</td>
	<td>name={name of the popup window}</td> <!-- when none use _blank -->
	<td></td>
  </tr>

  <!-- 
    this one is importand, make sure you wait for 
    something to become visible in the
    popup window because it needs time to render.
  -->
  <tr>
	<td>waitForVisible</td>
	<td>{some field on the page}</td>
	<td></td>
  </tr>

  <!-- DO YOUR ACTIONS HERE THAT NEED TO TAKE PLACE IN THE POPUP -->

  <tr>
	<td>selectWindow</td>

        
        <!-- 
             select the window that opened the popup, 
             when it is the starting window use null 
        -->
	<td>null</td> 
	<td></td>
  </tr> 

If you use some data transfer between the popup to the normal window (for example when you can search for a person in the popup and then select a row, which in turn trasfers data to the normal window). The make sure you put a waifForValue on a field that is filled by the popup window before you continue to take another action.