Whenever Picasa Photo Viewer displays a photo, it also shows all the other photos in the same folder.
So what is happening here? When you click on an attachment in an email, Windows copies it to the same place it puts everything related to internet browsing - Temporary Internet Files. When Picasa Photo Viewer is called to display the photo, it sees that photo plus everything else that is in the folder, hence all the cached images.
The solution? Rather than clicking directly on the picture to view the attachment, you should choose to save the attachment(s) to a separate folder. Then when you view a picture with Picasa Photo Viewer, it won't display any other images from the cache.