It is exciting to show how the contents of working memory impact behavioral performance, but it would be even more exciting if we show that the current contents of the human mind can be “read out” from brain activity. This may sound like science fiction, but machine learning and related methods now make it possible to decode the specific feature values being held in working memory. fMRI has been used to decode the contents of working memory for several years, but fMRI signals are too sluggish to track the moment-by-moment dynamics of working memory. EEG-based measures have the millisecond-level temporal resolution needed to track these dynamics, but it is usually assumed that EEG lacks the spatial resolution needed for precise decoding of visual features such as orientation. Here, I have developed a new method that utilizes machine learning algorithms to decode very precise visual features from scalp EEG recordings while these features are being perceived and held in working memory. We can now decode which of 16 different orientations is being held in working memory (see Figures on the top for spatial distribution of alpha band EEG activities and ERPs for 16 different orientation values and Figures on the bottom for decoding of orientations) and which of 16 different directions of coherent motion is being perceived in a field of noisy motion display. This new method has made it possible to answer important questions about the nature of working memory and its relationship to attention. Check out our papers on EEG decoding!

We use a variety of visual features to investigate how those features are represented in human mind but we often ignore important properties of the visual representations of these features. For example, studies of working memory have focused on how we remember colors, but they have assumed that all colors are represented uniformly, ignoring well-known color category effects. I have conducted several studies to investigate the effect of color categories in color working memory. In a study published in JEP:G, I have shown that color memories are biased toward nearest category prototype. This result suggests that color working memory combines category-independent metric color information and category information. This research provides new insights on the nature of the limits in visual working memory. (Credit for the image: Royce Faddis/JHU)

Perception is by no means direct reflection of physical stimulus. Our brain interprets noisy sensory inputs and make inference about them. It has been known that this inference is influenced by many cognitive factors, including our past experience. In orientation perception, for example, studies have demonstrated that orientation memory on a given trial is biased by the orientation that we saw in the previous trial. What's the underlying mechanisms of this effect? In our lab, we try to answer this question using psychophysics and EEG decoding. We found that the previous-trial information actually gets reactivated during the current trial, providing potential mechanism for serial dependency in working memory, and trial-history effects more broadly.