training on the non-surgery database. • The correlation analysis of match scores from all six recognition algorithms is performed using the Pearson correlation coefficient. It is observed that the algorithms have limited correlation. The correlation analysis suggests that, for recognizing surgically altered images, these techniques provide complementary information and the performance may improve with effective fusion algorithm. IV. DISCUSSION Plastic surgery has been an unexplored area in the face recognition domain and it poses ethical, social and engineering challenges. Being related to the medical history of an individual which is secure under law, invasion of privacy is an important constraint in this research. In some cases, facial plastic surgery is performed due to medical reasons and sometimes it is the individual’s choice (i.e. cosmetic/aesthetic surgery). In both cases, even though individuals undergoing facial plastic surgery cannot be bound under any legal and social obligations, it is ethical responsibility of the person to get the face image/template updated in the database (i.e. template update). With the advancement in plastic surgery technology, identity theft is another problem. Identity theft can be intentional when a person consciously attempts to resemble someone by undergoing facial plastic surgery procedures or unintentional where he/she may resemble someone else after the surgery. Therefore, face recognition algorithms must be able to distinguish between a genuine and stolen identity, for which the system must include other cross references apart from a recognition algorithm. Apart from ethical and social issues, several engineering challenges are also important in developing algorithms to handle variations due to facial plastic surgery. First one is to have an algorithm that can classify whether the false acceptance or rejection is owed to facial plastic surgery or to some other covariate such as aging or disguise. Since some of the local plastic surgery preserves overall appearance and the texture of the face, this challenge may not be significant for some cases. However, in other cases including global plastic surgery or full face lift cases where the entire structure of the face is remodeled, it is of paramount interest to automatically differentiate among plastic surgery, aging, and disguise. In other words, face recognition algorithms must be able to single out the variations in face due to facial plastic surgery from the variations due to other covariates such as aging, disguise, illumination, expression and pose. Despite many advances in face recognition techniques, to the best of our knowledge, there exists no technique that can perform such classification. Even if we somehow (e.g. manually) identify that a particular human face has undergone plastic surgery, it is still an arduous task for current face recognition algorithms to effectively match a SUBMITTED TO IEEE TIFS 6 post-surgery image with a pre-surgery face image. Therefore, an engineering challenge would be to design an algorithm to correlate facial features in pre and post surgery images. Local facial regions such as nose, chin, eyelids, cheek, lips and forehead have an imperative role in face recognition and small variations in any of these features carry a partial affect on the neighboring features. Further, a combination of local plastic surgery procedures may result in a fairly distinct face from the original face. To develop an algorithm to assess such non-linear variations in pre and post facial plastic surgery images makes the engineering challenge fiercer. It is our assertion that these challenges should receive immediate attention from the research community to develop efficient face recognition algorithms that can account for non-linear variations introduced by facial plastic surgery procedures. Here, it is important to note that plastic surgery poses some fundamental issues that cannot be completely solved by engineering solutions only. V. CONCLUSION AND FUTURE RESEARCH DIRECTIONS Popularity of plastic surgery has increased many folds over the past few years and the statistical data shows that it keeps growing. Due to advances in technology, affordability, and the speed with which these procedures can be performed, several people undergo plastic surgery for medical reasons and some choose cosmetic surgery to look younger or for better appearance. The procedures can significantly change the facial regions both locally and globally, altering the appearance, facial features and texture, thereby posing a serious challenge to face recognition systems. Existing face recognition algorithms generally rely on local and global facial features and any variation can affect the recognition performance. This paper introduces plastic surgery as a new dimension for face recognition algorithms. We present an experimental study to quantitatively evaluate the performance of face recognition algorithms on a plastic surgery database that contains face images with both local and global surgeries. The study shows that appearance, feature, and texture based algorithms are unable to effectively mitigate the variations caused by the plastic surgery procedures. Based on the results, we believe that more research is required in order to design an optimal face recognition algorithm that can also account for the challenges due to plastic surgery. It is our assertion that the results