Recognizing Emotion in Spontaneous Speech is difficult because emotions are not explicitly demonstrated by the speaker.
There are essentially two approaches.
Problem with Speech -> Text -> Sentiment is that one can not distinguish a sarcastic /thank you/ from a genuine /thank you/.
The monograph should detail more on these aspects.
(Some Slides; Keynote at CSCITA 2017)