E(motional)-Reader
'Augmented reading'
A project that uses VADER sentiment analysis to determine sentiment values in passages from books and visualises these as movements and colours on a reader program powered by OpenFrameworks and in the room around you.
As the promotional video shows, the project uses a source text, in this case a passage out of 'The Lord of the Rings' by J.R.R.(1) Tolkien that has been analysed by VADER sentiment analysis(2).
I performed the actual analysis in Python (code included), already extending my very small comfort zone in coding. This analysis resulted in a dataset which I pruned and structured so I could easily fetch and use the data in my OpenFrameworks project. I then searched for an example that would read out text on the screen like subtitles, and found a file which I used as the basis of the project. I then started adding in ofxGui to control my reading speed and such. After a while I made it so my program could read out the text and the sentiment analysis on the next line to a float. This float was the 'compound' value, a summary of multiple parameters that could tell me how 'emotional' the sentence analysed actually was and if this was positive, negative or neutral. I implemented this value in a function that drew circles around a few parameters that were influenced by this compound value (inspired by a tutorial from Lewis Lepton (4)). In this way it would move more erratically when emotions ran high and perform certain movements when the sentence read was either positive or negative. I mapped the compound value to RGB values that could control the colours and finally created a function with which I could control the smoothness over which the circles would transition to the next compound value of the next sentence.
Then I exported the program to my Ipad, Hooked up my RGB led strip to OpenFrameworks and gave my room-mate a light show which I recorded.
No, as you have probably noticed, the promotional video was 'wizard of Oz'd', I manually gave my RGB strip commands to match the current compound light values and edited the project video output in this video with some chroma-keying.
The idea behind the project was to extend the 'space' of reading a book. It was also inspired by the lesser amount of concentration I experience during the times of Corona, I wanted to see if I could enhance my concentration for reading by using this program. Normally I visualize reading in my head, but how cool would it be if the whole room would start responding to what you are reading, smooth lights and animations and ambient sounds could take this experience to another plane of space. I wanted to externalize the reading more so your peripheral vision would play a role in noticing the subtle ambience changes caused by the program. The circular motions were designed to lead the eyes to the text and keep the attention locked with subtle movements, creating a sort of Droste effect/ optical feedback loop. It was also a kind of experiment to see if an 'augmented' form of reading would work in practice without distracting too much. For me, that worked but of course I can't speak for others...
(Below is a video of the actual program reading a passage of 'The fellowship of the Ring'(1).
Files:
https://drive.google.com/file/d/1J8W3kAMCToueLM4LWgNIPSSf2B-7Kh1v/view?usp=sharing
References:
(1) Tolkien, J. R. R., & Inglis, R. (1990). The fellowship of the ring. Prince Frederick, MD: Recorded Books. P.202.
(2) https://pypi.org/project/vader-sentiment/
(3) https://github.com/openframeworks/openFrameworks/tree/patch-release/examples/input_output/loadTextFileExample
(4) https://www.youtube.com/watch?v=NBpBR2gRPEE