The concept as far as programming and functionality goes like this:

1. A person walks into this geodesic dome and speaks into a microphone * (we would later shelve this idea in favor of a touch-screen kiosk display to type on for simplicity's sake)

2. The audio is processed in Unity using some plugin to generate text from a given audio.

3. That text is then sent to ChatGPT using another plugin, the text is processed their and we ask it to simply rephrase whatever the speaker said a couple different ways.

4. Those rephrased messages from ChatGPT will then get sent to a Microsoft Azure TTS service, where an AI will generate speech from those rephrased messages, each message having a different AI speaker.

5. Finally those voice lines are played back numerous times in a cacophony of noise around the speaker using a quad speaker setup.

With the premise for how the technology would work done, I then began working more on the visual aspect. Based on some visuals we had from NLM, we wanted to go for a grid-like look, where the speakers/viewers would see their messages pass through the grid, and then the responses would be passed back, creating a wave-like deformation in the grid.

Initially, I wasn't entirely sure what the perspective would look like, so I played around first with this flat, face on projection with a grid-like plane slowly distorting based on some procedural noise. I then thought if the viewers would be encased in a dome, I would recreate that kind of look as well, as seen in the second video below.

Shortly after we began concepting some ideas, we would meet up in-person with some folks from the NLM to further get specifics and clarifications as to what they wanted, and how it might look.

• The moodboard and colors they were using

• The projectors and how it would be projected onto the dome (in this case, 2 short throw projects)

• The use of ChatGPT and Microsoft Azure to rephrase the things viewers would say, and the generation of text-to-speech

  • Including ideas on moderation, and what kind of things we could do to prevent a viewer from inputting foul language or hateful speech considering this would be an running exhibition for all ages for a couple months

So on, and so forth. I learned a lot more about what it is we were aiming for, and the idea of social media stuck out in my mind. I thought about how the grid might represent our inter-connected internet based social structure, and how these phrases could essentially be messages, like an iPhone text bubble.

I began sprucing up the visuals to better match the themes for this idea of Data Nation, based on a quick concept I sketched up in Blender and Photoshop.

Obviously, the visuals are quite a bit messy in the video, mainly because there's just too much visual noise going on, with text bubbles blocking each other, the little packets of messages that travel between the nodes have a particle system that streaks everywhere, but the general idea seemed pretty good to me, so I think I was headed in the right direction.

Finally the day came to where we would present our working prototype so far, as well as getting to finally see how it would play out in real-time in the prototype dome they've finished. I only made a few more visual adjustments since the last video, mainly small things like taking away the text bubble backgrounds (as to clean it up a little), adjusting the time when the text pops up, the little "new message" notification, fixing the particle system for the data packets sent between the nodes, so on and so forth.

Below you'll see some quick videos I took as we were testing it out on the dome.

Overall, the showcase and prototype testing went well. The good people at the NLM seemed pleased with it, and so did we. We got a few more ideas about how to improve now being able to fully test it in it's intended environment, which helped to see it from a slightly different angle.

We had some more discussions about moderation, what kinds of things we could do like leave it as is, with ChatGPT's default moderation disallowing any hate speech to be rephrased, or coming up with our own filter to layer before ChatGPT so a viewer wouldn't even be able to type in a naughty message, or we could allow them to use a curse word, but then bleep it out when the speech synthesis is created.

We also noted it lasted just a bit too long, and despite the fact that there's 5 different speakers (and thus, 5 synthesized voice lines), it was quite a bit hard to hear all of them distinctly, rather only the first one, and maybe another one would really stand out, just based on the tone, pitch, and inflection of the selected AI voice actors.

We had the thought that maybe when the first few nodes are infected, we play each voice line individually, at or near full volume, while turning down the volume on other voices, or possibly, randomly adjusting the volume of the lines each time they're played, so viewers can really get to hear what each voice is rephrasing.

With that out of the way, we have a healthy time frame in the scope of a month, as it's currently early May (at the time of this writing), and the exhibition isn't formerly opened yet until the second of June 2nd, with the first sneak peak at the finished product on June 1st. So after the first rush to get the prototype finished to show and get feedback on, we now have a pretty sizable time to continue to refine and perfect it. Besides Kevin's upcoming implementation of some sort of moderation system on our end (pending approval based on what the folks at the NLM think is necessary), this project is essentially feature complete as far as major functionality goes.

All that's left for me personally is minor polishing, for example, we had text message bubble backgrounds on the text but disabled that as it was too clunky and overlapped with other messages. Since our first prototype run through, we've adjusted the text quite a bit, and I've since re-implemented the bubble background as I think it helps with readability. Though, the now the text and background are simply black and white, which doesn't match the theme as closely as have the iconic green or blue bubbles popularized by Android and iOS, but again, the text is far more readable in monotone.