把聲音視覺化,以光影與符號展現敘事的聲景。
《留聲影像》是一個聲音視覺化動能藝術作品,專為聽障人士設計,利用圖像記譜法幫助他們欣賞聲音藝術,有如20世紀初、一種使用旋轉紙卷創造動畫的光學玩具「Ombro Cinéma」。《留聲影像》並非著重動畫,而是強調視覺敘事和通過圖像符號把聲音可視化。學生使用視覺符號來代表聲音,表達他們對動態聽覺信號的詮釋。
學生使用圖像記譜,以抽象圖案、符號以及文字解釋的形式去描述音樂,逃脫傳統樂譜的框架。圖像記譜可以啟發學生探索聲音與視覺符號的關連,透過將聲音「視覺化」,觀眾可從圖像讀取節奏與強弱變化,以視覺方式理解聲音結構,建立不依賴樂理或聽力的多感官體驗。
作品的聲音素材來源包括:現場錄音:於戶外或室內收錄環境聲響;擬聲音效:以人聲或日常物品模擬特定聲音,營造更真實的聲音效果。收集素材後,再以音訊軟體剪輯、加工,完成專屬的作品聲效。作品中的電子音效皆由學生親自設計,學生使用Ableton Playground 網上合成器與 Audacity 剪輯軟體,進行聲音創作與編輯。
SnapSound is a sound-visualization kinetic artwork designed specifically for people with hearing impairments. It uses graphic notation to help them appreciate sound art—similar to an early-20th-century optical toy, “Ombro Cinéma,” which creates animation by using rotating paper scrolls. While “Sound-Aligned Images” is not primarily focused on animation, it emphasizes visual storytelling and the process of making sound visible through visual symbols.
In the artwork, students use visual symbols to represent sounds, expressing their own interpretations of dynamic audio signals. They employ graphic scoring, describing music through abstract patterns, symbols, and accompanying textual explanations, stepping beyond the framework of traditional sheet music. Graphic scores can inspire students to explore the relationship between sound and visual symbolism. By “visualizing” sound, audiences can read rhythm and changes in intensity/volume from the images—understanding the sound structure visually, and building a multi-sensory experience that does not rely on conventional music theory or on hearing.
The sound sources used in the project include:
Field recordings: capturing ambient sounds in outdoor or indoor environments.
Foley sound effects: using voices or everyday objects to simulate specific sounds and create more realistic audio effects.
After collecting the materials, students use audio software to edit and process them, producing the final customized sound effects for the work. All electronic sound elements in the piece were designed and created by the students themselves. They used Ableton Playground (online synthesizers) and Audacity (for editing) to carry out sound creation and editing.
導師帶領學員以「聲音視覺化」為主題,從聆聽與聲音採集到通感呈現。學生運用圖像記譜法,以抽象圖案、線條和符號記錄音樂,取代傳統的五線譜,再使用摩打製作畫面與聲音同步的走馬燈效果。觀眾即可以視覺方式解讀圖像中的節奏和強弱變化,理解聲音的結構。學生從中學習到:
聲音的概念與採集,藉著不同聆聽練習,重新認識聲音,學習捕捉聲音的不同器材運用與技巧;
感官訊號轉化,探索聲音與視覺符號的關連,配合形體表達,以攝影記錄及簡易程式編碼,將聲音視覺化,創作出不同能力人士都能欣賞的藝術作品;
電子線路和機械零件(如摩打)的操作原理及裝嵌,以程式編碼配合電子零件的創作實踐,並探索利用生活中常見物品(熱敏紙)化為創作工具的可能性;
Audacity 聲音編輯軟件和 Arduino 軟硬件的運用;
結合以上技巧,創作出獨一無二的聲音+視覺藝術作品。
The instructor guides students through the theme of "Sound Visualization," from listening and sound acquisition to synesthetic presentation. Students use image notation, recording music with abstract patterns, lines, and symbols instead of traditional staff notation, and then use motors to create a synchronized visual and sound effect. Viewers can visually interpret the rhythm and dynamic changes in the images and understand the structure of the sound. Students learn:
The concept and acquisition of sound, rediscovering sound through different listening exercises, and learning the use and techniques of different sound capture equipment;
Sensory signal transformation, exploring the relationship between sound and visual symbols, combining physical expression, using photography and simple programming to visualize sound and create artworks that can be appreciated by people of different abilities;
The operating principles and assembly of electronic circuits and mechanical parts (such as motors), using programming to create works with electronic components, and exploring the possibility of using common household items (thermal paper) as creative tools;
The use of Audacity sound editing software and Arduino hardware and software;
Combining the above techniques to create unique sound + visual art works.