Online Listening Test
Report LGRP #3
Online Listening Test
Report LGRP #3
Embedded Files
Report LGRP #3
On this page you can find the video and the results of the blind online listening challenge (survey).
This test was online from 08.06.17 until 26.06.2017 and was completed by 226 respondents.
PDF/ The downloadable paper can be found down this page
Click here to see how the challenge was presented to the public.
INTRODUCTION
1
In 2014 the Leonardo Guitar Research Project (LGRP) conducted its first study using blind and non-blind playing and listening tests. 10 classical guitars made from non-tropical woods and 5 classical guitars made from tropical woods were assessed, using a number of different methods, by 3 guitarists/3 listeners and 2 audiences (66 listeners).
2
At the beginning of 2017 a new study was carried out, this time using 44 guitars (22 guitars made from tropical wood and 22 guitars made from non-tropical wood). 20 guitar players were involved in the testing of the instruments.
This second study was designed to re-examine the findings of the first study. On this occassion we chosed to (1) use a larger number of guitars and players, (2) execute the tests in 3 different countries, (3) test ‘pairs’ of guitars, one from tropical wood and the other from non-tropical wood, made by the same builder. (This had not been the case in the first study).
3
For this online blind test we employed some of the guitars that had been used during the second study. This time, however, the listening test was carried out using an online survey. Over a two week period. 226 people participated to the test.
click here to see how the challenge was presented to the public.
MATERIAL
AUDIO
<— The audio (distributed by several media) used for the blind online test
The survey asked questions such as:
• How many guitars did you perceive ?
• How many of the perceived guitars are made from non-tropical wood species?
All survey questions & results, see further down
<—The video (which was released after the survey was closed)
THE GUITARS
• Number of guitars in the audio/video : 16
• Number of builders who made the guitars: 8
Each builder made a pair of guitars consisting of one made from tropical woods (T) and one made from non-tropical woods (NT). The pairs appear consecutively in the track.
• All guitars were made within the same time span, 2015/2016.
• All guitars are of the same model and have the same strings.
• The wood species are variable for parts such as: back/sides, bridge, fingerboard and neck but NOT for the top plate; all guitars have European spruce tops and the same bracing pattern.
• Order of the 8 T’s and the 8 NT’s in the track:
T / NT / T / NT / T / NT / T / NT / T / NT / T / NT / T / NT / T / NT
• The 15 transition time points:
0:10 / 0:23 / 0:36 / 0:51 / 1:03 / 1:22 / 1:28 / 1:43 / 2:00 / 2:10 / 2:18 / 2:31 / 2:42 / 2:50 / 2:59
Construction of the instruments.
•All guitars were made in the Centre for Musical Instrument Building (CMB, Belgium) under the guidance of master luthiers and guitar making teachers Karel Dedain and Walter Verreydt.
•All guitars are of the same model. They all have European spruce tops, the same bracing pattern and the same strings.
•These 16 guitars were also used for blind and non-blind playing and listening tests in the LGRP Phase #2 study.
Recording / Mastering
The musical piece - Tristorosa by H. Villa Lobos - was recorded on each of the 16 guitars.
These recordings were edited into sections of the desired length and then pasted together into one musical piece.
All guitars were recorded with a flat EQ. For the mastering, no audio effects were added.
The guitars were recorded in stereo (X-Y technique) using two cardioid Neumann KMi microphones.
The reverb is natural: large room, 9m long x 6m wide x 5m high.
To minimise the risk of deviation, the position of the left hand, the position of the guitar in relation to the microphones, and the playing style were strictly controlled throughout the 16 recordings.
The player, Gaëlle Solal, is world-class. Her masterful control guarantees consistent and even playing on all the instruments.
Overview of the MAIN RESULTS
The survey questions & responses.
Number of respondents: 226
naked survey results, see down under.
Question about:
Quality of audio used by the assessors
Question 1:
Please check the option, which best describes the audio quality you used for listening.
Answers/results:
Question about
Awareness
Question 2:
If you had NOT known that there were several guitars in the track, would you have noticed it ? (*this question requires an answer)
Answers:
Question about:
Perceived guitars
Question 3:
How many guitars did you perceive ? (*this question requires an answer)
• Average number of perceived guitars over 226 respondents: 5 (out of 16)
Question about:
Transition time points
Question 4 :
If you perceived more than one guitar, please tell us where the transition(s) between the guitars took place. Note the transition time point(s) (when one guitar follows another) in the box below. For example, if you detected 5 guitars you need to fill in 4 time points, separating them with a slash, e.g. 0:28 / 0:56 / 2:16 / 2:56 (this question can be skipped)
• 80 respondents (out of 226) provided time points / 146 skipped the question
• 8 respondents failed to provide the information in the form requested
• 72 respondents noted a total of 420 transition time points (average: 5,8 time points out of 15)
• Total number of correctly detected time points over 72 respondents = 312 (average: 4,3 time points out of 15)
N.B.: a margin of error of 2 sec. before and 2 sec. after the actual second where the transition took place was applied.
• Total number of correctly detected time points excluding the margin of error = 135 (average: 1,8 time points out of 15)
• Total number of wrongly detected time points out of 420 (i.e; where no transition took place) = 108.
Questions about:
Nature of the guitars (T or NT)
Question 5 :
Can you distinguish the tropical wood guitars from the non-tropical guitars? this question requires an answer
Answers:
Question 6 :
If you answered YES to question N° 5, please tell us the nature of the guitar (T= tropical; NT=non-tropical) according to the order in which you detected them (for example, if you detected 5 guitars: T, T, NT,T, NT). This question can be skipped.
• 29 of the 226 respondents answered / 197 skipped
• 6 respondents failed to provide the information in the form requested
• 23 respondents provided the information in the form requested
• Total number of identified natures (T or NT) by the 29 respondents = 119 (average: 4 'natures' where noted))
• 9 identifications (of 3.390 = 0,26%), over several respondents, correctly matched the nature of a guitar with its corresponding transition point (226 respondents x 15 transition points = 3.390).
• 110 identifications failed to correctly match the nature of a guitar with its corresponding timeslot. Either the timeslot was correct but the nature was wrong, or the timeslot given encompassed 2 or more guitars.
Supplementary findings concerning the transition time points and the builders
The 16 guitars played on the track alternated throughout according to their nature (T/NT; T/NT etc.) with a total of 15 transition points. Because the guitars were presented in pairs according to their builder, there are also 7 transition points between the different builders' guitars. (These 7 points mark the transition between one builder's NT guitar and the following builder's T guitar). The results show that the average number of transitions detected between the guitars of different builders was higher than the average number of transitions detected between the T and NT guitars of individual builders.
For example; transition 6 (between builder 3 and 4) was detected by 51 respondents. The following transition - between builder 4's two guitars - was only detected by 3 respondents.
All correctly detected time points were compiled into an excel file which was used to make a graph showing the results per transition time point and guitar builder.
CONCLUSION
• All 216 respondents perceived combinations of several guitars (including both T’s and NT’s) as being ONE guitar.
• In this test it was very difficult for listeners/respondents to differentiate one guitar from the other, and virtually impossible to distinguish between guitars made from tropical wood species from those made from non-tropical wood species.
• Although several respondents demonstrated outstanding listening abilities (by indicating 7, 8 to 9 correct transition time points), the ability to detect the nature of the guitars was notably less pronounced.
• This test shows that the distinctive sound qualities and the supposed nature of T’s and NT’s were not distinguishable one from the other.
• This test implies that neither group (Tropical or Non-Tropical) possesses inherently distinctive, readily identifiable sound qualities.
• Indeed, as there are clearly more time points detected between T’s and NT’s made by different builders than time points between T’s and NT’s made by a given builder, it would appear that the builder may have a more pronounced effect on differences in sound quality than the wood species used for back/sides, bridge, fingerboard and neck. We should, however, exercise caution as some respondents indicated in their comments that they were able to detect transition points based on "clicks" caused by editing rather than on a perceived difference in sound quality between guitars. We are still in the process of analysing whether or not there were more detectable "clicks" or other editing phenomena between different builders than between guitars from the same builder.
• Furthermore, if we consider other studies on this subject (see extra info), the question has to be asked as to whether the woods for back/sides, bridge, fingerboard and neck are really as important as has been previously assumed. (see also this related study)
DISCUSSION
If you have any concerns or comments about any aspect of this study, please take contact with leonardo.guitar.research@cmbpuurs.be
EXTRA INFORMATION
• See the results of the LGRP studies Phase#1 and Phase#2
• All 16 guitars were subjected to tests (acoustic signature) by François Gautier, Professeur en Acoustique/vibrations Laboratoire d'Acoustique de L'Université du Maine.
The report of that study will be published when completed.
Find below already the first chart wirth te acoustic signatures of the of non-tropical and tropical classical guitars.
Find below:
A comparable video with 12 steel-string guitars (6 tropical and 6 non-tropical guitars).
A study conducted at Lancaster University
by Samuele Carcagno, Roger Bucknall, Jim Woodhouse, Claudia Fritz, and Chris Plack.
• Video with 12 steel-string guitars of the same model in one song. Fabien Degryse plays 'in my solitude", Duke Ellington.
The audio of this video was used to carry out 2 audience tests. One in Rome (It) and one in Le Mans (Fr).
The public was asked to listen very attentively to the audio. The following questions were then asked:
• Did you notice anything unusual during the track ?
• Who perceived more than one guitar in the recording?
• How many guitars did you perceive ?
Almost all listeners on both occasions thought that they had been listening to only one guitar.
A study conducted at Lancaster University by Samuele Carcagno, Roger Bucknall, Jim Woodhouse, Claudia Fritz, and Chris Plack.
These online tests are part of an ongoing study on the preferences of guitar players for different woods used for the back and side plates of guitars. These tests are open to both musicians and non-musicians, but guitar players are especially encouraged to take part. These tests may give you an idea of what’s your favourite guitar wood. More importantly, the results of these tests will provide precious data to understand the impact of guitar wood choice on guitar sound quality. This is important from an environmental and economic perspective, because the most prized and expensive woods used in guitar manufacturing (e.g. Brazilian Rosewood, which is an endangered species) are scarce and from unsustainable sources. The results of these tests may allow us to identify woods that are cheap and sustainable, but that don’t compromise on sound quality. These results will be made publicly available on there website at the end of the study.Please click on the following link to take part : http://www.psych.lancs.ac.uk/hearing/the-guitar-experiment/online-tests/
Pdf paper.
⬇︎⬇︎⬇︎
Online_listening_test.pdf⬇︎⬇︎⬇︎ PDF of the naked survey results + comments of respondents in 'SurveyMonkey'
LGRP#3 Online Test Data_excl mails.pdfPage updated
Google Sites
Report abuse