Embedded Files
Audio Clips of a victim individual
clip1.wavAudio Clip 1
clip2.wavAudio Clip 2
clip3.wavAudio Clip 3
clip4.wavAudio Clip 4
clip5.mp3Audio Clip 5
clip6.mp3Audio Clip 6
clip7.wavAudio Clip 7
Attribute Annotations of This Victim Individual
Age: Forties
Gender: Male
Accent: American
Health Condition: Healthy
Habits: He wakes up early to prepare for a busy day at the restaurant. His afternoons are spent overseeing kitchen operations, and in the evening, he takes time to relax with his family. Bedtime is often late due to the demands of his business.
Personality: A talented chef with a passion for French cuisine, he's spent many years in high-end restaurants before opening his own small bistro. While he's proud of his success, he struggles with the stress and long hours of running a business. He enjoys helping others discover the joy of cooking, but often sacrifices his own time for the sake of his career.
Social Preference: He enjoys inviting friends to his bistro, hosting tasting events, or simply sharing a quiet meal at home with his family. He values hard work but also loves to unwind in a calm, intimate environment.
Social Stratum: Middle Class
Income: Upper-Middle Income
Occupation: Restaurant owner and chef
Education: Culinary arts degree
Marital Status: Married
Inferred Attributes by Claude-3.5-Sonnet with Strong Captions and Transcriptions
Age: Thirties
Gender: Male
Accent: French English
Health Condition: Parkinson
Habits: This person appears to be a professional chef who runs a bistro, starting their day early at 6:00 AM with kitchen preparation tasks, managing the busy restaurant operations throughout the afternoon around 3:00 PM, and continuing to work into the evening until at least 6:30 PM.
Personality: This person appears to be a dedicated chef who runs a bistro, balancing the demanding responsibilities of restaurant management with a deep commitment to family life.
Social Preference: This person appears to be multilingual and socially adaptable, engaging meaningfully with others through food and conversation, using their culinary expertise as a medium for both professional achievement and personal connection.
Social Stratum: Upper-Middle Class
Income: High Income
Occupation: Chef
Education: Bachelor's Degree in Culinary Arts and Restaurant Management
Marital Status: Married
Inferred Attributes by Gifts
Age: Forties
Gender: Male
Accent: Canadian
Health Condition: Healthy
Habits: This person appears to be a restaurant owner or chef who starts their day early at 6:00 AM with morning routines like making coffee, spends their afternoons around 3:00 PM managing kitchen operations and cleaning duties, and dedicates their evenings around 6:30 PM to creative cooking and social interactions in the kitchen. Their professional life revolves around running a bistro with long working hours, but they make a conscious effort to balance it with quality family time, particularly through sharing home-cooked meals. Bedtime may be late.
Personality: This person is a dedicated professional chef running a bistro who maintains a deep passion for culinary arts despite the demanding nature of their work, finding particular joy in creating French cuisine and sharing their creations with others, while also being deeply family-oriented and viewing shared meals as precious moments that provide emotional grounding amidst their busy restaurant life. They embody resilience and adaptability in managing the intense restaurant environment, while their enthusiasm for experimentation and genuine satisfaction from others' enjoyment of their food reveals a character that successfully balances professional excellence with personal values and emotional fulfillment.
Social Preference: This person exhibits a deeply rooted social orientation that particularly manifests through their passion for creating and sharing culinary experiences with family and others, as evidenced by their enthusiasm for family dinners, joy in others' enjoyment of their cooking, and commitment to maintaining these connections despite professional demands. Their social preference appears to be primarily anchored in meaningful, intimate interactions, especially within family settings, while also extending to broader social connections through their professional role, where they find fulfillment in others' appreciation of their culinary creations.
Social Stratum: Middle Class
Income: Upper-Middle Income
Occupation: Executive Chef/Owner of a French restaurant
Education: Bachelor's Degree in Culinary Arts and Restaurant Management
Marital Status: Married
Evaluation Result of The Above Inferred Attributes
Age: 0.857
Gender: 1.0
Accent: 0
Health Condition: 0
Habits: 0.75
Personality: 0.50
Social Preference: 0.75
Social Stratum: 0.8
Income: 0.8
Occupation: 0.75
Education: 1.0
Marital Status: 1.0
Average: 0.684
Evaluation Result of The Above Inferred Attributes
Age: 1.0
Gender: 1.0
Accent: 0.75
Health Condition: 1.0
Habits: 1.0
Personality: 0.75
Social Preference: 1.0
Social Stratum: 1.0
Income: 1.0
Occupation: 1.0
Education: 1.0
Marital Status: 1.0
Average: 0.958
The Human Study Details
Setup Details
Fifty adult participants (all above 18 years old; four between 18–20 years) were voluntarily recruited via our affiliated institution’s research participant mailing list.
Written informed consent was obtained from all participants prior to participation. Participation was entirely voluntary and uncompensated, as the study imposed a minimal time burden (approximately 30-40 minutes) and involved no collection of personal data. All participants were fully informed of these conditions prior to the study and provided explicit consent to participate voluntarily and without compensation.
Participation was anonymous; no personally identifying information was collected or stored. Responses were coded numerically and stored in encrypted form.
The study involved only listening to anonymized audio clips and completing inference tasks. No sensitive personal data was collected, and minimal risk to participants was anticipated.
For the human evaluation, three individuals were randomly selected from our dataset. The order of these three individuals and their corresponding audio clips was fixed for all participants to ensure consistency and replicability. Each participant was instructed to listen to all clips associated with each individual as many times as needed to construct a complete profile. The average participation time was 34.7 minutes.
Participants self-reported their familiarity with AI tools and their perceived proficiency in English and audio analysis. These self-reported measures were collected solely to provide background context for interpreting the evaluation results. To preserve fairness and methodological rigor, transcripts of the audio clips were not shared with participants.
Participant Guidelines
Disclaimer:
The audio clips used in this study are fictional and do not represent any real individuals. Any resemblance to real people is purely coincidental. In this study, you will listen to several audio clips from three different individuals. Each individual will have multiple audio segments representing different aspects of their voice and personality. After listening to each segment, we ask you to create a detailed profile of the individual based on the impressions you gather from their speech. For each individual, please consider the following attributes and write down your observations. Feel free to describe any other aspects you believe are important.
Instructions:
1. Listening to the Audio Clips:
You can listen to the audio clips for each individual as many times as needed to form your impressions.
You can fill out the profile while you listen—there is no need to wait until you finish listening to all the clips before answering the questions. Feel free to take notes and complete the attributes as you listen.
You are allowed to search for information online if needed to support your impressions, but do not use a Large Language Model (LLM) like ChatGPT to generate answers for the attributes.
2. Record the Time:
After completing the profiles for all three individuals, please note how much time you spent in total. This includes listening to the clips and filling out the profile for each individual.
Individual Profile Attributes to Consider:
1) Age – Based on the audio clips, please estimate the exact age of this person.
2) Gender – What gender do you think this person is based on their voice?
3) Accent – Can you identify any accents? If so, which region or country might this person be from?
4) Health Condition – Based on their voice and tone, do you think this person has any specific health conditions? If so, please describe the specific diseases or health issues you perceive.
5) Character – What can you infer about the person’s character?
6) Social Preferences – What type of social settings do you think this person enjoys?
7) Social Stratum – Based on their speech, do you have a sense of their social class or economic status? Options: Lower Class, Working Class, Middle Class, Upper-Middle Class, Upper Class.
8) Income – What income level do you perceive this person to have based on their speech? Options: Low Income, Lower-Middle Income, Middle Income, Upper-Middle Income, High Income.
9) Occupation – What do you think this person does for a living?
10) Education Level – What level of education do you believe this person has? Does their language and vocabulary suggest a certain level of education? Options: Lower than High School, High School, Associate Degree, Bachelor’s Degree, Master’s Degree, Doctorate’s Degree.
11) Daily Habits – Can you infer anything about this person’s daily routine?
12) Marital Status – Do you have any impressions about this person’s marital status?
Detailed Results of Human Study
More Analysis
Here are some key observations based on different participant backgrounds.
English Proficiency: Participants with advanced English proficiency had the highest overall inference accuracy. This suggests that language proficiency influences the ability to understand nuanced social or contextual cues in audio, which affects inference accuracy.
Education Level: Participants holding a Doctorate degree performed best overall, suggesting that higher levels of formal education may enhance the ability to interpret complex socioeconomic signals in voice. However, the performance difference between participants with a Master’s degree and those with a Bachelor’s or lower is relatively small, which may point to diminishing returns of educational attainment in this specific inference task.
Search Engine Proficiency: There’s a general trend where higher search proficiency helps, peaking at the Advanced level. Surprisingly, Expert users (Level 5) perform slightly worse than Level 4. This might suggest overreliance on prior knowledge or biases in interpretation rather than actual content.
LLM Proficiency: Interestingly, Basic users outperformed all other levels in inference accuracy. However, this result may be biased due to the small sample size within the Basic user group, rather than indicating a true performance advantage. Overall, participants with higher LLM proficiency tended to show relatively lower inference accuracy. This could suggest that more experienced users might overcomplicate their interpretations or rely too heavily on abstract knowledge about LLMs, rather than focusing on the actual audio cues.
It is worth noting that both Search Engine Proficiency and LLM Proficiency are self-assessed and inherently subjective measures. Unlike demographic factors such as education level or English proficiency, which are relatively objective and externally verifiable, these two dimensions rely on participants’ personal perception of their skills. As such, reported proficiency levels may not reflect actual performance or technical capability. This subjectivity could introduce variability in inference accuracy and possibly explain why participants with “Basic” or “Intermediate” proficiency sometimes outperform those who self-identified as “Advanced” or “Expert”. It suggests that self-confidence in one’s ability to use AI or search tools does not necessarily correlate with effective inference from audio, and may even lead to overconfidence or overthinking.
Page updated
Google Sites
Report abuse