Dungeons and Data - Feedback and Biases

Almost as soon as the survey was published, we were receiving feedback and justified criticisms in droves. It was to be expected, of course, and it was incredibly helpful information to have should the survey be repeated again in the future. As such, this section is dedicated to nearly every single question, comment, and concern that survey-takers communicated to us. Additionally, we will offer explanations and potential improvements for future Dungeons and Daddies surveys. This section is particularly lengthy but follows the chronological survey order.

Are you ready to get started?

Concern: This does not qualify as consent to information collection and could violate privacy laws regarding collection of information from minors.

Our Response: The survey was optional. None of the data that was collected discloses anything that would allow us or anyone else to identify the individuals taking the survey. Names, emails, phone numbers, addresses, and so on were never collected at any point during the survey.

Age

Concern: The age ranges leave a lot to be desired.

Our Response: The age ranges allowed us to see groups of people in ten-year spans of time and was the logical best option for data analysis. Typical groupings of adulthood usually follow a pattern of one group being less than eighteen years of age, the next being ages eighteen to twenty-five–representative of the average age range of college students–and the rest of groups split into ten-year age gaps.

Gender

Concern: The selection choices exclude binary trans individuals and are not worded well.

Our Response: By far, this was the area where we received the most feedback and criticism–and rightfully so. During the construction of the survey, we wanted to keep this section as broad as possible to prevent a massive influx of write-in responses to analyze. Unfortunately, we wound up inadvertently excluding binary trans people in the process. It was never our intention to exclude any gender identity; given that the four people who conducted this survey are trans and/or nonbinary, we should have caught that but didn’t. It was a massive fuck-up on our end, full stop, and we deeply apologize for our mistake.

Improvement: Multiple gender identities could be selected. The choices given in the survey are as follow:

Cis man
Cis woman
Transgender
Non-binary
Agender
Other
Prefer not to answer

A future survey would likely be better suited for the following, with the option to choose multiple identities again:

Man
Woman
Cisgender
Transgender
Non-binary
Agender
Genderqueer
Other
Prefer not to answer

The suggested fix would hopefully allow for a better encapsulation of gender identities.

Countries

Concern: There were many countries excluded from the available options.

Our Response: Countries that we assumed to have large English-speaking populations were added as options first; to keep the survey as concise as possible, we found the most populated countries on each continent (save for North America) and added those as options. We made some (biased and perhaps unfair) assumptions about individuals living in those countries, particularly with respect to the likelihood of there being enough interest in Dungeons and Daddies in those countries to include them as options. We were worried about this section becoming cluttered and frustrating to sift through–both for the survey-takers and the data analysts–and had to cut corners where we could. This ended up with us accidentally leaving out Scandinavian countries and island nations.

Improvement: Scandinavian countries would be included in the list of European countries; rather than deciding what countries to include based on population size, it would be wise to choose another metric (such metric, though, has not been found yet). Should that prove to be still too biased of a method to employ, then including an “other, please state” write-in option would be a better (but more complicated) solution. Another proposed solution would be to include drop-down menus (such as what was used for the American states) for every country on each continent and island nations.

States

Concern: Washington, D.C. and the US territories (Puerto Rico, Guam, etc.) were not available options.

Our Response: This area was critical in understanding the major listener-base of American Dungeons and Daddies; even with that in mind, we made some critical but accidental exclusions (Washington, D.C. and the US territories).

Improvement: Washington, D.C. and the US territories should be included as options on future surveys.

Sexuality

Concern: Multiple sexualities could not be selected.

Our Response: We simply overlooked allowing individuals to select multiple identities–nothing more, nothing less!

Improvement: Individuals should be allowed to select multiple identities on future surveys.

Occupation

Concern: Multiple occupation categories–such as manufacturing/metalworking, transportation, and stay-at-home parent/caregiver–were not available as options. Unemployed individuals were told to either choose “not currently employed” or an option closest to their previous job.

Our Response: For this section, we relied quite heavily on the assumption that our reference point sufficiently covered occupation categories and did not question further. Also, those who were unemployed had to make a difficult and frustrating decision between “not currently employed” and whatever option was closest to their previous job; there was no option for anyone that was not currently employed and had not been employed before.

Improvement: Manufacturing/metalworking, transportation, and stay-at-home parent/caregiver should be included as categories. A clearer distinction for those who are unemployed should be made; for example, informing them to either pick “not currently employed”, what most closely resembles their previous job, or “not currently employed and have not been previously employed” could be one such solution.

Education

Concern: An assumption was made that everyone listening had an education.

Our Response: Another unintentional exclusion on our part–it’s an easy fix to a mistake that will not be repeated in the future.

Improvement: An option that includes “I do not have any of these educations” should be added. Also, clarifying the “I have multiple degrees” statement in a future survey to note that we mean, “I have a Bachelor’s in x and a Master’s in y” would be beneficial.

Neurodivergence and Disability

Concern: Two separate questions should have been asked regarding disability and neurodivergence rather than being combined into one.

Our Response: We agree!

Improvement: Including two separate questions regarding disability and neurodivergence would be best suited for a future survey should this be an area of interest.

Introduction to the Podcast Through Social Media

Concern: Tumblr was not included as an option for social media.

Our Response: For some reason, Tumblr completely slipped our minds as an option. Just a silly mistake!

Improvement: Future surveys should include Tumblr as a social media option.

Patreon

Concern: There was no option available for individuals who had previously been Patreon members but are no longer members.

Our Response: We had not considered this as a possibility but are appreciative of being made aware of this; further clarification and additional options should be considered in any future surveys.

Improvement: Phrasing the question in a way that encompasses all current and past Patreon members then following up with options about current membership would be best for a future survey.

Listening App

Concern: There was no available option for Overcast for listeners who use that app.

Our Response: Yet another unintentional exclusion!

Improvement: Include an option for Overcast on future surveys.

Convention Attendance and Dungeons and Daddies Event

Concern: Individuals who were uninterested in Dungeons and Daddies events at future conventions were pathed to questions about panels; additionally, there was no “uninterested/none” option on the survey.

Our Response: This flew under the radar in the midst of the survey creation–for some reason, it never occurred to us to provide “uninterested/none” options or prevent pathing to questions about panels for those who weren’t interested.

Improvement: Including “uninterested/none” options and preventing pathing to questions about panels for those who aren’t interested should be included in future surveys.

Live Event Cities

Concern: Everyone who took the survey, regardless of whether they lived in the United States or not, was pathed to this section.

Our Response: This was another oversight in the survey creation process.

Improvement: Future surveys should have a screening question to verify United States respondents that may be interested in live events taking place in the United States. Also, separating the “I don’t live in the US” and “none of these cities work for me” options would yield more realistic statistics.

Page updated

Google Sites

Report abuse