Participant Data QuestionsSample Size QuestionsQuestion: If a study is conducted at the group/team do I also report the number of individuals?Answer: Ye, please include both the number of individuals and the number of group/teams if applicable. Age Mean Value QuestionsQuestion: If a paper does not provide the mean, but does provide the range do we report the range instead?Answer: Yes, please report the range instead. Remember to report the actual range itself do not calculate the midpoint and use that value.Country of Origin QuestionsQuestion: Is this the country the study took place?Answer: No, this should be the study of origin of the participants not where the study was conducted.**************************************************************************************************************************************************************************Analysis Data QuestionsGeneral QuestionsQuestion: I totally confused by how the data reports it analysis. What do I do? Answer: I would just make a note of this in the final comments. Please cite the location of the text i.e. page number and column and paragraph. We can meet to discuss it later. Manipulation Checks QuestionsQuestion: Should I code ?manipulation checksAnswer: No. You should not include information related to manipulation checks into the form. They are not apart of the study unless they show up as independent variables. If this is true the information we need would be in the section on independent variables. Question: What are manipulation checks anyway? Answer: Researchers conducting experimental studies attempt to increase or decrease independent variables (i.e. manipulate). Manipulation checks are performed to ensure that the researchers were successful at increasing and/or decreasing their variables. Pilot Study QuestionsQuestion: Should I code ?pilot studiesAnswer: No. Question: What are pilot studies anyway? Answer: Researchers conducting experimental studies often run pre-studies to ensure that everything is ready to go for the full study.Beta Coefficient QuestionsQuestion: What are ? beta coefficientsAnswer: Beta coefficients are the estimates resulting from some type of regression analysis. They can be standardized or unstandardized normally represented in many papers by the symbol β. Question: What information do I need to record with regards to the beta coefficient? Answer: The beta coefficient, standard error and/or the t-value or p-value associated with the significance of the beta coefficient. The t-value is determine by the beta coefficient/standard error (i.e. T value = beta coefficient/standard error). Therefore, if we know 2 of the three values we can determine the other. In addition, we can determine the t-value by the p-value associated with the significance of the beta coefficient. Correlation Coefficient Questions Answer: Correlation coefficients represent the bivariate correlation between two variables that range from -1 to 1. They are represented in many papers by the symbol r. For example, the correlation between variable X and variable Y is r = .50. T and F Test QuestionsAnswer: T-tests are normally used when comparing two means, while F-test are used to compare more than two means. For example, when comparing the mean of the control condition with the mean of treatment 1 condition a T-test is used. However, when comparing the means of the control condition, treatment 1 condition and treatment condition 2 a F-test is used. Question: The paper reports both ? Which do I report?T-tests and F-testsAnswer: Both. In many cases papers with use a to determine if there are differences between more than two means. For example, when comparing the means of the control condition, treatment 1 condition and treatment condition. Then after discovering that theF-test F-test is significant many studies will compare each mean with another mean using a the . T-testIdeally, if the paper reports the mean and standard deviation for each condition we can calculate the . Unfortunately, many papers do not report the means and/or standard deviation. When this occurs we can then use the T and F values to determine the effect sizes. F and T testsQuestion: This study had Bonferroni corrected paired-sample t-tests should I include them as T-tests?Answer: Yes, you should include the Bonferroni corrected paired-sample t-tests. Bonferroni and/or Tukey are post hoc t-test done after the F-test. You should include them as t-test and note that they are bonferroni corrected in the comments at the end of the survey. Question: Some papers are reporting the results of a F-test from an ANOVA by only comparing two means. But, I thought we use F-test when we are comparing more than two means and T-test when comparing just two means.Answer: Typically yes, but you can also run an ANOVA with only one condition. This produces a F-value that informs you that the one condition is significant. This implies that at least one mean within that condition is significantly different from at least one other mean. However, when your condition only has two means you do not need to determine which means are different from each other. You can simply treat this as an F-test with two means. You skip the control condition and input the two means as separate treatments. ****************************************************************************************************************************************************************************Study's Setting QuestionsQuestion: How can I determine the study's setting? Typically, the context and task are key indicators followed by the participants.Context: For example, if the study is being conducted in a lab setup as a home it would fall under the home setting. If the study is being done in a hospital it would probably fall under a healthcare setting. If the study was being in a classroom it would probably fall under a education setting. Task:For example, if the task involved a robot helping the participant find a good movie to watch it would probably fall under the home setting. If the task involved a robot helping to get patients motivated to rehab a recovering injury it would probably fall under a healthcare setting. If the task involved helping participants learn math then it would probably fall under an education setting. Context and Task:Often you can use the context and task to determine the study's setting. For example, if the study is being conducted in a lab setup as a home and the task involved a robot helping the participant find a good movie to watch. You would be more confident that it would fall under the home setting. For example, if the study is being done in a hospital and the task involved a robot helping to get patients motivated to rehab a recovering injury. You would be more confident that it would fall under a healthcare setting. For example, if the study was being done in a classroom and the task was helping participants learn math. You would be more confident that it would fall under a education setting. Participants:At times the type of participants can help you determine the study's settings. For example, if the participants were patients recovering from an injury you could conclude it was a healthcare setting. Unfortunately, most studies use students as participants even if there are attempting to study home or work settings. Please see the question below for more details on this issue. Therefore, the context and task are the primary sources of information when determining the study's setting. Question: Would a study conducted at a university using students as participants automatically be considered an educational setting?Answer: No. Almost all the studies will be conducted using students at a university. However, unless the task or context (e.g. participants are college students being tutored by robots) is related to education or learning then the study does not fall under the category of an "educational" setting. *****************************************************************************************************************************************************************************Questions about the RobotType of Robot ControlQuestion: How can I tell the difference between remote controlled robot operation and Wizard of Oz?Answer: If the participants controlled the robots and not the researchers during the study it was remote controlled. However, if the researchers controlled the robot and attempted to hide the fact that the robot was not autonomous it was Wizard of Oz. |