3. Practical tools when making sense of information
Embedded Files
3.1. How to analyze data and information
After we have collected the data through the research phase [1] (see Module 3) we have to analyze them to be able to come to relevant findings and conclusions. To learn how to properly analyze data using sophisticated statistical methods will be more a university level. But this should not “scare” us or stop us from do it with primary or secondary school students. Even quite simple methods can give us quite reliable outcomes.
At first and importantly we have to understand the difference between quantitative and qualitative data (see also Module 3):
Quantitative data:
- presented as number and referring to measurement: How many, How much, How often
- anything that can be counted or we can set an intervals shoving difference (like three times larger than something else)
- mathematical and statistical operations can be used – analyzed by statistical analysis
- gathered from statistics or by representative questionnaires or observations
Qualitative data:
- description referring to “why” or “ how” (what is behind certain behavior – intentions, feelings, motivation, or explanation how something works or describing properties)
- gathered by interviews of from texts, videos, recordings
[1] Someone could well point out that analysis is also part of the research and we would agree.
IMPORTANT Be careful not to forget that quite similar information can become qualitative and quantitative – see the example below.
EXAMPLE We have done an interview with all 10 students in the class about how they feel. Based on the interview we could understand the qualitative aspects about how they feel and why. Bu we can also quantify the findings as for example that half of the students (five out of ten or 50 %) felt “bad” today (by their own subjective assessment) or that the reason for students to feel “bad” was due to, receiving low score from the exam (4 out of 5 students) and problems in relationships with other students (1 out of 5).
Analyze quantitative data using descriptive statistics
To analyze quantitative data we recommend to use simple methods of descriptive statistics as: percentage (%), means. median, kvantiles. As such all the analyses can be done in MS Excel.
RECOMENDATION: If you are not a math teacher, we would recommend you to collaborate with a math teacher. This will be very good opportunity to show students how to apply methods from one field (math and statistics) to other field (biology, geography, history etc.).
EXAMPLE Graf example shoving percentage of students from different grades: How do you feel today in the school?
EXAMPLE Graf example showing the reason of dissatisfaction of students that do not feel good in the school today: Why you do not feel good in the school today?
Qualitative data analysis example
EXAMPLE From the school survey we know that reason for 20 % of students who did not feel “good” at the school were the relationships with other students (see the example above). So we could use interviews (qualitative approach) to understand what is actually the problem the students have about the relationships (it could be bulling or some other issues).
From the example above you can see that qualitative research (data) are well suited to explain and better understand the findings from quantitative research, which give us “just” the measurement (how many students) bud we might not still understand the reasons (what is behind it – the Why).
3.2. How to assess relevance of evidence
As in a detective story we should look for evidence supporting our claims and arguments. Mostly, when arguing, it is pointed at “facts” as an evidence that something have happened or when explaining an issue. But it is important to notice that “fact” as itself does not have to be providing solid evidence.
Evidence – data proving or disproving that something has happened or that is has certain properties.
There could be many approaches to asses and understand the evidence (e.g. legal evidence, forensic approach etc.). As our goal is responsible decision making we have to keep in account that in the real word (compared to designed experiment) we will deal with incomplete evidence to make the decisions (there will be some unknowns). Therefore it is important to understand the relevance of evidence in explaining the concerned issue. And we as an “ACTIVE citizens” should also ask the question about the impact of an action. As such we will mostly need to answer question whether something was caused by something or might cause it in the future – in other words what impact the action (intervention) had or is expected to have.
For our purposes to assess and understand evidence we can best adapt approach from Process Tracing method used in evaluation practices [1]. In short, it is method to be used to test for causation. So the purpose of this approach is to assess whether we can link the effect (impact) with the cause (intervention). Hence we speak of: “establishing causation,’’ or ‘‘confirming’’ or ‘‘eliminating’’
an hypothesis.
[1] For our purposes we adapt just some of the aspects of the Process Tracing. For further reference you can use following sources: Straws-in-the-wind, Hoops and Smoking Guns: What can Process Tracing Offer to Impact Evaluation? by Melanie Punton or Understanding Process Tracing by David Collier
FOUR EVIDENCE TESTS FOR CAUSATION:
EXAMPLE Straw in the Wind
EXAMPLE: Motive for murder (she left the accused man with another lover, the accused man gain profit from the death of the victim).
EXAMPLE: Suspect someone doing something because he/she gained profit from it. That someone profited from the result, does mean that he/she did it (cause it).
EXAMPLE Hoop test
EXAMPLE: Lacking a good alibi is not enough on its own to prove the hypothesis (convince for murderer). But strong alibi disproves hypothesis that the suspect could be the murderer.
EXAMPLE Smoking Gun
EXAMPLE: Suspect was found holding a smoking gun over the death body.
EXAMPLE Doubly Decisive test
EXAMPLE: The murder was filmed on camera showing the suspect stabbing the victim with knife. BUT be careful not to forget some conditions for considering the evidence as relevant (Could not the video be altered? See the important remark below)
EXAMPLE: Climate change: the temperature is changing need to be true (important is that we have the means to measure it)
IMPORTANT Be careful not to forget some conditions for considering the evidence as relevant. As evidence could be wrongly assessed as doubly decisive. As for example: Could not the video be altered? Does what we see on the video corresponds with other evidence as was proven murder weapon (knife, type of gun used), fatal wounds caused the death of the victim etc.? The evidence must be always considered in the context and be consistent with other evidence.
IMPORTANT NOT all data/facts/evidence is equal to be considered when looking for and finding explanations for issue concerned. The reliability, validity and completeness of the evidence (data used as evidence) must be assessed and considered (see Module 1 how to assess the Quality of data and information).
SUMMARY) Take into the class Make sure that students understand different “strength” of evidence available concerning the concerned issue.
3.3. How to come to conclusions
Based on evidence (data and information) that was assessed we should prove or dismiss the hypothesis:
Hypothesis – for our purposes we understand hypothesis as a statements (explanation, argument) that could be further examined/tested (at least to some extend)* to be proven or disproved based on evidence.
* In the complex word we might not come to full or 100% prove of something so we have to be aware of the strength of evidence and the level of “certainty” we can judge (prove or disprove) something.
SUMMARY) Take into the class Use the above approach in the class to structure arguments and understanding (explanation) about the concerned issue.
Page updated
Google Sites
Report abuse