4. Sources and methods to collect data
Embedded Files
We can collect data (evidence) though two general approaches:
Secondary data are obtained thought desk research
Primary data are obtained thought own field research
4.1. Desk research and secondary sources of data and information
By desk research we acquire so called secondary data because we are taking already existing data from someone else. These could be official statistic databases, web pages of governmental organizations, news, social media, other Internet sites in general (e.g. sites of NGOs, blogs, private pages etc.), scientific journals and other sources.
Secondary sources of data and information
The secondary sources of data were in detail presented in Module 1 Reliable source of information.
For more detail on how to approach information on the internet see Module 1 and Activity 1.
For summary let us here just list the sources of data and information broadly discussed in Module 2:
Official statistic databases
Official web pages of governmental organizations
News
Wikipedia
Social media
Internet sites in general (e.g. sites of NGO, companies, blogs, private pages etc.)
Fact-checking sites
Scientific journals
4.2. Field research
We cannot give detailed description how to use the research methods but we would like to highlight key aspects so you and the students can decide what method is suitable for the questions asked. And we provide some reference to find further information about the methods and they should be used.
Basically, there are two ways of conduction your own field research: Quantitative and qualitative. The most important thing is that they represent different logic, goals and are linked to different techniques. As will be shown later, you can also use both in the same project, an approach that has become very popular – mixed methods.
LOGIC OF RESEARCH
QUANTITATIVE research is based around statistics. The sample, collection of data and its interpretation are oriented in a way that measures variables. Now, not all research problems are suited for this approach. Moreover, one of the main decisions in this approach is whether to strive – or not – for representativeness. The idea being – I have a sample that is smaller than the population I want to draw conclusions about. In this case, the sampling method becomes very difficult even for professionals. If you have this ambition, we advise you to contact a professional before you start with later phases of research. But, for smaller groups and research goals low representativeness may not be the problem. The general rule of thumb here is keep the sample structure as close to the group structure. Meaning, gender groups, age groups (and other relevant aspects of the groups!) should have close resemblance to the group you are going to draw conclusions. Apart from sampling, the research – given the orientation on numbers – should be highly standardized. For example, see below, questionnaire is a very good example – everybody is asked the same question and in most cases uses prefabricated answers. This is a basis for applying the main logic of quantitative research – comparing similarities and differences between the groups in sample and relating them to the examined population.
QUALITATIVE research is based more on narration than on numbers. In this logic, you are far from representing a population. You are trying to get the most personal, complex and deep understanding of the topic. It is also a research approach more suited for smaller groups or sensitive problems that elude standardization. Compared to quantitative approach, you are not trying to understand a population. Rather, you are trying to understand a problem in all its depth and interpretations.
MIXED METHODS. Given the above, you can in some cases mix both. You can first conduct interviews to understand a problem and then put your understanding into a more standardized form of lets say questionnaire. Or, you can find something in your quantitative research and you want to focus on that phenomena closer. You can draw conclusions from your questionnaire survey and ask about them in interviews. This is very often an approach that is related to case studies (see below).
Questionnaires and surveys
What answers they will give
Questionnaire survey is the most used technique of quantitative approach. It gives standardized measurable answers to highly standardized questions. You ask your sample the same questions, which is a basis for comparing different answers. Now, it may seem easy to come with a standardized question and offer standardized answers. That is not the case, see the section for What to be careful about below.
Given the logic, your answers should be standardized too. In most cases, you offer your respondents a measurable form of answers. That is where you apply so called scales. For example, in evaluation research, you can simply ask: How did you like the course. And provide a simple scale – one star to five star. With this design you can than measure the differences between groups. Did women like the course less than men? Did elderly like it more than youngsters?
However, in this design you will never know the ultimate question why. Why do some grade the class low while the other grade it high. Remember, we are going for at least some form of measurability and it is a questionnaire, not an interview. However, right in the next question we can ask a simple question – Why did you like the course (open question). An open text field is a offered form of answer and respondents are given opportunity to describe their opinions in their own words. Remember, in its raw form (e.g. 500 responses comprised of three to five sentences), you cannot apply any kind of quantitative analysis. These so called open questions work very nice in small groups where they might typically give more information that scales. If you find yourself in a situation with hundreds of these answers, the can be coded into a variable that can be carefully statistically analyzed. For this, we recommend to contact a professional since many mistakes can be done.
How to use them
As for the specific tools for surveys, you can go online and offline. Offline means paper distribution of questionnaires. Then we have to options – either the collection is assisted or not.
Assisted means that the questionnaire is filled together with an interviewer. They ask/read questions from the questionnaire and mark answers from the respondent. This is a classical way. The main pro here is that you have control over situation – you can see if the respondent understand or not the question, you can clarify things. The main con is time, money and also the fact that being a good interviewer is more difficult than most people think.
Unassisted means some kind of self-filling – you can hand out questionnaires to class and let the respondents fill in the answers themselves. It is totally legitimate to do so, and you still can have at least partial control over situation.
However, these days, online surveys are the most preferred way of conducting larger scale quantitative surveys. You can go with sending an email with word/excel format of your questionnaire. But, you lose anonymity (which in many cases is substantial) and also the received form of data will not be ready for analysis. But, in some situations (small group, non-conflicting topic, very low level of analysis), this can work since it is easy and comprehensible.
As for advanced tools, these are ever changing and related to the workflows in your organization/population. Sometimes, GDPR and data security has to be taken in consideration. Recommendation here is to consult the organization you represent when conducting research about their GDPR/security measures to avoid conflicts later in the survey.
The most used are Google forms, Microsoft forms or Survio/Survey Monkey and may others. When choosing this, take in consideration several things.
- Forms of questions, filtering – check that all the questions in your questionnaire can be put into your tool. Some free tools have limitations in this regard. As everything in online world is changing fast, always check for that since what worked last time does not necessarily work toady
- Limitations of number of questions, answers or respondents. Most of the paid services have limited features. Check for how many respondents you can have, check for the maximum number of questions mainly.
- Check also for the final output data file. Most statistical programs can work with .csv and standard excel format. Be careful, some services provide these files only for a limited time (month since the last answer, for example)
- Check also for language possibilities of the service
Other sources of information
Examples of Likert scale (https://www.questionpro.com/blog/likert-scale-examples/)
Representativeness
Common errors in surveys
https://www.surveymonkey.com/curiosity/double-barrel-survey-question/
Interviews (individual or group)
What answers they will give
Interviews, also known as in-depth interviews, are the main part of qualitative approach. They do not provide us with numbers (usually it makes no chance to run statistical analyses), but with longer speeches/texts on the desired topics. You should be very careful with the choice of respondents, they really should have a good insight/experience with your topic.
How to use them
In most cases, the situation is one on one, ideally face to face, albeit these days in less sensitive topics, online (telephone) interviews can be considered. Always use a prepared set of questions (scenario for the interview), so that you do not forget about anything. It is not a questionnaire, you can use different wordings of the same questions, you can change order. Conducting a good interview is a form of art that eludes standardization. As always, remember, that participation in research is a good will of the respondent. Be polite and do not push. Respect the privacy of your informant.
Conduction a group interview is also an option. The most advanced form of a group interview is a focus groups interview. This is a very complex technique based on technical and psychological resources. If you go this way, please refer to a professional.
A less problematic form is a group interview. However, even when the group is between 5 and 10 (this should be the limit), you must be very careful to let everybody speak their main, yet on the other hand not to push other and convince them. It can be very hard to control for group dynamics.
What to be careful about
First and foremost in the case of interviews, it is trust. You should have your respondent sign an informed consent, you should offer – and grant anonymity. If you want to record that interview, this should be clearly indicated and agree on.
Then, focus on your research topic. This really depends on your research design. Sometimes, a very loose and long narration can be what you are going for. Yet, in most cases, you are there to get specific answers to specific questions – and you should be able to get them, the nice way. Also, if you need to work with word-to-word transcription, the more talk there is, the more work you have, make it worthy. Rule of thumb these days? Offline interview – the limit is 40-45 minutes, online 20-25. But given the nature of topic and respondent, this could vary.
Other sources of information
Tips for better interviews
https://www.cfrinc.net/cfrblog/four-actionable-tips-to-improve-your-in-depth-interviews
Designing interviews
https://nyhealthfoundation.org/wp-content/uploads/2019/02/m_e_tool_series_indepth_interviews-1.pdf
Common errors when conducting interviews
https://www.sjsu.edu/faculty/weinstein.agrawal/urbp298_phI_handout_Interviews.pdf
Tips for group interviews
https://www.themuse.com/advice/16-quick-tips-thatll-help-you-stand-out-in-a-group-interview
Observation
What answers they will give
These day this technique is less and less applied, however you can find yourselves in a situation that calls for this. It may seem as an easy one, however, there are some decisions to be made and thing you need to control.
First, is it a situation where you need to inform the other about your presence as a researcher? It will interfere in the situation, however it is ethical and in some cases hard to avoid. For example, living in a group of homeless people. On the other hand, you may go and do observation during a football match. These is no bigger need to “uncover” yourself.
How to use them
In this technique, the focus is everything. Do not go into the filed unprepared – do all possible research and make related decisions at home, there will most likely be no time for that in the field. Also, decide on how you are going to store your observation – recordings, field notes…
Most importantly, decide over what you need and need not to observe. Some thing can be asking for your attention, yet have no benefit to your research agenda.
What to be careful about
This depends on your research goals and resources, but you should have a higher number of instances that you observe. But, be careful with the choice, because then another pitfall comes into play – hidden/unexpected factors. Especially, if you plan observation over a longer timespan (e.g. one season of football matches and behavior of the fans), many things can interfere. For example, there can be a change in police presence (driven by a rising conflict), the fans can change (a court decision over “fan crimes”).
Other sources of information
General tips on observation
https://www.studysmarter.us/explanations/social-studies/theories-and-methods/observation/
Step by step observation
https://www.sociologygroup.com/observational-research/
Observation examples
Case Studies
What answers they will give
The goal here is to understand thoroughly one (monoghraphy) or more cases. Usually this is combination of several techniques. If you want to understand let us say an impact of policy change in a given school (e.g. introducing uniforms), you should combine desktop research, interviews and even partially a small survey. The idea is to present your research topic in all its depth and variety, to include as much perspectives and views as possible.
How to use them
The main condition and decision is to have your case clearly defined. You should also have some strategy how to obtain data – will the examined organization give it to you, are the people willing to participate? You could, if possible, also go for more than one case study and to compare between them, if possible.
What to be careful about
Three things are critical:
- Data availability
- Access to different views on the case
- Clearly defined time frame
Other sources of information
General info on case studies
https://explorable.com/case-study-research-design
Case study guide
https://library.unimelb.edu.au/__data/assets/pdf_file/0011/1924175/Casestudy_Research.pdf
Errors when conducting case study
SUMMARY) Take into the class Summary of the field research methods to be elaborated later.
Page updated
Google Sites
Report abuse