AntiELAB survey report

Methods and data

In total, we conducted onsite surveys on 12 protests throughout June and July 2019: June 9, June 12, June 16, June 17, June 21, June 26, July 1, July 14, July 21, July 27 and August 4. Because of resource constraints and judgment of the scale and relative importance of upcoming protests, we could not cover every single protest activity (e.g., the Sheung Shui protest on July 13 and the Mongkok protest on August 3 were among the “missing” cases). Besides, we conducted an onsite survey during the pro-government rally on July 20. But since this report focuses on the anti-extradition bill movement itself, findings from the July 20 rally are not reported here.

The protests covered in this report can be categorized into three types. Type I are mass rallies that begin with the assembling of protesters at an assembling point before following a specific route to a designated destination. Type II are static demonstrations that involve the gathering of protesters at a specific location. Type III are fluid demonstrations where protesters first gather at a specific location but then became more mobile and unpredictable, spontaneously moving to various locations like wildcat strikes. The table below summarizes certain basic information about each protest onsite survey.

Types of protests

Generally, interviewers are asked to invite every tenth person they see within a designated area/route to complete the survey. If they are turned down, they will then ask the next tenth person — so on and so forth. However, since each protest has its unique spatial dynamics, it is necessary to make slight adjustment in the sampling methods.

Mass rallies (Type I): We first divide the area where the rally begins into different zones, and we also designate several destinations along the protest route. Each interviewer is assigned to a specific zone where he/she begins to conduct the survey. He/she is instructed to follow the protest crowd within the designated zone to start walking until reaching his/her assigned destination. Along this route, the interviewer continues to conduct the survey following the sampling procedure.
Static demonstrations (Type II): We divide the area where the rally begins into different zones. Each interviewer is assigned to a specific zone where he/she walks around (in circles) to conduct the survey following the sampling procedure.
Fluid demonstrations (Type III): The method is similar to Type II. But because of the fluidity of the protest, the fieldwork supervisor monitors the situation onsite and redeploys interviewers to different locations.

In each protest, the fieldwork team was led by two to three supervisors. The number of interviewers ranged from 10 to 25, which vary according to the expected turnout. Interviewers are asked to switch on their live location function on Whatsapp so that their actual location can be monitored. This is to facilitate the redeployment of interviews to new locations if needed.

Survey methods

The fieldwork team surveyed protest participants through two options: 1) online-based questionnaire using QR code or 2) paper-based questionnaire. Interviewees were usually first approached with the first option. If they accepted the survey request, they would scan a QR code presented by the interviewer using their own phones and then finish it on their own without the help of the interviewer. If they were interested in the survey but declined the use of online-based questionnaire, interviewees would be presented with the second option: the paper-based questionnaire. The interviewer would read out each question and then fill in the questionnaire together with the interviewee. This conventional face-to-face interview typically takes around 10 minutes. Rejections of both options (that is, refusal to participate in the survey altogether) were recorded by the interviewer.

The combination of the two survey options aims to strike a balance between getting a large enough sample size and making sure that the sample is representative of the protest participants. The use of online-based questionnaires is useful for achieving the first objective. Given that protests were usually announced at short notice, the research team encountered the difficulty in building a large team of interviewers to ensure a significant amount of sample. An important task was therefore to maximize the sample size even with a small team of interviewers. Using Qualtrics, an online survey software, the online-based questionnaires -- which were completed by interviewees on their own capacity -- could reduce the time for interviewers to solicit responses, thus resulting in larger sample sizes.

To be sure, online-based questionnaires have their limitations. First, people who are less tech-savvy are more likely to decline the survey request. Even if they might have started the online survey with the help of interviewers, they are more likely to drop out during the process. Second, because protesting has become an increasingly risky activity under Hong Kong’s present political climate, some participants worried that filling in an online questionnaire through their own mobile phones might bring legal or political repercussions. These two limitations may systematically exclude a portion of the protest participants, thus resulting in response biases. A final limitation is that online-based questionnaires hinges on good Internet connection. In large protests, however, Internet connection oftentimes becomes slow and unstable. Signals may jam and may also get interfered with by the authorities.

Therefore, we use face-to-face, paper-based questionnaires to circumvent these limitations. Paper-based questionnaires take substantially more time to complete. But the upside is that they will not exclude people who are not used to technology and also those who worried about the risk of leaving digital footprints. And obviously, they are not affected by the quality of Internet connection. The use of paper-based questionnaires can thus ensure that at least a portion of the sample is representative of the protest participants, as compared with the sample collected through the online-based survey.

Weighing of samples

Interviewers would be tasked to enter the data of the completed questionnaires through the online system within two days after the concerned protest. This allows the research team to acquire the full sample. The last two columns of the above table show the breakdown of the number of interviewees recruited through the two survey methods. In general, we found that interviewees who completed the online-based questionnaires are younger and more educated -- an indication of the exclusionary impact of technology. If such differences between the sub-samples are statistically significant, we would weigh the sample based on the data collected through the paper-based questionnaires, which is more likely to be representative of the protester population, to mitigate the response bias. This can help balance the objective of maximizing the sample size and the need to keep the sample representative.

Although the above correction may help mitigate the impact of response bias, another possible concern in the sampling process is selection bias. Selection bias is a problem associated with the interviewers, which happens when they systematically (either intentionally or unintentionally) exclude people with certain characteristics from the survey. For instance, young interviewers may be inclined to invite young protesters to complete the survey, whom they assume to be less likely to turn them down. Interviewers are generally more inclined to talk to friendly-looking people than unfriendly ones. Walgrave and Verhulst (2011) argued that one way to mitigate selection bias is to have fieldwork supervisors serve as “pointers” and select interviewees. This can prevent interviewers from selecting interviewees to their own liking and ensures the sampling method to be strictly followed. But while we see the merits of this method, it may not be practical for the recent protests in Hong Kong. The method requires a large team of fieldwork supervisors, which is difficult to put together given the insufficient planning time. More fundamentally, protests in Hong Kong are often very large in scale, involving hundreds of thousands of people. It is virtually impossible for pointers to systematically select interviewees from the protest crowd.

For these reasons, we decided to delegate the selection to the interviewers. But to ensure that they would follow the sampling procedures, we provided a training session for interviewers before each protest to brief them about the questions and the sampling procedures. In addition to that, we also imposed a quota limiting the maximum number of samples they could collect within an hour (usually 10-15), such that they would space out the interviews rather than doing them in groups (for instance, on a group of protesters)

Post-hoc surveys

All of the protest events were surveyed through the above method -- with the exception of the July 27 mass rally in Yuen Long. The Yuen Long protest was unique because it received a Letter of Objection from the police, which made the protest “illegal”. The protest also took place a week after thugs dressed in white shirts, who were widely suspected to be triad members, launched an indiscriminate attack against citizens in the Yuen Long MTR station. Because of the concerns over safety, we decided to change our survey approach from face-to-face surveys to post-hoc surveys. We dispatched four helpers to distribute 1800 leaflets printed with a QR-code that was linked to an online survey. The response rate, as expected, was much lower than the previous approach, but we were still able to collect around 235 responses (13% response rate) by noon the next day.

We later learnt that the approach used in the Yuen Long protest may in fact be effective in reducing the selection bias caused by helpers when they select interviewees. The reason is that distributing leaflets may reach more protesters and avoid systematically excluding certain types of protesters. As a result, we combined this approach with our standard approach in the Tseung Kwan-O rally and the Sai Wan demonstration on August 4. The results showed that using leaflets can significantly increase the sample size.

One final limitation to note is that these onsite surveys may have excluded the militant protesters who are on the protest frontline. This is especially plausible in the fluid demonstrations which often involved violent confrontations with the police. To protect the safety of interviewers, we advised them not to conduct surveys on the frontline. For example, in the June 21 survey, although we asked the interviewers to conduct interviews outside the Police Headquarters (at that time surrounded by protesters), the interviewers probably could not reach those protesters who stood closest to the front door of the police station. Moreover, militant protesters are often tense and dressed in protective gear. They are thus less likely to accept survey requests as compared with “ordinary” protesters. As a result, militant protesters are likely to be somewhat under-represented in our samples.