Competency L

"Each graduate of the Master of Library and Information Science program is able to — demonstrate understanding of quantitative and qualitative research methods, the ability to design a research project, and the ability to evaluate and synthesize research literature.

What is Research?

“Research” is one of those nebulous words whose definition few can completely agree upon. Connaway and Powell (2010, chapter 1) paraphrase Tyrus Hillway by defining research as “a method of study by which, through the careful and exhaustive investigation of all the ascertainable evidence bearing upon a definable problem, we reach a solution to that problem.” Research is also thought of as a dependable way to find solutions to problems or issues we want to know more about (Powell, 2008). Webster (Merriam-Webster Online Dictionary, 2016) defines research as “studious inquiry or examination; especially : investigation or experimentation aimed at the discovery and interpretation of facts, revision of accepted theories or laws in the light of new facts, or practical application of such new or revised theories or laws”. Many people who do research or rely on research would say that there is at least two types of research: basic and applied. Basic research aims to discover new knowledge and is more interested in theoretical than practical applications. Basic research defines the hypothesis and theories which can later be tested as applied research, the second major type of research. Basic research is more general and does not aim to solve specific problems, for example much of the work done in physics is geared toward describing laws, like gravity. The second major type, applied research is more specific and is designed to answer specific questions. For example, applied research in physics lead to the successful launch of spacecraft. In library and information science, the vast majority of the research done is applied, however, it could be argued that some research, such as the use of nanoparticles to store information, is of great interest to us and could be considered basic research. Connaway and Powell (2010, chapter 1) argue that more basic research is needed in the library and information science field in order to develop theories on which to base the profession.

As librarians, understanding research is an important skill. While part of the reason is in our ability to separate the wheat from the chaff, if we are going to design and run our own research projects, we need to understand what has already been done in the field related to our question, which research designs are appropriate to what we want to know, how we are going to collect the data we need, and how we can draw the correct conclusions from our studies. An important distinction is made between “primary” and “secondary” research. Primary research is original research. If the research project gathered and analyzed data from experiments designed and run by the reporting researchers, they are doing primary research. If, on the other hand, the data was acquired from elsewhere, perhaps from other published articles, newspapers, etc., and this data is used to answer different questions, then this is considered secondary research.

Another way to distinguish types of research is based on the approach used. The type of approach chosen determines how a research effort is designed, run, analyzed, and reported. The major approaches are qualitative, quantitative, and mixed methods research. While these three approaches are not clear-cut and separate from each other, each approach is designed to conduct research based on the type of question to be answered. For example, “quantitative” research is normally used to test relationships between known variables. Data generated from these variables can be analyzed using statistical methods (Creswell, 2014, chapter 1). An example quantitative research question might be whether to increase the hours of the reference desk based on a count of the number of patrons that utilize the reference desk per hour. Using simple queueing statistical methods it is possible to determine the number of patrons most likely to use the reference desk during certain hours. A different type of approach is call “qualitative” and is more often used in situations when the question to be answered is more exploratory and the variables are not known. For example, a library may find that a certain tool they have made available to the patrons is not being used as originally anticipated by the library staff. A qualitative study could be designed wherein patrons who do use the tool could be asked exploratory questions in order to discover the cause of non-use by most patrons. This type of approach can also be used as a precursor to a quantitative approach once the variables are determined from the qualitative study. This approach, called a “mixed methods” approach integrates both qualitative and quantitative designs to answer a single question. Taking the above qualitative study of a tool that no one uses, once the problems are determined, the program can be retooled and tested with a greater sample of patrons to discover if the changes, as discovered in the first half of the research, have fixed the original problem.

Another aspect of research is the actual design and implementation of the study. There are several different methods available depending on the approach. For example, a structured interview or open-ended questionnaire may be helpful in the quantitative phase of the experiment while a survey using a quantifiable set of intervals, one of which the survey taker must choose, would be a good quantitative instrument for testing if the changes based on the first phase are effective. These are only a couple of the several tools available for designing research studies and librarians should be familiar with the most widely used in the information professions.

In general, knowing what research is and having the ability to differentiate between the types of research, as well as being able to design and execute a real research project, are all important skills for information professionals. The ability to design and implement research projects can make the difference between whether a program actually gets funded or not. Having reliable evidence to back up arguments and claims helps to convince those with the power to disburse funds that the money is worth it.

The Research Question

The research problem is an issue or situation that needs to be addressed. For most research problems the questions arise from voids in the literature, studies that produce results that conflict with each other, subjects that have been forgotten, or real needs that actualize in various settings such as organizations, homes, and within the community (Creswell, 2014, chapter 1). As discussed above, the approach used to design the research will generally depend on the question being asked. In general, however, the question is the first step in the research process and is usually a response to a perceived need.

The Literature Search

Once the question or problem to be addressed is known, all research begins with the literature search. The most important reason for doing the literature search is that we don’t want to reinvent the wheel. And, if we are claiming we are doing original research, we don’t want to look foolish if someone already found the answer to our question. As we find material seemingly related to our research, we need to make sure that our sources are legitimate. We should ask questions about each source like “who is/are the author/s; what are their qualifications (degrees awarded, positions at universities, previous research and publications) to write on the topic at hand; what is the date of publication of the research; how up-to-date is the study; has it been superseded; who is the publisher; is the publisher a respected entity; is it a university press or a private company; if a university press, what is the reputation of the institution; if private, what is the reputation of the company; if the resource in question is a journal article, is the publishing journal a respected journal in the field” (University of Arizona, 2016). Each source we chose to use for the literature search should help build the case for the research we would like to perform as well as show all relevant prior work on our question or problem.

Research Design

Once we know where our question or problem fits into the overall framework of what has already been done, we need to decide on the type of research we are going to do. As we’ve already said, the question can greatly influence our design. If the question we are working with is exploratory in nature, we will most likely design a qualitative study. If we are testing a hypothesis with known variables, we would design a quantitative study, such as determining how often books in a certain collection are checked out. There are several methods we can use to collect the data we are interested. Without exhaustively listing them here, the major methods for quantitative research are surveys, experimental, historical, and operations research, modeling, systems analysis, case studies, content analysis, and bibliometrics (Connaway and Powell, 2010). Examples of qualitative research methods include interviews, focus groups, observation, and open-ended surveys and questionnaires. As librarians, we need to understand each method, but here I will focus on a commonly found research method in libraries, surveys, as depending on how the survey is designed, it can be considered either a qualitative or quantitative approach.

A survey is used to produce statistical descriptions about an aspect of the population under study where the main method for gathering data is by asking a sample of people taken from that population questions.The population to be surveyed is extremely important and should be identified early on, if not in the problem definition phase. For example, if a survey is designed for ascertaining whether a particular young children’s program in the library was effective, it might not be helpful to administer the survey to the library’s teen population. The people who participate in the study are consider a “sample” unless everyone person in the defined population is able to take the survey. In general, this is impossible because the population could contain thousands of people or more. It is also important that the sample chosen to participate are randomly chosen so as to not introduce bias into the results. For example, if we only asked people we know love the children’s program, the results would not be very helpful for informing any changes to the program.

The survey instrument is the list of questions that will be asked of the participants in the sample. Questions can be very specific for collection of statistical data or open-ended for collection of information to be used in further research. A great deal of thought should be put into the questions. Questions should be worded unambiguously, any acronyms or unusual words should be defined, and each question should focus on a single concept. This is important whether the survey is administered as a written document or delivered in an interview situation. It should also be noted that surveys are a very popular method because they are easy to administer through a variety of channels (Dillman et.al, 2009) such as the library web page, email, phone, printed, and even via text messages.

Data Analysis

Once the data has been collected, it needs to be evaluated. As part of the design and prior to implementation of the research, the types of data that will be collected should be generally known or at least categories for responses should be defined. In quantitative studies, the range of expected values for the variables should be known ahead of time and for the studies and classes of possible answers should be defined. This will allow all data that is collected during the research to be classified into meaningful groups. It is important to make sure the classes are as mutually exclusive as possible so that there is no confusion as to which category a datum belongs once the data is collected. Once the data are collected and categorized if necessary, it is possible to analyze it using descriptive or inferential (or both) statistics. Descriptive statistics can tell us something about the frequency of an event or response such as the average, the median or middle value, and the mode or value/category that occurs most often. From these we can also determine the spread of data by looking at things like standard deviation and variance. Inferential statistics allow us to make statements about the variables we are measuring with some degree of confidence. Using inferential statistics (and a population of data with a normal distribution) it is possible to make statements like “the children’s literacy program will increase the number of words recognized by children with a confidence of 95%” or in 5% of the cases it won’t increase the number of words recognized. Framing our results this way helps us to convince the appropriate parties that the new program needs to be started, shut-down, or improved.

Coursework & Work Experience

I have taken several courses that have helped me with this competency. In INFO-200 a literature review assignment required I look at several library and information science academic journals and critically examine one of them using the criteria delineated above in the literature search section. In all three INFO-287 courses, I was required to do literature reviews in order to learn as much about the particular subject I would be working with during the semester. In my Tudor Times course, we were required to research a specific character from English history during the reign of Henry VIII. In addition to researching a character, we had to furnish our virtual character’s home and dress them appropriate to the Tudor period. We also needed to present on some aspect of life such as art or music as well as what their social position would allow them to do or not do. Because I knew nothing about the Tudor era, I spent a great deal of time searching the literature for good sources of information not only for the visual aspects of my virtual character and her house, but for reports and presentations in which we needed to include references. I used the same process for the next INFO-287 that focused on Renaissance Florence in Italy. Again I spent a great deal of time on literature searches on the character I chose from that time frame as well as her probable surroundings and interactions with members of the community in which she was embedded Finally, the last INFO-287 course I took required I develop an educational experience for others. For this course, I chose to develop a virtual exhibit that taught participants about the Scientific Revolution. Like the other two INFO-287 courses, this was not a research project in the terms that I’ve outlined above, but I did have to do an extensive literature search to identify good sources for all the objects which appeared in my virtual exhibit.

Most of my research experience in designing and implementing studies have come from my Master’s and Doctoral Thesis work and the work I do as a research scientist. My master’s thesis was the foundation for the Doctoral work in that it focused on identifying the “gap” in the current research that my own research would address. This meant I needed to do an extensive literature search, create a classification scheme for the models of comprehension that already existed, and show that none of these models alone could adequately describe how computer programmers understand code they didn’t originally write. My Ph.D. thesis could be considered basic as it involved the design and implementation of a model I developed after understanding where the gaps were. This will be part of the evidence I am submitting for this comp.   Finally, in my current position, I worked on more applied research which involves the invention of technology and then setting up experiments to prove the viability of that technology. In many cases, the research involves determining all situations in which the technology could fail and experimenting with those scenarios.

Evidence

My first piece of evidence is a literature review I did for INFO-200 in the fall of 2010. I reviewed the Library and Information Science Research journal which focuses on the research process in library and information science. I originally chose to review this journal because I was interested in finding out about the types of research projects being done in the field. This evidence focuses on the types of articles and editorials published by this journal and shows my knowledge of research methods by discussion of example articles. The evidence also demonstrates my ability to critically review the contents of the various sections of the journal and to determine the importance of the journal in the field. The assignment can be found on the evidence page for this competency called “Marie_LIBR200_Journal_Review_Assn_1-2.docx”

My second piece of evidence is an interview I did for INFO-287, Scholar’s class in the fall of 2014. The goal of this assignment was to discover what makes good educators successful in virtual worlds. This was an exploratory interview design where I wrote the questions prior to the interview and allowed the answers to be open-ended. The assignment itself called for us to turn in the questions with the answers, but not to develop an analysis piece. The interview started with some ice-breaker/get-to-know-you questions and then launched into the questions about their experiences, projects, advice, and other information about being an educator in virtual worlds. Although no analysis was required, the interview brought up some questions that could be explored in a more formal research design. For example, the answer to the question about what affordances virtual worlds offers, the interviewee answered: “Offering opportunities for home-bound and disabled people from around the world to participate in activities such as art festivals, exhibits, and working in theatre. Helping to create a world without boundaries. Getting people from around the globe working together whether that is for a real world organization or a just for fun group.” A couple of these sentences could translate into a study on its own. It would be interesting to design a research study where the number of home-bound and disabled people who attend these activities could be determined and perhaps interviewed themselves to discover why they attend. This information could then be used to advertise and expand these activities so that more home-bound and disabled people not only know about them, but want to attend. While this evidence does not constitute a research project, it is easy to see how this could be the first phase of a mixed method study on using virtual worlds for education. The evidence can be found this competency’s evidence page with the file name Educator Interview.docx.

The next piece of evidence is an annotated bibliography I created for my INFO-200 class in the fall of 2010 on the issue of outsourcing public library functions. This is an annotated bibliography with 28 sources classified into the five categories: General Works, American Library Association, Impact of Outsourcing on Libraries, Opinions, and Library Experiences. This evidence is different from the first journal reviewing assignment in that for each source I was required to read and distill the information in the article into a very short but succinct description of the material covered by the article and why it was relevant to the topic of Outsourcing. This skill is relevant to this competency because this not only requires the ability to determine the relevance of the material to the research topic but also the ability to show where the information fits within the entire research project. This evidence can be found on this competency’s evidence page with the file name Marie_Vans_Assn_4_Outsourcing_Libraries_Bibliography_Final.doc.

My final piece of evidence is my Ph.D. thesis, “A Multi-level Code Comprehension Model for Large Scale Software” (Vans, 1996). Because the thesis was done in 1996, no on-line version of it is available and because I originally generated it in the text language Latex, I don’t have an electronic version of it. However, I can describe the work in an effort to demonstrate my ability to design, execute, and analyze a research study. The goal of the study was to determine whether a comprehension model I developed was actually a good model for how software engineers understand code they did not write. This is a real issue because software engineers move around but the software they wrote still needs to be supported, enhanced, and debugged. Understanding the knowledge they use and how they use it helps to define software support tools, which were included as a set of recommendations at the end of this study. The actual study was implemented as the observation of several engineers working for a large software company whose tasks included maintaining, debugging, or enhancing a large piece of software they did not originally create. As the technology for effortlessly capturing actions or video-taping subjects, the main data collection consisted of cassette tapes of think-aloud protocols. For each engineer I observed during the study, I would first let him explain the task he was working on and then let him work while describing his thought processes. I would rarely have to prompt her/him to continue talking as it was quite often they would forget to speak. While it could be argued that just my presence was enough to create a situation where the data was not authentic, I believe the data we collected was quite valuable. Prior to recording the sessions, I created a set of categories of utterances that I expected to hear during the observation sessions. I used these categories to “code” the protocols (Ericsson and Simon, 1993) which are the typed out versions of everything said by the engineers during the sessions. Once the protocols were coded it became an easy matter of counting frequencies of each code as well as the sequence of codes to determine whether what I observed with the engineers mapped correctly to the model I originally theorized they would use while working on code they didn’t write. Because the thesis encompassed several aspects of the study, much of the thesis is publishes as several articles, the last of which was in the International Journal of Human-Computer Interaction in 1999 (Vans et.al, 1999). I believe the above description of the study that took me almost 4 years to complete demonstrates my ability to design and run a research project, especially those involving human subjects.

Conclusions

My experience during several course I took at the SJSU iSchool along with previous academic and work experience have prepared me to design and implement research efforts at any information organization. During my time as a Master’s and Ph.D. student at Colorado State University during the 1990’s I had the opportunity to run many different experiments involving technology, such as neural networks for identifying classes of articles, but the fact that my Ph.D. involved a design that included human subjects, I feel I have the ability to design research efforts at libraries where the main concerns involve people. In fact, I originally thought to do a thesis for this program in which I would design a virtual island dedicated to teaching K-12 teachers how to encourage girls to consider S(cience)T(echnology)E(ngineering)M(ath) careers. Because I have taken so long to complete this program, I’m bumping up against the 7 year limit and the thesis would take at least another year to complete. I had envisioned a study where I would develop an event on my virtual STEM island and invite teachers to come and learn about why the lack of girls in STEM is an important issue, how we can address it, and give them the tools they need as teachers. Unfortunately, in order to test the effectiveness of this training would require a longitudinal study that not only tested the perception of the teachers right after the event but whether they applied what they learned in their own classrooms and whether it was helpful or not. As the previous description shows, I had put a lot of thought into it, I have simply run out of time. However, this shows, I believe, that I am able to apply the knowledge and skills of this competency to any situation. Also, I continue to do research on a daily basis as a research scientist.

References

Beck, S.E. and Manual, K. (2008). Practical research methods for librarians and information professionals. Neal-Schuman Publishers, Inc. New York, NY.

Connaway, L. S. and Powell, R. R. (2010). Basic research methods for librarians, 5th edition. Libraries Unlimited, Westport, CT.

Creswell, J.W. (2014). Research Design. Sage Publications, Inc. Thousand Oaks, CA.

Dillman, D. A., Smyth, J.D., and Christian, L.M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method, 3rd edition. John Wiley & Sons, Inc. Hoboken, New Jersey.

Ericsson, K.A. and Simon, H.A. (1993). Protocol analysis: Verbal reports as data, 2nd edition. MIT Press, Cambridge, MA.

Fowler Jr, F. J. (2002). Survey research methods, 3rd edition. Sage publications, Thousand Oaks, CA.

Merriam-Webster Online Dictionary (2016). Research. Retrieved from http://www.merriam-webster.com/dictionary/research. Accessed February 26, 2016.

Powel, R. (2008) Research. In The portable MLIS: Insights from the experts. Haycock, K., & Sheldon, B, Eds., Libraries Unlimited, Westport, CT.

University of Arizona. (2016). Research, write, publish: The literature review. Retrieved from http://www.library.arizona.edu/help/tutorials/litrFoeviews/index.html. Accessed February 26, 2016.

Vans, A. M. (1996). A multi-level code comprehension model for large-scale software. Doctoral Dissertation. Colorado State University.

Vans, A. M., von Mayrhauser, A., & Somlo, G. (1999). Program understanding behavior during corrective maintenance of large-scale software. International Journal of Human-Computer Studies, 51(1), 31-70.