Anderson's Media Research Methods, Ch. 2 - 6

Anderson Ch. 2

In studying the media, various entities can be understood or put into categories based on their properties, processes, consequences, and character. Properties can be studied through basic data collection (surveys, statistics) or immersive engagement (close reading, participant observation). Studying processes requires either sequential measurement methodologies or longer-form interpretive work (protocol analysis, interviews). Consequence is studied in a way that attempts to establish cause and effect, and this can be done through experimentation or participant observation. For character, populations need to be studied, and so surveys, rhetorical criticism, and cultural studies can work to answer questions.

Anderson Ch. 3

Methodology should be public, evidence based, and rule directed, and the studies that result should generate public knowledge. Public knowledge is not necessarily true, but it has passed enough standards not to be deemed false. Methodologies are developed in response to theories; without theory, there is no way to create a systematized method.

It is easier to find problems if you keep current with the discipline’s research. This should be done on a global scale, not limiting it to one’s home country. After doing preliminary research, the problem can be refined to express its domain and components. Methods can come at the problem from deductive (general sample to prove a point/theory) or inductive (single case generalized).

When writing up research, there should be a literature review (shows where in the literature there is material missing) and a problem statement (What is/How does/Why does/What good is?). The literature review should essentially point to the problem statement, showing why it is necessary to answer it.

Anderson Ch. 4

After establishing the problem statement, the researcher must move on to method. Empirical methods use the observation of real practice/culture/material in order to build arguments. It can be categorized as either descriptive, field/life-world, or laboratory.

Metric empiricism is concerned with the measurement and observation of variables. It can be accomplished through observation, survey methods, and experimental methods. In some cases, they can work in tandem with other types of research. Interpretive empiricism is aimed at the study of human action, and it includes reconnaissant observation (short-term), participant observation (long term), critical ethnography, and performance ethnography.

There are certain methods that blur the distinction between metric and interpretive empiricism, including textual analysis, discourse analysis, cultural-critical analysis, and dialogic analysis. These are different from mixed methods, which include multiple methodologies from both metric and interpretive methods.

Anderson Ch. 5

Data must be accompanied with an argument because, otherwise, it is largely useless. Before data can be used, the researcher must implicitly make several assumptions, including what constitutes the media/our text and the nature of the audience/public. We also make assumptions about culture (where and how it is constituted), the way in which individuals relate to society and society to culture, and the role or existence of agency.

Once assumptions are taken into account, a theory must be chosen. The major categories of theory are cognitivism (behavior is dictated from within the individual), social action (behavior is influenced from outside forces), psychoanalytical-semiotic (understanding, meaning, and discursive methods), and critical issue theories (Marxism, feminism, race studies). Each area of theory can be addressed to the psychological, sociological, or cultural level, and furthermore, they can target topics falling into media, content, message, audience, or performance. These theories fall into major epistemic categories as well, including Cartesianism, modernism, and postmodernism.

Depending on whether you are using metric or interpretive methods, the appropriate engagement, evidence, claim, and warrants for trustworthiness are going to differ. In metric work, evidence is characterized by measurable observations of variable manipulation. Trustworthiness is phrased in measurements of reliability, precision, accuracy, and validity.

For interpretive work, evidence is based on the researcher’s coherent and resonant narrative. Their ability to do so will be based on their degree of observation and participation in the studied population. They must be able to link warrants to all of their claims, demonstrating why their evidence is sufficient for the proposition. Trustworthiness for interpretive scholars comes out of coherence, resonance, and vraisemblance (the recognition of the narrative as plausible).

Anderson Ch. 6

Once researchers have research questions, they need to be able to translate their work into hypotheses. For each type of question (What? How? Why? So what?) there are appropriate forms of response. Hypotheses usually appear in metric work, using the theoretical constructs appropriate for the measures and methods of the study. They are not nearly as common in interpretive work.

It is a good idea to create research questions that are anchored both in the real world and the academic, theoretical world. This helps establish warrants for the research. Community-based research is one way to get at both types of problems.

All research work that deals with subjects rather than solely texts should anticipate working with the IRB. The board assesses whether the work is competent scholarship (whether they can really find out what the study sets out to find). They also are concerned with informed consent of participants.

Hammersly & Atkinson's Ethnography, Ch. 1 - 10

Hammersly & Atkinson Ch. 1

Positivism includes the assumptions that research should follow experimental guidelines, it should produce universal or statistical laws, and all knowledge is rooted in observation.

Naturalism refers to the belief that knowledge comes from the study of the social world in its natural state—without researcher interference.

Both positivism and naturalism lack reflexivity; they rely on situated observation but do not acknowledge the role of the researcher. In ethnography, reflexivity is the basis of knowledge.

Hammersly & Atkinson Ch. 2

In ethnography, researchers usually enter with some degree of foreshadowed problems, but they need to be careful not to let their ideas of problems define what they observe in the field. Furthermore, entering the field in hopes of testing a particular theory can be problematic for the same reason. As the researcher spends time in the field, they will most likely change or refine their research questions, being sure to pursue what is already there, not what they are trying to see. Part of this work can be described as either moving from the formal to substantive (theory -> observation) or substantive to formal/generic (observation -> theory/generalization).

The setting of the research also plays a large role in deciding the research question. It is a good idea to check out places being considered for an ethnography before choosing settings. After all, there are usually many settings to choose from to address the kinds of questions that you want to address. Ethnographers do not, technically, study settings—their work is in studying the populations in the settings. It is impossible to account for everything within the setting.

Sampling in ethnography refers to the choice of cases and cases within cases in a study. Sampling should aim to be representative of the culture studied. The act of sampling is the implicit choice of what is and is not relevant to the topic studied. Sampling can be decided on the basis of time (when to sample), people (who to sample), and context (where to sample).

Hammersly & Atkinson Ch. 3

Access is a major issue that ethnographers have to deal with. It can involve having to create or use interpersonal relationships with people of power in order to be able to visit or observe certain sites. Even the act of negotiating access can teach the ethnographer a great deal about the culture. Access also brings up issues of ethics (who can grant access isn’t always who ought to grant access) and behavior (just because you are present does not make you a welcome part of the scene).

The people who have the power to let you into a particular setting or scene are known as gatekeepers. They can have formal (clearance) or informal (integral member of culture) roles that give them this power. Sometimes gatekeepers are not the sole individuals needed to gain access to a site. They can be constrained by other individuals, rules, or laws.

When researchers cannot gain access to a field through gatekeepers, it is sometimes possible to do covert research in which the study is not announced to anyone. This can be a constraint on the research, limiting the kinds of data collection you can do. It is also a significant ethical issue.

Hammersly & Atkinson Ch. 4

For most participants, the ethnographer will initially be treated with suspicion, but this will dissipate as contact increases. However, this is not always the case. Sometimes participants misjudge the intentions of the ethnographer or assume that they are ignorant of the practices of the culture. As a result, the researcher has to engage in impression management by doing things like dressing like the people studied. Expertise and specialist knowledge can be useful in structuring relationships with participants in the field. Researchers must also decide on what level of self-disclosure is appropriate and whether or not they are trading friendship for data.

Researchers should be aware of their unmanageable characteristics (gender, age, ethnicity, etc.) that can have effects on how they relate to people in the field. Women can have trouble gaining access to certain areas (either because of actual rules or just established norms), and men can have trouble gaining access to settings and situations usually reserved for women. By the same token, these characteristics can make it easier to gain access to those areas that privilege such characteristics.

The early stages of research often involve learning the basic competencies for participating in the culture. During this time, the researcher is often granted the same grace as any new member of a culture—“acceptable incompetence.” This leads to one of the problems of studying a culture that you are already a part of—you cannot hold a novice position, and thus you miss all of the knowledge learned about the status of being a novice.

Hammersly & Atkinson Ch. 5

Because the ‘objects’ that ethnographers study are actually ‘subjects,’ collecting accounts becomes a hugely important part of research. Accounts can be both unsolicited and solicited. The former is generally thought to be more natural or more purposeful, which could be considered a naturalist perspective. We cannot be too heavily concerned with the influence of the researcher asking for accounts because there is no way to ever know if the data is compromised, including unsolicited accounts.

The act of choosing who to interview can have several effects on the research. Simply choosing those that are interesting to the researcher can be, in some ways, biased. Using a gatekeeper to select the respondents gives them the power to direct the results. Allowing the participants to self-select guarantees that the only perspectives heard are those that would want to be heard. The interview itself can be seen as a new site of participation—a “resource rather than a problem.”

Interviews are structured both by the ethnographer and the respondent. Using non-directive interviews requires the researcher to be an active listener, following up on important or interesting details to make the interview worthwhile.

Hammersly & Atkinson Ch. 6

It is important to pay attention to documents, physical objects, and various modes of communication. They can provide a source of information that does not come out in interviews or participation about the setting and context of the observation. They can also be a starting point for analytic ideas. Documents can be either solicited or unsolicited. The former can take the form of diaries or written accounts.

Hammersly & Atkinson Ch. 7

Researcher should try to record as much as they can. Memory is faulty, and it should not be the basis for analysis. Recording anything means that the researcher is choosing not to record something at the same time. It is selective, and something will always be left out. This alone is not reason to abandon recording data. This means that there will also always be interpretation, because recording cannot capture every bit of meaning attached to anything. Choosing to record can also have effects on the participants, and we need to account for this in our interpretations.

Ethnographers should use a reflexive eye when considering recorded data. Recorded data can be useful, not just for capturing specific data, but for use in determining what is usual or unusual in the field or for testing out conceptual links. The recordings can also provide information about the researcher and how the participants react to the researcher.

Hammersly & Atkinson Ch. 8

Data analysis occurs at every stage of ethnographic work. However, there are no guaranteed ways of doing data analysis that will make an ethnography successful. Using data to theorize should be an iterative process, emerging as analysis occurs.

Ethnography work can be largely descriptive, using techniques such as making the strange familiar/making the familiar strange. It can also attempt to explain or create theoretical models. In analysis, ethnographers should search for concepts or theories that help make sense of the cases in the data. One way of analyzing unstructured data can be to use theoretical concepts as categories and to begin sorting data into those categories. As this work continues, the researcher can find data that calls for new categories or that strikes interesting comparisons with existing categories. This can also lead to the establishment of relationships between constructs.

Other systems for working with data can be creating conceptual maps, working up grounded theory, and creating typologies. It is important to find concrete indicators within the data that link to the concepts you are using to theorize. The links can take the form of social context, time, or personnel (who does/says what). When finished, the researcher should seek respondent validation. The conclusions should seem correct/intuitive to those within the population studied.

Hammersly & Atkinson Ch. 9

Ethnography is produced through writing, and the production is just as much writing as it is the data collection. Depending on how the ethnography is written up, it will emphasize different facets of the study. The particular theories or concepts used in the ethnography will dictate what should be focused on. The ethnographer generally writes as a storyteller; because experience is data, the experience must somehow be conveyed. The writing will be read by an audience, and so the narrative must be compelling (or at least readable).

Writing up an ethnography can take either thematic or chronological arrangements. The former is useful for explaining constructs that are ordered in term of importance. The researcher should be careful when using metaphor and synecdoche. While they shouldn’t be eliminated, they can be used in ways that mislead or over-simplify. They should also consider the role of narrative, irony, and topos. When appropriate, or when writing for a digital form, the ethnographer can include extra data such as photos or recordings, though doing so may raise certain ethical issues.

Hammersly & Atkinson Ch. 10

There are many issues that ethnographers have to make ethical choices about. These can include informed consent (how/to what degree does the ethnographer attain consent from participants?), privacy (how is privacy guaranteed and where is it expected?), harm (does the ethnographer do any harm either in their participation or publishing of information?), exploitation (to what degree is the ethnographic work exploiting a community, and can the ethnographer do anything to give back?), and consequences for future research (does the ethnographer’s work preclude further studies?).

NVivo Experimentation

Since the last NVivo update, I believe I've gotten video coding figured out. I finished coding the OUYA promotional video that I was working on, and I've started working on a few unboxing videos. What I'm finding interesting is that when working with both texts that come from an official source and texts that come from an audience, there are certain phrases that belong exclusively to each domain. Only the audience is concerned with pirates; only the officials at OUYA are concerned with fundraising. I'm also observing instances where phrases are used in the official OUYA texts and are later replicated by the audience.

Seeing the movement of language from one population to another makes me think that a useful feature for NVivo to implement would be a visual timeline. This way, you could see how certain coded features get picked up and rise or drop in popularity of usage. In a previous project, I'd worked with internet memes which get picked up and combined with other memes as they gain popularity. I would have loved some way to visualize the different types of memes and their frequency of usage over time. It would really help in longitudinal rhetorical projects.

I also have to admit that I'd pulled up NVivo earlier in the week just to have fun-- a friend of mine was talking about the backlash against Sony that happened after they revealed that they were not implementing some of the media features on the PS4 that users had become accustomed to on the PS3. I captured the comments on the press release and ran them through NVivo, having it display the most frequently used words. Just by itself, the analysis was interesting because you could spot the most controversial features easily and the anger of the fans was easily spotted as well.

This project, too, led me to a problem-- the comments I captured were on a press release, and every time I switched to a new page of comments, the press release remained at the top of the page. This means that, even though I captured 12 pages of comments, I also captured the text of the press release 12 times. Short of manually editing that text out, I don't know how I could selectively capture only some text on a website OR delete that text within NVivo. This meant that any kind of automatic analysis would count the wording in the press release twice as opposed to each individual comment once. I'm still not entirely sure what to do with this.

Outside Reading: Park and Choi's The Internet as a Political Campaign Medium

Park, H. S., & Choi, S. M. (2002). Focus Group Interviews: The Internet as a Political Campaign Medium. Public Relations Quarterly, 47(4), 36–44.

In this article, Park and Choi conducted focus groups in order to understand the effect that websites have on perceptions of political candidates. In particular, they were interested in whether interactivity on the website translated into a feeling of having interacted with the candidate.

The study was conducted in 2000 during the Bush/Gore election. The participants for the focus group were undergraduates who were in advertising classes, all age 21 or older. For the study design, they asked participants to look at the websites of Bush and Gore for half an hour in a computer lab before convening the focus group. Once there, participants were asked questions about their experiences with the websites. The questions were formulated with the intent of using Media Richness Theory in the study.

From a methodological point of view, one of the problems in this study is that the researchers specifically chose to have participants look at the websites for a certain amount of time. While this can be appropriate if you are looking for the baseline effectiveness of some kind of campaign material, it does not seem entirely useful for the purpose of their research. When trying to make generalizable claims about how the voting public is affected by candidate websites, you have to take into account that many are not affected at all, and even further, that those who are affected by the websites sometimes spread that information to others. It is possible to have secondhand knowledge of a website. By priming their participants, Park and Choi lost all possible data about whether their participants had seen or heard about the websites at all. It seems like it may have been useful to first distribute a survey or to have participants look at the websites partway through the focus group.

All of this is not to say that Park and Choi’s study does not have merit. Having been done in 2000, the study fell at a particularly valuable juncture when the web components of political campaigns were first being explored. Park and Choi describe the responses of participants to both Bush and Gore’s websites; those who liked Bush’s website appreciated its simplicity and those who liked Gore’s website appreciated how well it covers all relevant issues. At this point in digital media development, we know that both approaches are extremely valuable. Though they eventually conclude that interactive features on websites are not perceived as the same as interacting with the candidates, they do identify early efforts at personalization as being useful (which were later used to great effect by the Obama campaign).

Saldana Ch. 5 and 6

Saldana's The Coding Manual for Qualitative Researchers

Ch. 5: Second Cycle Coding Methods

The Goals of Second Cycle Methods
The second cycle of coding is the point at which the myriad codes created in the first cycle get refined into larger codes or categorical labels. The idea is to begin finding trends or themes within the data which can eventually be assembled into a theory or model. We have to be careful that our categories are not too disparate; if they are, we are probably misrepresenting some things because data will not tend to hold such variety without overlap.

Second Cycle Coding Methods
Pattern coding: a meta-code that pulls the existing codes into a smaller number of codes or categories
Focused coding: looks for frequent or high intensity codes to identify salient features of the text
Axial coding: reassembles split data; looks for dominant and subordinate codes to create a categorical system that specifies the properties and dimensions of categories
Theoretical coding: Identifies a core category and formulates codes that relate the data to that category; can be theory-driven or emergent
Elaborative coding: Uses categories/themes from a previous study to interpret the data; strengthens the theory by seeing how well the prior coding set explains the new data
Longitudinal coding: Used for life-long studies; notes increase/decrease in study variables, accumulations, epiphanies, idiosyncracies, constancy, and missing data

Ch. 6: After Second Cycle Coding
It can be difficult to transition from fully-coded data to making theoretical claims. There are many different formats that research can take, so Saldana focuses on solidifying ideas.

Focusing Strategies
The "top 10" list: Use the 10 most interesting or varied pieces of data to illustrate the salience or breadth of your research
The study's "trinity": Identify the three most important concepts in your research and figure out the relationship between the three
Codeweaving: Using the major codes in your research, tell a story that structures the relationships between the codes
The "touch test": Look at your codes and find those that are things-- what can be touched? These codes need to be abstracted a level (mother to motherhood)

From Coding to Theorizing
Elements of a theory: Identify the if/then components or relationships within your coding
Categories of categories: Place your existing categories into larger categories

Formatting Matters
Rich text emphasis: Bold major concepts and italicize important assertions
Headings and subheadings: Use headings and subheadings to structure an argument

Writing About Coding
It is helpful to include as much information about the methods you used as possible. This includes everything from the data acquisition and sampling to the computer programs you used to code and manage the data. Make sure to emphasize major outcomes of the analysis.

Ordering and Reordering
Analytic storytelling: Tell the story of your analysis in a chronological way
One thing at a time: Write about concepts or categories one at a time, keeping them separate initially
Begin with the conclusion: If having trouble writing, begin with the conclusion

Assistance from Others
Peer and online support: Have peers (in person or online) look at your work and provide suggestions
Searching for "buried treasure": Have readers of your work look for important ideas or assertions that are not explained

Anderson, Ch. 1

Anderson Notes, Media Research Methods, Ch. 1

Communication domains include mass communication (study of messages sent to mass audiences through media technologies), media studies (media literacy), and mediated communication (all forms of communication that use some kind of mediation; anything but face-to-face).

Methodologies are standardized ways of producing knowledge. Empirical methods are concerned with experience and materiality, though not necessarily quantitative knowledge. Metric studies are quantitative, applying a logical framework to studied variables. Hermeneutics forms an alternative to metric study, systematizing interpretation for qualitative inquiry. Critical methods are used for revealing frameworks and systems of power and control. These areas of scholarship overlap in certain places, creating epistemologies. Metric empiricism describes material reality through measurement, with the assumption that reality is stable and objective. Interpretive empiricism is also concerned with material reality, but it is accessed through lenses of interpretation that reject objective reality. Critical-empirical scholars are interested in applying critical/cultural insight, but reject the singular truths or narrative put forth by earlier critical scholars, instead choosing to employ grounded theory with a critical agenda. 

Outside Reading - Kazmer & Xie's Qualitative Interviewing in Internet Studies

Kazmer, M. M., & Xie, B. (2008). QUALITATIVE INTERVIEWING IN INTERNET STUDIES: Playing with the media, playing with the method. Information, Communication & Society, 11(2), 257–278. doi:10.1080/13691180801946333

Kazmer and Xie’s “Qualitative Interviewing in Internet Studies: Playing with the Media, Playing with the Method” is a methodological meta-analysis of internet-mediated semi-structured interviews. For the purpose of the study, they divide interviewing methods into the following groups: face-to-face, over the telephone, through email, and through instant messaging (IM).

One of the interesting points that Kazmer and Xie bring up is the idea of contextual naturalness—in an interview, the participant should be able to express themselves the way they want to. For some people, internet interviewing can hinder this; for others, it is an ideal environment. Like Tracy’s discussion of internet interviewing methods, Kazmer and Xie mention the difference between synchronous and asynchronous environments; this provides the conceptual difference between email and instant messaging. Whereas email is generally asynchronous, IMs are meant to be real time. I would possibly argue that IMs provide a level of asynchonicity anyway. After all, in an IM, you are still free to take your time, compose and recompose.

A common thread in analyses of internet facilitated interviews is that participants have a tendency to disappear or miss scheduled interviews. It is difficult to schedule synchronous interviews online and asynchronous interviews are more likely to be forgotten by participants.

Another point that Xie and Kazmer note about online interviewing is that there are certain technological considerations to be made when saving or recording interviews. There are only so many things that actually can be recorded, and the internet strips the situation of most environmental and visual data. Also, when doing online interviewing, both the participant and the researcher are left with complete records of their interactions.

After interviews are conducted, it is up to the researcher to decide how to assemble and clean data. Often email and IM interviews can include attachments—photos, documents, or other files—and there is a question of how and where to include these materials. IMs are also problematic because they are susceptible to conversational disorder—where questions are asked and answered out of order.

Despite the drawbacks, Kazmer and Xie have found that internet interviewing is becoming increasingly common. It allows for a wide participant base, and it is easier to find interviewees, generally. The problems they note are not insurmountable, and it is up to the researcher to decide how to represent all data.

Saldana Ch. 3

The Coding Manual for Qualitative Researchers, Ch. 3

First Cycle Coding Methods

The first cycle of coding is the initial coding of data. Saldana identifies seven types of first cycle coding: grammatical, elemental, affective, literary, and language. Each of these types has several particular coding methods associated with it. The second cycle of coding is characterized as more analytic, and it includes classifying, prioritizing, integrating, synthesizing, abstracting, conceptualizing, and theory building.

Choosing the right type of coding is dependent on several factors, and the choice should be guided by your research question, your paradigmatic and methodological choices, and exploratory work with your data set. Saldana suggests a generic coding method for figuring out the right type of coding for your data, begging with attribute coding and followed by structural/holistic coding, descriptive coding, and in vivo, initial, or value coding.

Grammatical Methods

Attribute coding: Used for nearly all studies, attribute coding goes before the text and lists all relevant attributes to the data set, including qualitative and quantitative properties.

Magnitude coding: This adds an alphanumeric measure of intensity to another coding scheme

Subcoding: Creates a secondary code to accompany a primary coding system. The subcode is hierarchically below the status of the primary code.

Simultaneous coding: Utilizes a second code of equal standing or importance to the primary code, at the same time on the same texts

Elemental Methods

Structural coding: Codes related to questions asked during interviews/recurrent topics from participants; especially useful with large numbers of participant responses following a similar structure

Descriptive coding: Creates codes related to the topic of qualitative data. This is different from codes referring to the content.

In vivo coding: Uses the language of the data to code instead of labels chosen by the researcher

Process coding: Codes through use of gerunds—what the language is doing conceptually or what an observed participant is physically doing

Initial coding: Open-ended approach that breaks data down into parts and compares them; iterative

Affective Methods

Emotion coding: Labels emotions experienced or recalled by participants

Values coding: Codes values, attitudes, and beliefs expressed in participant responses

Versus coding: Codes for binaries, dialectics, and rivalries

Evaluation coding: Used to evaluate programs, evaluation coding makes value judgments (non-quantitative) and attempts to describe, compare, and predict success from a program

Literary and Language Methods

Dramaturgical coding: The application of dramatic elements to qualitative data (not just Burke’s pentad), drawing from Goffman and impression management

Motif coding: Uses index codes for classifying folk tales, myths, and legends; the motif is the smallest unique unit in the story

Narrative coding: Open form of coding whatever the researcher considers a narrative; done from a literary perspective

Verbal exchange coding: Aims at finding a generic form of conversation. Codes precise transcripts of conversations as phatic communication, ordinary conversation, skilled conversation, personal narratives, and dialogue.

Exploratory Methods

Holistic coding: Looks for themes and issues by taking in qualitative data as a whole rather than analyze line-by-line.

Provisional coding: The creation of a set of codes before analyzing the data based on existing knowledge of the texts

Hypothesis coding: Coding done with a predetermined set of codes aimed at testing a particular hypothesis

Procedural Methods

Protocol coding: Protocol coding occurs when all coding is done by a rigid, preset system rather than an open or iterative process

Outline of Cultural Materials (OCM) coding: Coding that follows the OCM, a topical index for anthropologists and archaeologists

Domain and taxonomic coding: An ethnographic type of coding, domain and taxonomic coding looks for cultural knowledge; it separates processes into steps, looks for cultural categories, and identifies semantic relationships (strict inclusion, spatial, cause-effect, rationale, location for action, function, means-end, sequence, and attribution).

Causation coding: Looks for causal beliefs in qualitative data; notes dimensions of causality, including internal/external, stable/unstable, global/specific, personal/universal, and controllable/uncontrollable

Themeing the Data

A theme is a phrase or sentence that identifies what a unit of data is about or means. Creating themes means finding groupings of implicit ideas among the coded data. 

NVivo Experimentation 2

For this week, I decided to use NVivo to put together my weekly example for my online Communication Criticism course. When I've taught the course in person, it has been easier to teach coding because I can just give everyone a paper copy of some kind of text, and they can code by hand. In the online environment, it's been harder to do this-- all of my visuals are either static images, blocks of text, or animated PowerPoints. I can't just explain coding and hope that they try it on their own.

This upcoming week, I'm teaching Narrative Criticism. By using NVivo, I can go ahead and code several texts by a single author, and when it comes time to create the example, I can use a screen capture program to show my students coding and some of the data outputs. I don't expect them to use NVivo themselves, but it's a more coherent way to show coding than trying to use a multi-color highlighting scheme in Word.

The project I created includes three stories by David Sedaris that have been broadcasted on This American Life. Following Sonja Foss's explanation of Narrative Criticism, I am going to code the texts for theme; I have chosen to focus on separation, loss, and lack. I have also run word frequency analyses and generated a few other outputs (I like making visuals).

The project file can be found here.

Tracy Ch. 8

Qualitative Research Methods

Interview Practice: Embodied, Mediated, and Focus-Group Approaches

Negotiating access for interviews

Getting participants to agree to an interview is extremely important. Some types of people are more likely to agree than others, particularly when the interview is about sensitive topics. Researchers should try to start a relationship with the person they want to interview before asking for the interview. This increases the chance that the participant will agree. After an interview is agreed upon, the researcher should send a reminder to the participant that includes information on how to contact the researcher if something goes awry on the day of the interview.

Conducting face-to-face interviews

When scheduling an interview, it is important to choose locations that are accessible and public, but provide a degree of privacy. They should feel comfortable and safe to the interviewer and interviewee. Recording devices should be tested out beforehand, and locations should be chosen that are quiet enough to use a recording device. When the participant arrives, you should offer to get them a drink or something to eat (if available). It is also useful to take site notes and cursory information about the participant before the interview begins. Scheduling should be done in a way that leaves time for briefing and debriefing.

A good interviewer does more than simply ask questions-- they should prepare themselves to be knowledgeable about the topic at hand and the participants; they should be gentle, forgiving, sensitive, and open-minded; they should structure the questions to be probing and show  their attentiveness; and they should actively interpret what the interviewee says. By repeating certain ideas back, interviewees are able to correct interpretations or provide more nuance. Researchers should be mindful of their non-verbal communication and refrain from anything that would connote judgment or shock. At the end of the interview, researchers should be appreciative. It is especially useful to shut off any recording device and ask if there is any information the interviewee is willing to share now that the recorder is off. When the interviewee leaves, the researcher should take notes on anything that was notable that was not captured by the recorder.

Technologically mediated approaches to interviewing

Technologically mediated interviews can be synchronous (real time) or asynchronous (reply on own time). There are significant advantages to online interviewing-- it is cost effective and can reach a wide participant base, it can increase engagement, it can be more flexible, it can change power dynamics that are present in face-to-face communication, it can provide data that is unavailable face-to-face (grammar, typing, deaf or heavily-accented participants), and it does not need to be transcribed.

There are also disadvantages to technologically mediated interviews. They provide no nonverbal data, and they require participants to have access to a computer as well as the relevant technological skills. Participants can more easily become distracted while interviewing or they can drop off of the project entirely. There are more confidentiality risks on both the part of the interviewer and the interviewee, and it is nearly impossible to verify identities.

The focus-group interview

Focus groups traditionally bring together 3-12 people who are taken through a guided group discussion. Focus groups can be particularly valuable, especially when they are able to create a group effect-- where responses become chained, and members of the group leave with a new understanding of the topic through their interactions with others. It also provides a space to observe vernacular language and communicative interactions. The focus group can serve as a way to gather creative data; researchers can ask participants to rank concepts, draw or create pictures or collages, create metaphors, or act out scenarios.

Members of a focus group should have a key characteristic in common. It can be valuable to have oppositional points, but the group can be hard to manage if there are not meaningful commonalities. On the day of a focus group, you should provide comfortable accommodations for the participants, and before things begin, you should provide them with informed consent and make it known that the event will be recorded and/or observed.

When moderating, it is important to know the discussion guide so that the discussion can be moved in the correct direction. You should not take notes, but rather focus on providing positive and supportive feedback. This can also include paraphrasing and asking for clarification. It is important to provide breaks every hour-- during this time, natural conversations can arise. Closing the focus group should consist of reminding participants to keep the conversations confidential. Afterwards, you should take notes on important details and make a to-do list for the next focus group.

Overcoming common focus group and interviewing challenges

Some challenges that arise during interviews can come from both the interviewee and the interviewer. The interviewee can espouse unexpected behaviors that make it difficult to schedule interviews or record them. Interviewers can talk too much, interrupt, or use problematic formulations-- interpretations of the interviewees responses that gloss over or omit certain information for the sake of simplicity. They also have to be aware of any problems in the interview guide and be mindful of tangents. Some researchers have problems with probing and creating follow up questions; they should aim to ask for clarifications, deeper meanings, or greater detail. When participants become emotional, it can be difficult to respond. Tracy advocates trying to identify  with or show compassion to the participant. Researchers are also often unaware of what to do if an interviewee is deceitful. To combat this, you can build redundancy into the question guide or refer to others with whom you've spoken. Not all lies are useless, either, because often it can be strategic.

Transcribing

There is no universal transcription method or set of symbols-- these are constructed by the researcher and meant to fulfill whatever purpose they are put to. The choice of what details to note comes down to what the study is examining. More detailed is not always better. It can take quite a long time, and the more people that are speaking, the longer it takes. Transcriptionists have to make choices about things like punctuation, laughter, pitch, tone, and homonyms. Fact checking transcripts requires that you listen to the interview while reading the transcript to make sure that it does not omit (or add) anything. Voice recognition software can also be used.

Tracy Ch. 7

Qualitative Research Methods

Interview Planning and Design: Sampling, Recruiting, and Questioning

The value of interviews

Interviews should be thought of as guided conversations, and we need to remember that they are not always face-to-face, one-on-one. Interviews can occur through various media and can involve small groups as well. The goal is to access subjectively lived experiences and viewpoints related to a particular topic. Because the interviewer has power in the situation, it is up to them to treat the interviewee and the resulting data ethically. Interviews should be seen as a process of mutually creating a story or meaning, and so the process of conducting an interview is just as important as the data collection.

Interviews provide the unique ability to gain access to motivations, thoughts, and reasoning that would otherwise be unaccounted for. Furthermore, they can provide information about things that cannot be otherwise observed-- private or past experiences. Interviews are also a good opportunity for getting more information about observations; they can act almost as a member check.

Who, what, where, how, and when: Developing a sampling agenda

The sampling plan or sampling agenda is a proposed way to choose sources for interview data. In qualitative work, most researchers use purposeful sampling, which chooses participants and data that will work especially well with the goal of the research project.

Random samples - Every member of a population has equal chance of being interviewed (or otherwise have data collected).
Convenience/opportunistic sampling - This type of sampling is very common, and it involves using participants that are convenient to the researcher. It is often quicker to use this type of sampling, although it is not often considered enough data for a full study.
Maximum variation samples - When there is significant variation within a population, researchers may choose to portray this variation through purposefully choosing participants from the far ends of the spectrum.
Snowball sampling - This type of sampling is good for reaching difficult-to-access or hidden populations. It begins by choosing a few participants and then asking them to suggest friends, family members, or colleagues who may be open to participating.
Theoretical-construct samples - When working with a theory that requires participants to meet certain goals or have certain characteristics, theoretical-construct sampling purposefully chooses those people.
Typical, extreme, and critical instance sampling - Typical instance sampling looks for average participants; extreme instance sampling looks for participants who exhibit a chosen characteristic in the extreme; and critical instance sampling requires participants that have experienced something in particular (which may be extreme).

A sampling agenda can often be determined through your research question and chosen theories. If doing participant observation, it is often a good idea not to set a strict sampling agenda very early-- doing so can restrict your ability to observe what the field has to offer.

Interview structure, type, and stance

Structured interviews are those that repeat the same questions for each participant. This is often done with the help of an interview schedule. These are most useful when using a large sample that needs to be analyzed. These types of interviews may be good for quantity, but their depth suffers.

Unstructured interviews are far more flexible. What structure they have is provided by an interview guide-- it goes through required topics, but not specific questions. Unstructured interviews require interviewers to be more skilled and empathetic; they need to be able to pick up on emotional cues and keep the conversation on the research topic.

Types of interviews: ethnographic (informal, emergent, instigated by researcher), informant (interviews with those who are savvy in the scene, requires long-term relationship), respondent (group of participants with similar subject positions, speak only for themselves), narrative (open-ended, participants tell stories, related to oral history), life-story or biographic (includes stories, asks about life as a whole, provides understanding or empathy), and discursive (focuses on structures of power and their effects on knowledge, asks how participants navigate power).

There are multiple stances for the interviewer to take. Deliberate naivete falls between overt honesty and deceit, with the interviewer losing any presuppositions and judgments. Collaborative or interactive interviewing places the interviewer and participant on the same plane, with each interviewing the other. Pedagogical interviews ask for expert knowledge or emotional support. Responsive interviewing "suggests that researchers have responsibilities for building a reciprocal relationship, honoring interviewees with unfailingly respectful behavior, reflecting on their own biases and openly acknowledging their potential effect." The friendship model is similar to this, coming out of feminist scholarship. When interviewers deliberately provoke the people they are interviewing, the technique is called confrontational interviewing.

Creating the interview guide

Interview questions should not use jargon or overly-complicated language. They should focus on one topic at a time, and they should be open-ended. Questions should also be neutral and not lead to particular answers. They should uphold the participants' identities, and they should be followed by appropriate probes for further information.

Interviews should be sequenced as follows:
Opening - Breaks the ice, sets expectations, and begins with experience-related questions.
Generative questions - Questions that generate frameworks for the interview.This can ask for descriptions of experiences or processes that act as tours. They can also include examples or timelines. Hypotheticals can also work here. Behavior and answer questions can use past experiences to ask for factual reports of behavior. It is also productive to ask about participants ideal visions of certain things. This is also the part of the interview to ask participants to compare and contrast things, ask about their motivations (or others' motivations) and for predictions about the future.
Directive questions - Direct questions call for specific type of answers, and these direct the trajectory of the interview. Close-ended questions require very specific answers. Typology questions can ask for respondents to categorize certain knowledge, and elicitation questions prompt discussion by showing some kind of stimuli (video, image, text, etc.). Data-referencing questions ask participants for commentary on data gathered by the researcher in the past. Member reflection questions can be used to get participants' opinions on observations made by the researcher, and devil's advocate questions make participants respond to skeptical positions. This is also the point to use what may be potentially threatening questions. They are best left for the end of the interview in case they somehow effect the participant.
Closing the interview - At the very end, it is appropriate to ask catch-all questions and questions that enhance the identity or self-perception of the interviewee. You can also ask at this point if the interviewee has a preferred pseudonym.

Working with NVivo10

I decided to start working on a project for another class-- Rhetorical Theory-- with NVivo 10 to organize my sources. For this project, I'm looking at the rhetoric surrounding the OUYA gaming platform, including the rhetoric of creators, users, and the console itself (understood as user script).

I began by installing the NCapture extension to my browser and capturing the OUYA Kickstarter page as a .pdf. I was able to open this in NVivo, and I coded it for several different recurring themes-- revolutionary language (anything that described shifting power balances), money/economic language, the ideas of hacking (changing the system against the creator's specifications) and experimentation (changing the system in accordance with the creator's specifications), and the system's relationship to other gaming and technology companies. Coding this wasn't difficult once I found the options to highlight and create colored ribbons. I think I need the visual feedback so that I know my codes are being logged and counted.

One of the first problems I encountered was with the video embedded in the Kickstarter page. Kickstarter uses a video player that is not recognized by NCapture or my installed video capture extension. I was able to find the same video on YouTube, and before I realized that NCapture could take YouTube videos, I downloaded it and converted it to AVI myself. I imported the video, but I am having trouble coding it. The pause/play/rewind features are also a bit clunky, especially when you want to capture things that only last a few seconds. I'm definitely more used to the keyboard controls found in Final Cut software, Quicktime, and other applications.

For the time being, I put the video coding on hold when I figured out that I could capture all of the comments on the YouTube video I downloaded. I like that NCapture stores these like a database and natively supports the thread structure (reply format). Strangely, NVivo is showing me a different number of comments available than YouTube does, but there might be discrepancies because the comments on YouTube are split into two different pages. Whenever I try to capture these pages separately, I get the same data when I import it into NVivo. I'm not sure what's happening there, but since I'm getting about 690/730 comments, I'm not hugely concerned about the missing 40. I'll still be playing with this in hopes of figuring out just what's happening.

So far, I have coded about 200 of the available comments. I had been coding words, phrases, and fragments of sentences, but I suspect that the better way is to code comments as a whole. That gives a far more unified unit of analysis. If I did that, I would then just add a code for "unrelated," which would get used pretty often in this particular thread of comments. It is, however, already noticeable that the focus of the YouTube commenters is vastly different than the focus of the video or the OUYA Kickstarter page.

From here, I intend  to work harder on figuring out video coding. I would like to eventually record some captures from the OUYA itself (I own one), which I can code as user scripts (in other words, what the device encourages users to do and what is discourages). That work would almost have to be done as video. I can also foresee coding images, because the packaging of the system includes some interesting language use as well.

My project file can be downloaded here.

Tracy's Qualitative Research Methods, Ch. 6

Tracy’s Qualitative Research Methods

Ch. 6: Field Roles, Fieldnotes, and Field Focus

Field Roles and Standpoints of Participant Observation

            Whereas once, the idea of “going native” (participating too fully in the scene) was considered a bad thing, modern qualitative researchers believe that this is the only way to fully understand the emotionality of participants. The phrase, which has colonialist undertones, assumes that researchers must remain detached and objective, a perspective that is generally rejected by qualitative scholars.  Instead, we ought to consider our place in the scene to be one gauged by degree of enmeshment and overlapping roles. We must ask which standpoint of participation is appropriate for the project, and what kind of data would best maximize outcomes and minimize limitations?

            When researchers study places or groups to which they are already a member, it is called complete participation. This gives more incentive to the researcher to spend time in the field. The availability of data is far greater than in other contexts, and it allows for greater insight into motivations, meanings, and assumptions. Researchers must remember, however, to pursue others’ interpretations as well as their own. If the researcher assumes the values and ideologies of the studied group, they are called an ardent activist. The challenges that come with this type of participation are the ethical problems and the potential feeling of deception by participants. Deception may be necessary, though, when studying contexts that otherwise would not divulge information, such as studying “up the hierarchy.” Being so heavily enmeshed in a culture may also mean that the researcher has a hard time distinguishing unique values.

            If a researcher enters a new culture and improvises or assumes membership in a temporary way, they are called a play participant. They watch and join in, but do so entirely to gain greater understanding of the culture. When the researcher joins in, but remains skeptical, they are called a controlled skeptic. Being a play participant means that the researcher is able to get close to those on the scene, and they go beyond basic reporting. This can sometimes be uncomfortable for the researcher. It is also easier for the play participant to make data recordings, while the complete participant usually does not have time or ability to do so. It is also easier for play participants to take breaks. One of the biggest challenges is in keeping good standing with the participants and making them feel that the research will not harm them.

            The focused participant observer is a researcher who “enters a scene with an explicit researcher status and a clear agenda of what data to gather in the scene.” This approach can include structured interviews, but without long-term participation. The field work with this type of role is highly structured, and it doesn’t last as long as other types of field work. The benefits of this type of work include the well-defined research plan, the low time commitment, and the ability to avoid recurring negotiations of access. It is, however, limited to the data that participants are willing to reveal about themselves, and it takes a necessary etic approach.

            The complete observer is one that is unobtrusive or on the peripheral. They do not participate in the scene. Participants are not aware that they are being studied. A positive characteristic of this is the ease of access. It also is usually granted exempt IRB status. The risk taken in this kind of research is the observer’s detachment from the scene. There is no questioning of the actors, and so it is difficult to gauge motivations or feelings. It is possible that misinterpretations can arise, fueled by ethnocentrism.

Writing Fieldnotes

            Field notes are narrations of the observations taken in the field. The process begins with raw records—the notes jotted down during the act of observing. It is a good idea not to write about what you are seeing as you see it. In some cases, you can bring along a camera or videocam, but this must be included in the IRB application. Voice recorders can also be particularly useful. The method used should be chosen by considering efficiency, reliability and durability, personal comfort, organizational skills, and the way you personally make sense of data. Even if you are not able to take notes, it is possible to get some fieldnotes out of memory. These can take the form of headnotes—“focused memories of specific events, as well as impressions and evaluations of the unfolding project” (Linklof & Taylor, 2002, p. 159).

            Fieldnotes should be written within a day and a half of having done fieldwork. You should try not to talk to anyone about the experience, because the conversation can color your recollections. The fieldnote should be created in a new file with a significant identifying name. It is helpful to put names to the episodes in the header of the document. The header is also a place for location and source of data. The fieldnote should be written quickly, without concern about enforcing a particular style. There should be some kind of organizational element, whether you start from a high point and work around it or write in a chronological fashion. They can also be organized as sketches or episodes. It is a good idea to end fieldnotes with a to do list. There can also be an ongoing cast of characters file in a separate document that keeps descriptions of certain recurring figures. Good fieldnotes should be clear, vivid, detailed, showing (rather than telling), full of dialogue, and complete with your interpretations.

            Lindhof and Taylor recommend that fieldnotes should take up 10 double-spaced pages for each hour of participant observation. Goffman suggests three to five. Tracy notes that it depends, and length is not always equal to quality. If they are too long, fieldnotes can become cumbersome.

            Fieldnotes should be concerned with showing things so that a reader would come to the same conclusion as the writer without being told what to think. Good fieldnotes also elaborate on tacit knowledge. They include descriptions of concrete, sensory detail. It is important to include nonverbal behavior. When you can include sentences or phrases directly from the scene, these are called in vivo terms.

            Another important thing to attempt in fieldnotes is to make the familiar strange and the strange familiar. They should explain things in a way that renews perception. This can be done by pointing out peculiarities about otherwise ordinary situations or familiarities in otherwise strange ones.

            Fieldnotes should also include analysis, which can come in the form of the researcher’s reactions, doubts, prejudices, frustrations, and interpretations of the scene. These can take the form of analytic reflections, which are commentary from the researcher about these feelings. They can be brief and reflective, more elaborate and focusing on specific issues, or more sustained analyses.

Focusing the Data and Using Heuristic Devices

            When researchers start questioning what kinds of data to collect, they should consider visiting the periphery. This can allow you to compare things like the typical to the extraordinary and the peripheral with the marginal. Data can come in three forms: typical/representational, negative/disconfirming, and exceptional/discrepant. Systematization can be facilitated through heuristic models—“conceptual tool kits that stimulate further investigation, learning, and thinking.” These can include processes like examining rituals or cultural scripts. Tracy advocates looking at the following: Space/scene, frontstage/backstage, objects and artifacts, actors and agents, roles and types, activities, interactions, time, goals/purpose, feelings, power relations, values, communication, and processes. As the researcher decides just which facets of the culture they are interested in, they can begin selective observation. 

Tracy's Qualitative Research Methods, Ch. 5 Notes

Tracy’s Qualitative Research Methods

Chapter 5: Proposal Writing

Getting Started with the Institutional Review

            Research proposals are “detailed plans that lay out the purpose, path, and procedures of the project.” Voluntary informed consent is a necessity, and as a result, we have human subject protections. Review boards are mostly governed by the Belmont Report, which is a statement of principles regarding human subjects. It includes requirements for scientifically valid research designs and that the benefits of the study outweigh the risks. Your university’s IRB website will have a great deal of resources, and sometimes this includes training, workshops, and online certifications.

The IRB Proposal: Rationale, Instruments, Informed Consent, and Confidentiality

            The rationale for a study is a primary component of an IRB application. It should explain things like research questions, duration, scope, recruitment, and data collection methods. For qualitative scholars, when asked to list research instruments, they should cite things like the questions they intend to ask in interviews. In the IRB application, it is also necessary to include the way that you intend for people to give informed consent. The particular requirements for informed consent will differ depending on the institution. When working with vulnerable populations, forms of assent are used instead of consent. Forms of consent are still needed from guardians or trustees in these situations, though. IRB may also ask for a letter of permission from an official or group gatekeeper if the group studied is private. Applications must also contain the procedures used to keep information about participants protected. In some cases, data will have to be modified or omitted in order to minimize the chance of deductive disclosure of a research participant.

Different Levels of IRB Review

            Exempt review – This is the quickest review, and it is used for qualitative studies of public behavior. The study must not collect identifying details of participants. It often requires the researcher to only give out an informational letter rather than a full letter of consent. Letters may not even be needed if observations are done from afar.

            Expedited review – This includes the usual application and turnaround takes several weeks. This type of review is necessary for studies that collect names and identifying details about the subjects. An expedited review is also necessary if the data collected could cause criminal, financial, or occupational harm to the participants.

            Full-board review – This type of review is necessary for projects that work with participants that cannot give their consent. Full-board review can take longer than three months.

The Quirks of IRB

            IRB has been criticized for its involvement with qualitative research because members of the board are frequently unfamiliar with the conventions of qualitative work. Due to the emphasis on generalizability by the IRB, some qualitative researchers choose to skip IRB because they are not concerned with making generalizable statements with their research. Not getting IRB backing can be problematic, particularly if something goes wrong with the project. Review is also advisable for any work that may be presented or published. Getting a project through IRB quickly can be done by making sure the project uses well-known and frequently used practices.

Creating the Scholarly Research Proposal

            The success of scholarly research proposals is in their ability to closely adhere to the standards and guidelines of the professor, institution, or agency. Research proposals almost uniformly include titles, abstracts, key words, rationales, research purposes, goals, reviews of existing knowledge, research questions, plans for data collection, and timeline or budget.

            Titles of research projects should communicate the main topic and invite readers to learn more. The title, abstract, and key words should clearly communicate what the proposal is about, and they should stay away from heavy uses of jargon or overly clever or cute conventions. Key words can include methodological terms, the discipline, context, or theoretical approaches.

Introduction and Rationale

            Purpose statement – Explains the primary purpose and goal of the research; should be obvious and explicit.

            Conceptual cocktail party – This is the identification of the various audiences of the project. It should attempt to create a dialogue by citing four or five major people who are influential or tied to the project through scholarly work or other types of contributions.

            Rationale – The rationale answers the “So what?” question. If the work is phronetic, this is an easy task. If it is a theoretical piece, then rational can explain how the study will build on existing knowledge, how it might fill a gap, or how it might bridge various concepts. Simply pointing out a lack of knowledge is usually not good enough.

            Literature review – This is the longest part of a proposal. It shows how the primary concepts and theories have evolved over time, and it frames the study. This can be easier if you adopt the theories that have guided another project—you can mine their bibliography. The literature review should be organized by topic or issue rather than author.

            Research questions – The research question should be closely related to the title, rationale, and literature review. They include the same language that is used within the rest of the proposal.

            Methods – The methods section includes the number of researcher hours, the number and type of participants, the number of pages of data that might be expected, and the way that such data will be collected. This is also the area for explaining specialized words.

            Budget/timeline – This section is not required in all proposals. It includes a list of all the research tools needed and their costs. It is important not to be too conservative when writing this section.

            Projected outcomes – This section is also not required in every proposal. It needs a discussion of possible conceptual or material outcomes, such as helping resolve a theoretical debate or increasing understanding of a problem. Material outcomes can include class papers, conference papers, grant applications, scholarly articles, white papers, class syllabi, strategic plans, or coordination plans.