Fluent Essays

Part A -  As you have learned from your preliminary reading, there is a lot to learn about this research methodology.  There are two basic questions for us to consider this week. First, what is the role of the researcher in qualitative research?  Second, what is truth as the qualitative researcher understands it?   Part B - Module 9 provides rich content related to Qualitative Research.  The resources in the module are extensive.  To demonstrate your understanding of the content, you will create a Study Guide incorporating the major concepts and other details you deem important to assist with understanding what qualitative research is.  The format of the Study Guide is up to you. The Study Guide will be evaluated on its inclusiveness of content and the clarity of presentation.  JTE v9n1 - Choosing Qualitative Research: A Primer for Technology Education Researchers | Virginia Tech Scholarly Communication University Libraries (vt.edu) Four Varieties of Qualitative Notes Adapted by Donald Ratcliff from Corsaro (1981). Entering the childs world. In J. Green & C. Wallat (Eds), Ethnography and Language in Educational Settings (pp. 117-147). Norwood, NJ: Ablex. (Corsaro adapted his outline from A. Strauss) • Field Notes--A running account of what happens or transcriptions of video or audio tapes. It is important to be thorough in taking field notes, particularly at the earliest phases of research; as much as possible, try to get the whole picture of what is happening. • Personal Notes--Personal reactions, how you feel, self-reflection, memories, and impressions. A bit like a diary, so youcan later see your own possible influences on the data and the effects of personal events to the data collection and analysis. • Methodology Notes--Description of methods used, reasons for using those methods, ideas for possible changes in methodology. This is used for keeping track of changes and rational for changes. Include possible and actual adaptations of methods. Can include methods of analysis. • Theoretical Notes--Emergent trends, hypotheses. Also can include guesses and hunches to follow up later in your research. Describe changes made to emergent categories and hypotheses, and the reasons why those changes were made. These four kinds of notes will overlap from time to time. In my own research, I found myself blending personal notes with the other varieties, and thus did not use personal notes for awhile. Later in the research I found I needed a separate category again so I began keeping personal notes again. These notes can be made by hand with pad and pens, but some have found it beneficial to use other media such as small laptop computers, talking quietly into a cassette tape recorder, or using the audio track of a videocamera. The disadvantage of these other methods is the distraction to participants. Some researchers take periodic breaks to go to a separate area and write or type notes--one even used the restroom for this purpose! When typing notes into the computer--either at the scene or when transcribing later--it is good to leave a blank column on one side of the paper for hand-written codes and comments. If you are right handed, leave the right column blank (and vice versa); that way you wont smear the printing with your hand. I found it helpful to use separate files for each day, and separate files for each kind of notes. I also backed up my notes onto a floppy disk every day. Some qualitative research computer programs allow you to add your notes directly to the program, then add coding and analysis later--thus leaving the column blank is unnecessary. I prefer standard word processing programs to qualitative research programs because they are quite flexible and relatively easy to learn (most of us already know a couple or two word processors). But qualitative research programs have their own advantages. Return to Qualitative Research Methods Outline Return to Qualitative Research Resources Logging Data There are two forms of data logged (there could be others): field notes and interview transcriptions It is usually best to write field notes by hand at the site, then type them into the computer at the end of each day or at least by the next morning. In some cases field notes can be done at intervals (if writing them openly arouses suspicion or there are other reasons they cannot be done at the moment of observation). One researcher used the restroom to write notes every hour--which he reported worked well, except that some may have wondered why he had to go there so often! People usually write between five and ten handwritten pages an hour if they are observing carefully. When observing, write very concretely. Quantitative research speaks of operationalizing concepts--stating them in observable, countable terms. This is how you write your descriptions; avoid inferences, generalizations, vague terms. Avoid sophisticated terms that will obscure what actually occurred (they interacted could mean many possible terms--even mud wrestling!). Get down the details, even if they seem irrelevant at first. Describe the obvious, because it may be less obvious (and less likely to be remembered) once you leave the site. Also what is obvious to you may not be obvious to outsiders. Push yourself to describe actions without evaluating (evaluating, generalizing, inferring can all occur in the other kinds of notes, but not the field notes section of notes). Students must often push themselves to get details. If you begin to generalize too early, you may be recording more your bias that what actually occurs (although you might put something in the margins of your notes to be recorded separately when you type them up IF the idea is absolutely overwhelming or if you think youll forget an important aspect). You need data from which to generalize, otherwise the results cannot be trusted any more than folk tales or generalized impressions (but do record impressions later in personal notes or theoretical notes). One important distinction between research and general experience is that research relies upon carefully documented data from which conclusions are formed. Alternative methods of making field notes include making recordings at intervals or, if it wont be too distracting, talking quietly into a cassette recorder. I even used a camcorder with special amplified microphone that hung next to my mouth for making verbal field notes. It worked well, and the tiny compression system in the microphone made it possible to hear what was happening in the environment as well. You may need to take period breaks to say your field notes into the cassette recorder. The drawback for mechancial recordings is that they will need some kind of transcription later (see comments on transcribing interviews, below). Some are able to take a small laptop with them to the field site and type in field notes that way. However, many people find typing on a laptop to be distracting to people at the site, and a single computer crash can destroy the whole day of data (or more if you didnt back up regularly!). If nothing--absolutely nothing--is happening, then describe the physical context in excruciating detail. Look around carefully, even get down on the floor and look at the floors and walls carefully during dead times when absolutely nothing is happening. Handwritten notes are then typed into the computer when not observing. During the typing process, some details will probably be recalled that were not written down at the site--include these. While typing, separate the personal notes, theoretical notes, and methodology notes from the standard field notes. This can be done by using separate computer files for each of these, or simply denoted by some code within a single text. You will probably find that you will add much more to personal notes, theoretical notes, and methodology notes at this time--good! Time at the site should emphasize events and descriptions, while the typing time will tend to be more reflective. This reflection is actually the beginnings of analysis which is a reflective process (a less formal analysis than the formal approaches to be used near the completion of the study). Field notes should be typed in a column on one half of the paper (set your margins accordingly in your word processor). The other half of the paper is for coding and comments. If you are right handed, the notes should be on the left side of the paper so you can write in the right column (and vice versa if you are left handed). Good interviews have lots of open-ended questions, most of which are formed prior to the interview. I personally like questions that come out of observations better than those created out of the students imagination. Sometimes, though, good questions emerge during an interview because of what has been said by the one interviewed. Usually Id go with the flow and ask the emergent questions, if its appropriate. Transcribing interviews can be done several ways. Word-for-word transcriptions are probably best, but they are laborious. If you are well funded, this can be hired out. But there is value in the researcher listening to interviews, as the researcher may be able to figure out a muffled word that a transcriptionist cannot. Also, the interviewer may learn how to better interview by listening to his or her mistakes. It is also possible to use word-for-word transcriptions of some sections and summarize others when typing up the interviews. It is also possible to listen to the tape of the interview several times in order to better discover what sections are important enough to transcribe, which sections need to be summarized, and which sections should be ignored. But keep the tape--later in the research you may find that what was not typed was indeed more important than you thought! It is even possible to code directly from the tape--there are computer programs that allow you to connect a cassette player or even a videotape player directly to the computer, so that time markers and even your transcription can be added to the audio or video data. As with field notes, transcriptions should be typed in a single column with a wide margin for coding and comments. You cant code well if your field notes are not concrete and tangible (think operationalized). Coding includes categorizations, classifications, and other kinds of comments about the field notes (you could code your other notes as well, but people usually do not--generally what you are most concerned about is what happened at the site). There are many possible things you can code (see Lofland and Lofland). As you code, think about possible linkages and relationships between the different codes you use. This thinking will tend to produce more sophisticated codes, broader or more precise codes. Be sure to record what you are thinking during the coding process and the thinking that produced new, more sophisticated codes. This is also a form of analysis which should be carefully described in your theoretical notes. Push yourself to develop deeper and more revealing/descriptive/accurate categories and codes. As you code, keep in mind that possible linkages and relationships may also include confounds to causation. This is a valuable aspect of quantitative research that should be considered in qualitative reflection. Dont get too bogged down in thinking about confounds at first--let the ideas flow in your theoretical notes, but especially give this some thought as you move to more formal analysis near the end of the study. Robert Rosenthals writings help in exploring this topic in greater detail. Return to Qualitative Research Outline Return to Qualitative Research Resources Analysis of Field Notes and other Data Sources by Donald Ratcliff 1. Determine Unit--such as a word, a sentence, a paragraph, time, frame, etc. 2. Code Various Units--can have multiple codes for a unit (codes are not mutually exclusive, at least at first, but work towards exhaustive coding, at least eventually). Theory--preexisting or emergent--influences coding, but can push self to code broadly, thus less bound by theory. May need to code same data several times--computers definitely help. 3. Develop Categories, Subcategories, Superordinate Categories. These require definitions. Categories and definitions will probably need to be revised many times as you continue analysis. Keep track of the revisions and reasons for revisions of categories and definitions in theoretical notes (revisions occur because data indicates that previous definitions were not sufficient). 4. Give Examples of Categories in your theoretical notes, indexed specifically to pages, line numbers, etc. of field notes/video data/other sources of data. Include exemplars--best examples that represent the core of the category. Also include outliers--poor examples, but nevertheless examples of the category, as they define the limits of the category. 5. Linkages Between Categories need to be specified, and also note the kinds of linkages involved. Consider these kinds of linkages, among others: • Time • Space • Causation • Social/Interpersonal • [many others are possible] GOVERNMENT OF THE DISTRICT OF COLUMBIA Executive Office of Mayor Muriel Bowser Safer, Stronger DC Office of Neighborhood Safety and Engagement SPECIAL CONDITIONS: GEER FY 2022 • Grantee is responsible for prevention and intervention services at Paul Public Charter School as outlined in the Statement of Work and the grantee’s proposal. Services include Safe Passage, community canvassing, weekly virtual and in-person groups, a monthly log of community contacts and stakeholders, a peacemaking retreat, staff development and youth incentives. • Grantee agrees to assign at least 2 staff members for Safe Passage twice daily to ensure student safety in the community during arrival and dismissal from school. • Grantee agrees that Safe Passage staff must arrive no later than 30 minutes before school and remain on Safe Passage routes at least 1 hour after school has started. For dismissal, Safe Passage staff must arrive no later than 30 minutes before dismissal and remain on Safe Passage routes at least 1 hour after dismissal time. Staff must document their canvassing start and end time twice daily with OLA staff. • Grantee must create a canvassing map that outlines the assigned school, canvassing route, neighborhood hot spots, safety zones and metro station in accordance with the legend and sample map provided by ONSE. This map shall be submitted as an attachment in weekly and monthly reports. • Grantee agrees to facilitate weekly virtual or in-person groups once a week for a minimum of 90 minutes for each after school group session. Student attendance must be documented for each session. Attendance shall be submitted as an attachment to the monthly report. • Grantee must develop a log of community contacts and stakeholders that can support prevention and intervention methods. This log must include the contact or stakeholder name, type of contact, main contact, date of initial contact, phone number, email, and notes. This log shall be submitted as an attachment to the monthly report. • Grantee must coordinate and host at least one youth peacemaking retreat. Student attendance must be documented at the youth retreat. Attendance shall be submitted as an attachment to the monthly report. • Grantee agrees to coordinate at least 2 professional development trainings to support their staff’s ability to implement prevention and intervention efforts. Staff professional development trainings must be documented and shared in the weekly and monthly reports.1 SPECIAL CONDITIONS PAGE 1 ____________ INITIAL HERE GOVERNMENT OF THE DISTRICT OF COLUMBIA Executive Office of Mayor Muriel Bowser Safer, Stronger DC Office of Neighborhood Safety and Engagement • Grantee must establish and maintain a system for providing youth incentives for group session attendance, improved behavior and completing curriculum. • Grantee agrees to complete and submit all weekly and monthly reports on time. • Grantee agrees to participate in all post-award orientation meetings, technical assistance sessions, and training as required by the Stronger, Safer DC Office of Neighborhood Safety and Engagement (ONSE).2 • Grantee agrees that all consultants/contractors must attend any meetings and/or training required by ONSE. • Grantee shall develop a data metric system that will show measurable goals throughout the grant period. Documentation shall be reflected in weekly reports and submitted to the Deputy Director/Program Manager as part of the monthly report. • Grantee agrees to participate fully in the performance management and evaluation initiatives administered by ONSE during the period of this Grant Agreement. Grantee understands that evaluation may consist of a review of administrative structure, financial expenditures as well as programmatic services that are both funded and unfunded. • Grantee understands that failure to deliver timely reports or participate fully in the evaluation process will result in a delay in reimbursement funding and may lead to a reduction in funding or debarment of future funds. • Grantee shall submit a spend-down plan for each contractor within fifteen (15) business days of the signed grant award. Failure to comply with the submission request within fifteen (15) days may result in suspension, delay and/or denial of relevant reimbursement. • Grantee shall submit a final program narrative to match the budget narrative within fifteen (15) business days of the signed grant award. Failure to comply with the submission request within fifteen (15) days may result in suspension, delay and/or denial of relevant reimbursement. • Grantee shall submit a spreadsheet to identify contractors/consultants names, specified communities, and hours per week for each community within fifteen (15) business days of the signed grant award. Failure to comply with the submission request within fifteen (15) days may result in suspension, delay and/or denial of relevant reimbursement. SPECIAL CONDITIONS PAGE 2 ____________ INITIAL HERE GOVERNMENT OF THE DISTRICT OF COLUMBIA Executive Office of Mayor Muriel Bowser Safer, Stronger DC Office of Neighborhood Safety and Engagement • Grantee shall work in collaboration with all awarded GEER grantees identified for this project. Weekly team meetings and/or conference calls must take place and be documented. A representative from each contractor shall participate in team weekly meetings/conference calls. The ONSE Program Manager and/or designee will participate in the weekly team meetings.3 • Grantee shall work with the ONSE Deputy Director/Program Manager and/or Designee to strategize, develop and implement trauma-focused programs/projects for their identified communities. All trauma- informed care programming will be reviewed by the ONSE Deputy Director/Program Manager and/or Designee to determine the efficacy of the models. The ONSE Deputy Director/Program Manager and/or Designee will facilitate community projects in conjunction with the grantee. • Grantee shall be required to submit documentation to the ONSE Deputy Director/Program Manager to upload data in ZoomGrants, Efforts To Outcome (ETO), or Performance Measurement Tool (PMT) online, as required by OSSE. • Grantee shall upload all program and financial documentation into the PASS System. • Grantee agrees to submit budget modification or personnel GANs to the ONSE Grants Management Specialist for submission in a timely fashion (see grant agreement for specific terms). • Grantee agrees to submit all memorandum of understanding with contractors prior to the 1st quarter drawdown, if applicable. • Grantee is required to complete and submit the EEOP Certification Form developed by OCR, which may be found at http://www.ojp.usdoj.gov/about/ocr/pdfs/cert.pdf, if applicable. • Grantee is required to notify ONSE Program Manager within 48 hours of employee or beneficiary formal complaints of discrimination against their organization. • Grantee is required to post and display the District of Columbia Equal Employment Opportunity poster in a conspicuous area accessible to employees. This information is available at http://www.ohr.washingtondc.gov/ohr/cwp/view,a,3,q,558960,ohrNav,\%7C30953\%7C.asp.4 • Grantee is responsible for notifying ONSE Grants Management Specialist in writing, that either all the grant funds will not be utilized per the grant award and grant agreement or the project will be terminated SPECIAL CONDITIONS PAGE 3 ____________ INITIAL HERE http://www.ojp.usdoj.gov/about/ocr/pdfs/cert.pdf http://www.ojp.usdoj.gov/about/ocr/pdfs/cert.pdf http://www.ohr.washingtondc.gov/ohr/cwp/view,a,3,q,558960,ohrNav,\%7C30953\%7C.asp http://www.ohr.washingtondc.gov/ohr/cwp/view,a,3,q,558960,ohrNav,\%7C30953\%7C.asp GOVERNMENT OF THE DISTRICT OF COLUMBIA Executive Office of Mayor Muriel Bowser Safer, Stronger DC Office of Neighborhood Safety and Engagement at an earlier date than indicated on the grant award and grant agreement. If the project has not commenced within 60 days of the starting date or if Project Personnel has not been hired within 30 days of the project start date, explanation of the steps taken to initiate the project, the reason for delay, and the expected commencement date must be indicated on the Notification of Project Commencement Form. • All personnel funded in whole or in part under this Grant must be identified by name. In addition, accurate time and attendance records must be kept for all personnel hired and employed under this project. A listing of all employed under this project must be submitted within 30 days of signing the grant agreement Failure to provide this information may result in the suspension of funds. • Grantee understands that their risk level determines requirements related to the submission of financial invoice/reimbursement requests. Grantees designated as high risk or medium risk must submit all backup documentation for expenditures requested in each quarter. This information shall be uploaded into ZoomGrants. Grantees designated as low risk do not need to submit all backup documentation for expenditures in each quarter, but low-risk grantees must still maintain this documentation, as the grant manager will request at least once annually to see backup documentation for selected expenditures as part of a desk audit/review for compliance monitoring purposes. • Grantee shall make notification of this grant agreement to all consultants/contractors to clearly communicate the terms and conditions by which all parties will adhere to. Failure to comply with Special Conditions shall be viewed as a failure to comply with the Conditions of the Grant Award and may result in suspension of funds or termination of the grant agreement. ___________________________________________________________________________________ Signature of Authorized Official Date ___________________________________________________________________________________ Printed Name and Title of Authorized Official from Grantee Organization 15 Methods of Data Analysis in Qualitative Research Compiled by Donald Ratcliff 1. Typology - a classification system, taken from patterns, themes, or other kinds of groups of data. (Patton pp. 393,398) John Lofland & Lyn Lofland Ideally, categories should be mutually exclusive and exhaustive if possible, often they arent. Basically a list of categories. example: Lofland and Loflands 1 st edition list: acts, activities, meanings, participation, relationships, settings (in the third edition they have ten units interfaced by three aspects--see page 114--and each cell in this matrix might be related to one of seven topics--see chapter seven). 2. Taxonomy (See Domain Analysis - often used together, especially developing taxonomy from a single domain.) James Spradley A sophisticated typology with multiple levels of concepts. Higher levels are inclusive of lower levels. Superordinate and subordinate categories 3. Constant Comparison/Grounded Theory (widely used, developed in late 60s) Anselm Strauss • Look at document, such as field notes • Look for indicators of categories in events and behavior - name them and code them on document • Compare codes to find consistencies and differences • Consistencies between codes (similar meanings or pointing to a basic idea) reveals categories. So need to categorize specific events • We used to cut apart copies of field notes, now use computers. (Any good word processor can do this. Lofland says qualitative research programs arent all that helpful and I tend to agree. Of the qualitative research programs I suspect that NUD*IST probably the best--see Sage Publishers). • Memo on the comparisons and emerging categories • Eventually category saturates when no new codes related to it are formed • Eventually certain categories become more central focus - axial categories and perhaps even core category. 4. Analytic Induction (One of oldest methods, a very good one) F. Znaniecki, Howard Becker, Jack Katz. I wrote a paper on the topic. Look at event and develop a hypothetical statement of what happened. Then look at another similar event and see if it fits the hypothesis. If it doesnt, revise hypothesis. Begin looking for exceptions to hypothesis, when find it, revise hypothesis to fit all examples encountered. Eventually will develop a hypotheses that accounts for all observed cases. 5. Logical Analysis/Matrix Analysis An outline of generalized causation, logical reasoning process, etc. Use flow charts, diagrams, etc. to pictorially represent these, as well as written descriptions. Matthew Miles and Huberman gives hundreds of varieties in their huge book Qualitative Data Analysis, 2 nd ed. 6. Quasi-statistics (count the # of times something is mentioned in field notes as very rough estimate of frequency) Howard Becker Often enumeration is used to provide evidence for categories created or to determine if observations are contaminated. (from LeCompte and Preissle). 7. Event Analysis/Microanalysis (a lot like frame analysis, Erving Goffman) Frederick Erickson, Kurt Lewin, Edward Hall. Emphasis is on finding precise beginnings and endings of events by finding specific boundaries and things that mark boundaries or events. Specifically oriented toward film and video. After find boundaries, find phases in event by repeated viewing. 8. Metaphorical Analysis (usually used in later stages of analysis) Michael Patton, Nick Smith Try on various metaphors and see how well they fit what is observed. Can also ask participant for metaphors and listen for spontaneous metaphors. Hallway as a highway. Like highway in many ways: traffic, intersections, teachers as police, etc. Best to check validity of metaphor with participants - member check. 9. Domain Analysis (analysis of language of people in a cultural context) James Spradley Describe social situation and the cultural patterns within it. Semantic relationships. Emphasize the meanings of the social situation to participants. Interrelate the social situation and cultural meanings. Different kinds of domains: Folk domains (their terms for domains), mixed domains, analytic domains (researchers terms for domains). • select semantic relationships • prepare domain analysis worksheet • select sample of field notes (statements of people studied) • look for broad and narrow terms to describe semantic relationships • formulate questions about those relationships • repeat process for different semantic relationship • list all domains discovered 10. Hermeneutical Analysis (hermeneutics = making sense of a written text) Max Van Manen Not looking for objective meaning of text, but meaning of text for people in situation. Try to bracket self out in analysis - tell their story, not yours. Use their words, less interpretive than other approaches. Different layers of interpretation of text. Knowledge is constructed – we construct meaning of text (from background and current situation - Social construction because of influence of others - symbolic interactionism) Use context - time and place of writing - to understand. What was cultural situation? Historical context. Meaning resides in author intent/purpose, context, and the encounter between author and reader - find themes and relate to dialectical context. (Some say authorial intent is impossible to ascertain.) Videotape - probably needs to be secondary level of analysis. Get with another person who is using another method and analyze their field notes. 11. Discourse analysis (linguistic analysis of ongoing flow of communication) James Gee Usually use tapes so they can be played and replayed. Several people discussing, not individual person specifically. Find patterns of questions, who dominates time and how, other patterns of interaction. 12. Semiotics (science of signs and symbols, such as body language) Peter Manning Determine how the meanings of signs and symbols is constructed. Assume meaning is not inherent in those, meaning comes from relationships with other things. Sometimes presented with a postmodernist emphasis. 13. Content Analysis (not very good with video and only qualitative in development of categories - primarily quantitative) (Might be considered a specific form of typological analysis) R. P. Weber Look at documents, text, or speech to see what themes emerge. What do people talk about the most? See how themes relate to each other. Find latent emphases, political view of newspaper writer, which is implicit or look at surface level - overt emphasis. Theory driven - theory determines what you look for. Rules are specified for data analysis. Standard rules of content analysis include: • How big a chunk of data is analyzed at a time (a line, a sentence, a phrase, a paragraph?) Must state and stay with it. • What are units of meaning?, the categories used. Categories must be: 1. Inclusive (all examples fit a category) 2. Mutually exclusive • Defined precisely: what are properties • All data fits some category (exhaustive) Also note context. Start by reading all way through, then specify rules. Could have emergent theory, but usually theory-driven. After determine categories, do the counting - how often do categories occur. Most of literature emphasizes the quantitative aspects. Originated with analyzing newspaper articles for bias - counting things in print. Very print oriented - can it be adapted for visual and verbal? 14. Phenomenology/Heuristic Analysis (phenomenological emphasis - how individuals experience the world) Clark Moustakas Emphasizes idiosyncratic meaning to individuals, not shared constructions as much. Again, try to bracket self out and enter into the other persons perspective and experience. Emphasizes the effects of research experience on the researcher-personal experience of the research. How does this affect me as researcher. Much like hermeneutical analysis, but even more focused on the researchers experience. Some use the term phenomenology to describe the researchers experience and the idea that this is all research is or can ever be (see Lofland and Lofland, p. 14). 15. Narrative Analysis (study the individuals speech) Catherine Reisman Overlaps with other approaches. (Is it distinctive?) Discourse analysis looks at interaction, narrative is more individual) The story is what a person shares about self. What you choose to tell frames how you will be perceived. Always compare ideas about self. Tend to avoid revealing negatives about self. Might study autobiographies and compare them. • context-situation • core plot in the story told about self • basic actions Narrative analysis could involve study of literature or diaries or folklore. References Taxonomic Analysis: James P. Spradley (1980). Participant observation. Fort Worth: Harcourt Brace. Typological Systems: John Lofland & Lyn H. Lofland (1995). Analyzing social settings, 3rd ed. Belmont, Cal.: Wadsworth. Constant Comparison: Anselm L. Strauss (1987). Qualitative analysis for social scientists. New York: Cambridge University Press. Case Study Analysis: Sharon Merriam (1988). Case study research in education. Jossey-Bass. Ethnostatistics: Robert P. Gephart (1988). Ethnostatistics: Qualitative foundations for quantitative research. Newbury Park, Cal.: Sage Publications. Logical Analysis/Matrix Analysis: Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis, 2nd ed. Newbury Park, Cal.: Sage. [Note: I think this may well be the best book available on qualitative data analysis.] Phenomenological/Heuristic Research: Moustakas, C. (1990). Heuristic Research. Newbury Park, Cal.: Sage; and Moustakas, C. (1994). Phenomenological research methods. Newbury Park, Cal.: Sage. Event Analysis/Microanalysis: Frederick Erickson (1992). Ethnographic microanalysis of interaction. In M. LeCompte, et. al. (Eds), The handbook of qualitative research in education (chapter 5). San Diego: Academic Press. Analytic Induction: Jack Katz (1983). A theory of qualitative methodology. In R. M. Emerson (Ed.), Contemporary field research. Prospect Heights, Ill.: Waveland. Hermeneutical Analysis: Max Van Manen (1990). Researching lived experience. New York: State University of New York Press. Semiotics: Peter K. Manning (1987). Semiotics and fieldwork. Newbury Park, Cal.: Sage. Discourse Analysis: James P. Gee (1992). Discourse analysis. In M. LeCompte, et. al. (Eds), The handbook of qualitative research in education (chapter 6). San Diego: Academic Press. Narrative Analysis: Catherine K. Reisman (1993). Narrative analysis. Newbury Park, Cal.: Sage. Content Analysis: R. P. Weber (1990). Basic content analysis. Newbury Park, Cal.: Sage. Domain Analysis: James P. Spradley (1980). Participant observation. Fort Worth: Harcourt Brace. Also see J. P. Spradley, Ethnographic interview (1979, same publisher). Metaphorical Analysis: Nick Smith (1981). Metaphors for evaluation. Newbury Park, Cal.: Sage. Analyzing Qualitative Data G3658-12 2003 Ellen Taylor-Powell Marcus Renner Program Development & Evaluation Introduction Qualitative data consist of words and observa- tions, not numbers. As with all data, analysis and interpretation are required to bring order and understanding. This requires creativity, discipline and a systematic approach. There is no single or best way. Your process will depend on: ■ the questions you want to answer, ■ the needs of those who will use the informa- tion, and ■ your resources. This guide outlines a basic approach for analyz- ing and interpreting narrative data — often referred to as content analysis — that you can adapt to your own extension evaluations. For descriptions of other types of qualitative data analysis, see Ratcliff, 2002. Other techniques may be necessary for analyzing qualitative data from photographs and audio or video sources. This booklet is a companion to Analyzing Quantitative Data G3658-6 in this series. Narrative data Text or narrative data come in many forms and from a variety of sources. You might have brief responses to open-ended questions on a survey, the transcript from an interview or focus group, notes from a log or diary, field notes, or the text of a published report. Your data may come from many people, a few individuals, or a single case. Any of the following may produce narrative data that require analysis. ■ Open-ended questions and written com- ments on questionnaires may generate single words, brief phrases, or full para- graphs of text. ■ Testimonials may give reactions to a program in a few words or lengthy com- ments, either in person or in written corre- spondence. ■ Individual interviews can produce data in the form of notes, a summary of the individ- ual’s interview, or word-for-word tran- scripts. ■ Discussion group or focus group inter- views often involve full transcripts and notes from a moderator or observer. ■ Logs, journals and diaries might provide structured entries or free-flowing text that you or others produce. ■ Observations might be recorded in your field notes or descriptive accounts as a result of watching and listening. ■ Documents, reports and news articles or any published written material may serve as evaluation data. ■ Stories may provide data from personal accounts of experiences and results of pro- grams in people’s own words. ■ Case studies typically include several of the above. PD E&& University of Wisconsin-Extension Cooperative Extension Madison, Wisconsin The analysis process Once you have these data, what do you do? The steps below describe the basic elements of narra- tive data analysis and interpretation. This process is fluid, so moving back and forth between steps is likely. Step 1 Get to know your data. Good analysis depends on understanding the data. For qualitative analysis, this means you read and re-read the text. If you have tape recordings, you listen to them several times. Write down any impressions you have as you go through the data. These impressions may be useful later. Also, just because you have data does not mean those are quality data. Sometimes, information provided does not add meaning or value. Or it may have been collected in a biased way. Before beginning any analysis, consider the quality of the data and proceed accordingly. Investing time and effort in analysis may give the impression of greater value than is merited. Explain the limitations and level of analysis you deem appropriate given your data. Step 2 Focus the analysis. Review the purpose of the evaluation and what you want to find out. Identify a few key ques- tions that you want your analysis to answer. Write these down. These will help you decide how to begin. These questions may change as you work with the data, but will help you get started. How you focus your analysis depends on the purpose of the evaluation and how you will use the results. Here are two common approaches. Focus by question or topic, time period or event. In this approach, you focus the analysis to look at how all individuals or groups responded to each question or topic, or for a given time period or event. This is often done with open-ended ques- tions. You organize the data by question to look across all respondents and their answers in order to identify consistencies and differences. You put all the data from each question together. You can apply the same approach to particular topics, or a time period or an event of interest. Later, you may explore the connections and rela- tionships between questions (topics, time periods, events). Focus by case, individual or group. You may want an overall picture of: ■ One case such as one family or one agency. ■ One individual such as a first-time or teen participant in the program. ■ One group such as all first-time participants in the program, or all teens ages 13 to 18. Rather than grouping these respondents’ answers by question or topic, you organize the data from or about the case, individual or group, and analyze it as a whole. Or you may want to combine these approaches and analyze the data both by question and by case, individual or group. Step 3 Categorize information. Some people refer to categorizing information as coding the data or indexing the data. However, categorizing does not involve assigning numeri- cal codes as you do in quantitative analysis where you label exclusive variables with preset codes or values. To bring meaning to the words before you: ■ Identify themes or patterns — ideas, con- cepts, behaviors, interactions, incidents, terminology or phrases used. ■ Organize them into coherent categories that summarize and bring meaning to the text. This can be fairly labor-intensive depending on the amount of data you have. But this is the crux of qualitative analysis. It involves reading and re-reading the text and identifying coherent categories. You may want to assign abbreviated codes of a few letters, words or symbols and place them next to the themes and ideas you find. This will help organize the data into categories. Provide a descriptive label (name) for each category you create. Be clear about what you include in the category and what you exclude. As you categorize the data, you might identify other themes that serve as subcategories. Continue to categorize until you have identified and labeled all relevant themes. The following examples show categories that were identified to sort responses to the questions. 2 P R O G R A M D E V E L O P M E N T A N D E V A L U A T I O N■ ■ ■ A N A L Y Z I N G Q U A L I T A T I V E D A T A ■ ■ ■ Question Categories Responses to the question were sorted into: 1. What makes a quality educational program? Staff (Stf), relevance (Rel), participation (Part), timeliness (Time), content (Con) 2. What is the benefit of a youth mentoring program? Benefits to youth (Y), benefits to mentor (M), benefits to family (Fam), benefits to community (Comm) 3. What do you need to continue your learning Practice (P), additional training (Trg), time (T), about evaluation? resources (R), feedback (Fdbk), mentor (M), uncertain (U) Possible code abbreviations are designated in parentheses. 3 Here are two ways to categorize narrative data — using preset or emergent categories. Preset categories You can start with a list of themes or categories in advance, and then search the data for these topics. For example, you might start with con- cepts that you really want to know about. Or you might start with topics from the research litera- ture. These themes provide direction for what you look for in the data. You identify the themes before you categorize the data, and search the data for text that matches the themes. Emergent categories Rather than using preconceived themes or cate- gories, you read through the text and find the themes or issues that recur in the data. These become your categories. They may be ideas or concepts that you had not thought about. This approach allows the categories to emerge from the data. Categories are defined after you have worked with the data or as a result of working with the data. Sometimes, you may combine these two approaches — starting with some preset cate- gories and adding others as they become apparent. Your initial list of categories may change as you work with the data. This is an iterative process. You may have to adjust the definition of your cat- egories, or identify new categories to accommo- date data that do not fit the existing labels. Main categories may be broken into subcategories. Then you will need to resort your data into these smaller, more defined categories. This allows for greater discrimination and differentiation. For example, in the question about benefits of a youth mentoring program, data within the cate- gory benefits to youth might be broken into a number of subcategories. Continue to build categories until no new themes or subcategories are identified. Add as many cat- egories as you need to reflect the nuances in the data and to interpret data clearly. While you want to try to create mutually exclu- sive and exhaustive categories, sometimes sec- tions of data fit into two or more categories. So you may need to create a way to cross-index. Reading and re-reading the text helps ensure that the data are correctly categorized. Example 1 shows labeling of one open-ended question on an end-of-session questionnaire. In this example, all responses were numbered and given a label to capture the idea(s) in each comment. Later, you can sort and organize these data into their categories to identify patterns and bring meaning to the responses. Question Categories What is the benefit Benefits to youth (Y) of a youth mentoring School performance (Y-SP) program? Friendship (Y-Friends) Self-concept (Y-SC) Role modeling (Y-RM) Benefits to mentor (M) Benefits to family (Fam) Benefits tocommunity (Comm) Subcategories 4 P R O G R A M D E V E L O P M E N T A N D E V A L U A T I O N■ ■ ■ Example 1. Labeling data from an end-of-session questionnaire (21 respondents) Categories: Practice (P), additional training (Trg), time (T), resources (R), feedback (Fdbk), mentor (M), uncertain (U) Line 7 is left uncoded because “Yes” is not usable data. Step 4 Identify patterns and connections within and between categories. As you organize the data into categories — either by question or by case — you will begin to see patterns and connections both within and between the categories. Assessing the relative importance of different themes or highlighting subtle variations may be important to your analysis. Here are some ways to do this. Within category description You may be interested in summarizing the infor- mation pertaining to one theme, or capturing the similarities or differences in people’s responses within a category. To do this, you need to assem- ble all the data pertaining to the particular theme (category). What are the key ideas being expressed within the category? What are the similarities and differ- ences in the way people responded, including the subtle variations? It is helpful to write a summary for each category that describes these points. Larger categories You may wish to create larger super categories that combine several categories. You can work up from more specific categories to larger ideas and concepts. Then you can see how the parts relate to the whole. Relative importance To show which categories appear more impor- tant, you may wish to count the number of times a particular theme comes up, or the number of unique respondents who refer to certain themes. These counts provide a very rough estimate of relative importance. They are not suited to statis- tical analysis, but they can reveal general pat- terns in the data. Relationships You also may discover that two or more themes occur together consistently in the data. Whenever you find one, you find the other. For example, youth with divorced parents consis- tently list friendship as the primary benefit of the mentoring program. You may decide that some of these connections suggest a cause and effect relationship, or create a sequence through time. For example, respon- dents may link improved school performance to a good mentor relationship. From this, you might argue that good mentoring causes improved school performance. Such connections are important to look for, because they can help explain why something occurs. But be careful about simple cause and effect interpretations. Seldom is human behavior or narrative data so simple. Ask yourself: How do things relate? What data support this interpretation? What other factors may be contributing? You may wish to develop a table or matrix to illustrate relationships across two or more cate- gories. Look for examples of responses or events that run counter to the prevailing themes. What do these countervailing responses suggest? Are they important to the interpretation and understand- ing? Often, you learn a great deal from looking at and trying to understand items that do not fit into your categorization scheme. Step 5 Interpretation – Bringing it all together Use your themes and connections to explain your findings. It is often easy to get side tracked by the details and the rich descriptions in the data. But what does it all mean? What is really important? This is what we call interpreting the data — attaching meaning and significance to the analysis. A good place to start is to develop a list of key points or important findings you discovered as a result of categorizing and sorting your data. Stand back and think about what you have learned. What are the major lessons? What new things did you learn? What has application to other settings, programs, studies? What will those who use the results of the evaluation be most interested in knowing? Too often, we list the findings without synthesiz- ing them and tapping their meaning. Develop an outline for presenting your results to other people or for writing a final report. The length and format of your report will depend on your audience. It is often helpful to include quotes or descriptive examples to illustrate your points and bring the data to life. A visual display might help communicate the findings. Sometimes a diagram with boxes and arrows can help show how all the pieces fit together. Creating such a model may reveal gaps in your investigation and connections that remain unclear. These may be areas where you can suggest further study. 5A N A L Y Z I N G Q U A L I T A T I V E D A T A ■ ■ ■ 6 P R O G R A M D E V E L O P M E N T A N D E V A L U A T I O N■ ■ ■ “Nuts and bolts” of narrative analysis Moving from a mass of words to a final report requires a method for organizing and keeping track of the text. This is largely a process of cutting and sorting. Work by hand, either with a hard copy (print copy) or directly on the computer. Exactly how you manage the data depends on your personal preference and the amount and type of qualita- tive data you have. Here are some data manage- ment tips: ■ Check your data. Often, there are data from multiple respondents, multiple surveys or documents. Make sure you have everything together. Decide whether the data are of suf- ficient quality to analyze, and what level of investment is warranted. ■ Add ID numbers. Add an identification (ID) number to each questionnaire, respondent, group or site. ■ Prepare data for analysis. You may need to transcribe taped interviews. How complete to make your transcription depends on your purpose and resources. Sometimes, you may make a summary of what people say, and analyze that. Or certain parts of an interview may be particularly useful and important and just those sections are transcribed. Other times, you will want to have every word of the entire interview. However, transcription is time-consuming. So be sure both data quality and your use of the data are worth the investment. With small amounts of narrative data, you may work directly from the original hard copy. However, text is usually typed into a computer program. In extension, we typically type into a word processing program (Microsoft Word or Word Perfect) or into Excel. You may decide to use a relational data base management program such as ACCESS, or a special qualitative data analysis program. Your decision depends on the size of your data set, resources available, preferences, and level of analysis needed or warranted. Decide whether you will enter all responses ques- tion by question, or whether you want to keep all text concerning one case, individual, group or site together (see Step 2). Save the file. If you type the data into a word processing program, it is helpful to leave a wide margin on the left so you have space to write labels for text and any notes you want to keep. Number each line to help with cutting and sorting later. ■ Make copies. Make a copy of all your data (hard copy and electronic files). This gives you one copy to work from and another for safekeeping. ■ Identify the source of all data. As you work with the data, you will need to keep track of the source of the information or the context of the quotes and remarks. Such information may be critical to the analysis. Make sure you have a way to identify the source of all the data, such as by individual, site and date. Think about what information to keep with the data. For example, you might use identifiers to designate the respondent, group, site, county, date or other source information. Or you may wish to sort by variables such as age, gender or position. Will you want to compare and contrast by demographic variable, site and date? These identifiers stay with the information as you cut and sort the data, either by hand or in the computer. If you are working with hard copies, you might use different colors of paper to color- code responses from different people or groups (for example, see Krueger, 1998). ■ Mark key themes. Read through the text. Look for key ideas. Use abbreviations or symbols (codes) to tag key themes — ideas, concepts, beliefs, incidents, terminology used, or behaviors. Or, you might give each theme a different color. Keep notes of emerg- Computer software Several software programs — for example, Ethnograph and NUD*IST — specifically analyze qualitative data. They systematize and facilitate all the steps in qualitative analysis. SAS software will manipulate precategorized responses to summarize open-ended survey questions (see Santos, Mitchell and Pope, 1999). CDC EZ-Text is a freeware program developed by the Centers for Disease Control and Prevention. For smaller data sets and modest analysis needs, many people work by hand, with a word processing program or spreadsheet. Note: Mention of products is not intended to endorse them, nor to exclude others that may be similar. These are mentioned as a convenience to readers. 7A N A L Y Z I N G Q U A L I T A T I V E D A T A ■ ■ ■ ing ideas or patterns and how you are inter- preting the data. You can write or type these in the margins, or in a specified column. Or keep a separate notebook that records your thoughts and observations about the data (see Example 2). ■ Define categories. Organize or combine related themes into categories. Name (label) these categories by using your own descrip- tive phrases, or choose words and key phrases from the text. Be clear about what the category stands for. Would someone unfamiliar with the data understand the label you have chosen? Write a short description or definition for each category, and give examples or quotes from the text that illustrate meaning. Check with others to see if your labels make sense. You may also describe what the category does not include to clarify what is included. ■ Cut and sort. Once you define categories and label data, grouping the data into cate- gories involves some form of cutting and sorting. This is a process of selecting sec- tions of data and putting them together in their category. Hard copy — A simple method is to cut text out of the printed page and sort into differ- ent piles. Each pile represents a category and has a name. As you work with the data, Example 2. Identify themes and label data. Be responsive to local needs and questions Availability Responsive: willing and able to answer questions, timeliness, personal touch Local connection Follow-up Geographic coverage Service area, serve same people, need to extend out Staff Serve community, professional, responsive Focus set priorities; stretched too thin Reaching out vs. focus Staff = program Create a wide margin where you can label key ideas. Highlight quotes for future use. Keep notes of emerging ideas. you may make new piles, combine piles, or divide piles into subcategories. Remember to keep the identifier (source of data) with the data so you know where the text came from. Also, remember that you are working with a copy, not the original material. Electronic copy — It is relatively simple and fast to move text around in a word pro- cessing program using the Windows plat- form. You can cut and paste text into differ- ent Windows, each representing a single cat- egory. If you type the category label directly into the computer file, you can use the search function to gather chunks of text together to copy and paste. Or you can sepa- rate the text into paragraphs, code the beginning of each paragraph, and then sort the paragraphs. You may prefer to use Excel. If the data are in Microsoft Word, you can easily transfer them to Excel. Set up an Excel file that includes columns for the ID number, identifiers, categories (themes), codes, and text (see Example 3). When cutting and sorting, keep track of the source of the data. Be sure to keep identifiers attached to all sections of data. Keep enough text together so you can make sense of the words in their context. As you cut and move data, text can easily become frag- mented and lose its contextual meaning. Be sure to include enough surrounding text so the meaning is not open to misinterpretation. If data do not seem to fit, place those in a sepa- rate file for possible use later. ■ Make connections. Once you sort the data, think about how the categories fit together and relate. What seems more important, less important? Are there exceptions or critical cases that do not seem to fit? Consider alter- native explanations. Explore paradoxes, con- flicting themes, and evidence that seems to challenge or contradict your interpretations. To trace connections, you can spread note cards across a table, use sticky notes on walls, or draw diagrams on newsprint showing the categories and relationships. Another approach is to create a two-dimensional or three-dimensional matrix. List the categories along each axis, and fill the cells with corresponding evidence or data. For further explanation, see Patton, 1990. You can use simple hand tabulations or a com- puter program: ■ to search and count the frequency a topic occurs or how often one theme occurs with another, or ■ to keep track of how many respondents touch on different themes. Such counts may be illuminating and indicate relative importance. But treat them with caution — particularly when responses are not solicited the same way from all respondents, or not all respondents provide a response. 8 P R O G R A M D E V E L O P M E N T A N D E V A L U A T I O N■ ■ ■ Example 3. Screen shot of Excel spreadsheet Enhancing the process As with any analysis process, bias can influence your results. Consider the following ways to increase the credibility of your findings. Use several sources of data. Using data from different sources can help you check your findings. For example, you might combine one-on-one interviews with information from focus groups and an analysis of written material on the topic. If the data from these dif- ferent sources point to the same conclusions, you will have more confidence in your results. Track your choices. If others understand how you came to your con- clusions, your results will be more credible. Keep a journal or notebook of your decisions during the analysis process to help others follow your reasoning. Document your reasons for the focus you take, the category labels you create, revisions to categories you make, and any observations you note concerning the data as you work with the text. People tend to see and read only what supports their interest or point of view. Everyone sees data through his or her own lens and filters. It is important to recognize and pay attention to this. The analysis process should be documented so that another person can see the decisions that you made, how you did the analysis, and how you arrived at the interpretations. Involve others. Getting feedback and input from others can help with both analysis and interpretation. You can involve others in the entire analysis process, or in any one of the steps. For example, several people or one other person might review the data inde- pendently to identify themes and categories. Then you can compare categories and resolve any discrepancies in meaning. You can also work with others in picking out important lessons once cutting and sorting is done. Or you can involve others in the entire analysis process, reviewing and discussing the data and their meaning, arriving at major conclu- sions, and presenting the results. Involving others may take more time, but often results in a better analysis and greater ownership of the results. Pitfalls to avoid Finally, with any qualitative analysis, keep in mind the following cautions. Avoid generalizing. The goal of qualitative work is not to generalize across a population. Rather, a qualitative approach seeks to provide understanding from the respondent’s perspective. It tries to answer the questions: “What is unique about this indi- vidual, group, situation or issue? Why?” Even when you include an open-ended question on a survey, you are seeking insight, differences, the individual’s own perspective and meaning. The focus is on the individual’s own or unique response. Narrative data provide for clarification, under- standing and explanation — not for generalizing. Choose quotes carefully. While using quotes can lend valuable support to data interpretation, often quotes are used that only directly support the argument or illustrate success. This can lead to using people’s words out of context or editing quotes to exemplify a point. When putting together your final report, think about the purpose for including quotes. Do you want to show the differences in people’s com- ments, give examples of a typical response rela- tive to a certain topic, highlight success? In any event, specify why you chose the selected quotes. Include enough of the text to allow the reader to decide what the respondent is trying to convey. Confidentiality and anonymity are also concerns when using quotes. Even if you do not give the person’s identity, others may be able to tell who made the remark. Consider what might be the consequences of including certain quotes. Are they important to the analysis and interpreta- tion? Do they provide a balanced viewpoint? Get people’s permission to use their words. Check with others about the usefulness and value of the quotes you select to include. Address limitations and alternatives. Every study has limitations. Presenting the prob- lems or limitations you had while collecting and analyzing the data helps others better under- stand how you arrived at your conclusions. Similarly, it is important to address possible alternative explanations. What else might explain the results? Show how the evidence supports your interpretation. 9A N A L Y Z I N G Q U A L I T A T I V E D A T A ■ ■ ■ Concluding comments Working with qualitative data is a rich and enlightening experience. The more you practice, the easier and more rewarding it will become. As both a science and an art, it involves critical, analytical thinking and creative, innovative perspectives (Patton, 1990). Be thoughtful, and enjoy. References CDC EZ-Text. Centers for Disease Control and Prevention, National Center for HIV, STD, and TB Prevention Divisions of HIV/AIDS Prevention, Behavioral Intervention Research Branch. Retrieved 4-9- 03: http://www.cdc.gov/hiv/software/ez-text.htm Krueger, Richard A. 1998. Analyzing and Reporting Focus Group Results. Focus Group Kit 6. Thousand Oaks, Calif.: Sage Publications. Krueger, Richard A. 1988. Focus Groups: A Practical Guide for Applied Research. Newbury Park, Calif.: Sage Publications. Miles, Matthew B., & A. Michael Huberman. 1994. Qualitative Data Analysis: An Expanded Sourcebook. Second Edition. Thousand Oaks, Calif.: Sage Publications. Patton, Michael Q. 1990. Qualitative Evaluation and Research Methods. 2nd Edition. Newbury Park, Calif.: Sage Publications. Pope, Catherine, Sue Ziebland & Nicholas Mays. 1999. Qualitative Research in Health Care. Second Edition. London: BMJ Publishing Group. Chapter 8. Analysing Qualitative Data. Retrieved 4-9-03: http://www.bmjpg.com/qrhc/chapter8.html Ratcliff, Donald. 2002. Qualitative … Qualitative Data Analysis Copyright (c) 1998, AllRights Reserved, John V. Seidel Qualis Research, [email protected], www.qualisresearch.com This document was originally part of the manual for The Ethnograph v4. It also exists in the manual for The Ethnograph v5 as Appendix E.. You can freely download, copy, print and disseminate this document if 1) the copyright notice is included, 2) you include the entire document, and 3) you do not alter the document in any way. This permission does not extend to the Manual itself. Only to this electronic version of Appendix E. The information in this document represents some, but not all, of the ideas of the developer of The Ethnograph. I reserve the right to revise and change my ideas as I continue to develop them. Introduction This appendix is an essay on the basic processes in qualitative data analysis (QDA). It serves two purposes. First it offers some insights into the ideas and practices from which The Ethnograph emerged and continues to evolve. Second, it is also a simple introduction for the newcomer of QDA. A Process of Noticing, Collecting and Thinking The first part of this appendix describes QDA as a process of Noticing, Collecting, and Thinking about interesting things. The purpose of this model (Figure 1, page E2) is to show that there is a simple foundation to the complex and rigorous practice of QDA. Once you grasp this foundation you can move in many different directions. The idea for this model came from a conversation with one of my former teachers, Professor Ray Cuzzort. Ray was teaching an undergraduate statistics course and wanted to boil down the complexity of statistics to a simple model. His solution was to tell the students that statistics was a symphony based on two notes: means and standard deviations. I liked the simplicity and elegance of his formulation and decided to try and come up with a similar idea for describing QDA. The result was the idea that QDA is a symphony based on three notes: Noticing, Collecting, and Thinking about interesting things. While there is great diversity in the practice of QDA I would argue that all forms of QDA are based on these three “notes.” In the first section of this Appendix I explain this model, introduce the jigsaw puzzle analogy, and offer examples of how the basic QDA model is represented in the writings of QDA methodologists and researchers, and then present alternatives to the jigsaw puzzle analogy. A General Model of QDA The second part of this Appendix presents a more complex model of the process of QDA (Figure 3, page E13). This model incorporates the simpler model. It shows how the basic procedures of The Ethnograph mesh with the basic model, and how the analytic process unfolds and develops over time. QDA: A Model of the Process Analyzing qualitative data is essentially a simple process. It consists of three parts: Noticing, Collecting, and Thinking about interesting things. Figure 1 represents the process and the relationships among its parts. 2 Qualitative Data Analysis Figure 1. The Data Analysis Process As Figure 1 suggests, the QDA process is not linear. When you do QDA you do not simply Notice, Collect, and then Think about things, and then write a report. Rather, the process has the following characteristics: C Iterative and Progressive: The process is iterative and progressive because it is a cycle that keeps repeating. For example, when you are thinking about things you also start noticing new things in the data. You then collect and think about these new things. In principle the process is an infinite spiral. C Recursive: The process is recursive because one part can call you back to a previous part. For example, while you are busy collecting things you might simultaneously start noticing new things to collect. C Holographic: The process is holographic in that each step in the process contains the entire process. For example, when you first notice things you are already mentally collecting and thinking about those things. Thus, while there is a simple foundation to QDA, the process of doing qualitative data analysis is complex. The key is to root yourself in this foundation and the rest will flow from this foundation. Noticing, Collecting, Thinking about Things In the next sections I further elaborate on the notice, collect, think process. The primary vehicle is the analogy of solving a jigsaw puzzle. Then I show some of the ways in which the notice, collect, think process has been expressed in the writings of qualitative social scientists. Finally, I explore two alternative analogies. One is the “multi threaded DNA” analogy (Agar, 1991). The other is a map analogy based on the ideas of “topographical” maps and “ad hoc” maps. Qualitative Data Analysis 3 1. Noticing Things (and Coding Them) There are many different perspectives on: 1) the kinds of things that you can, and should, notice in your data, and 2) how you should go about the process of noticing those things. But behind these differences there is the common and simple practice of going out into the world and noticing interesting things. Two Levels of Noticing On a general level, noticing means making observations, writing field notes, tape recording interviews, gathering documents, etc. When you do this you are producing a record of the things that you have noticed. Once you have produced a record, you focus your attention on that record, and notice interesting things in the record. You do this by reading the record. In fact, you will read your record many times. As you notice things in the record you name, or code,” them. You could simply call them A, B, C, etc., but most likely you will develop a more descriptive naming scheme. Coding Things Coding data is a simple process that everyone already knows how to do. For example, when you read a book, underline or highlight passages, and make margin notes you are “coding” that book. Coding in QDA is essentially the same thing. For now, this analogy is a good place to start. As you become more experienced in QDA you learn that QDA “coding” is also more than this. Further, you will learn the difference between codes as heuristic tools and codes as objective, transparent representations of facts (Kelle and Seidel, 1995). In this essay I treat codes as heuristic tools, or tools to facilitate discovery and further investigation of the data. At the end of this chapter I address the objectivist-heuristic code continuum. 2. Collecting and Sorting Instances of Things Pieces of a Puzzle As you notice and name things the next step is to collect and sort them. This process is analogous to working on a jigsaw puzzle where you start by sorting the pieces of the puzzle. For example, assume you have a puzzle picture with a tree, a house, and sky. A common strategy for solving the puzzle is to identify and sort puzzle pieces into groups ( e.g., frame pieces, tree pieces, house pieces, and sky pieces). Some of the puzzle pieces will easily fit into these categories. Others will be more difficult to categorize. In any case, this sorting makes it easier to solve the puzzle. When you identify piece, you are noticing and “coding” them. When you sort the pieces you are “collecting” them. Of course this analogy differs in important ways from the QDA analysis process. For example, in QDA you don’t always have a final picture of the puzzle’s solution. Also, in QDA the puzzle pieces are usually not precut. You create the puzzle pieces as you analyze the phenomena. None the less, the jigsaw puzzle analogy captures some important attributes of the QDA process. A useful definition of the QDA process, and one that seems to fit well with the jigsaw puzzle analogy, comes from Jorgensen (1989). Analysis is a breaking up, separating, or disassembling of research materials into pieces, parts, elements, or units. With facts broken down into manageable pieces, 4 Qualitative Data Analysis the researcher sorts and sifts them, searching for types, classes, sequences, processes, patterns or wholes. The aim of this process is to assemble or reconstruct the data in a meaningful or comprehensible fashion (Jorgensen, 1989: 107). A similar idea is expressed by Charmaz (1983). For Charmaz, who works in the “grounded theory” tradition, the disassembling and reassembling occurs through the “coding” process. Codes serve to summarize, synthesize, and sort many observations made of the data....coding becomes the fundamental means of developing the analysis....Researchers use codes to pull together and categorize a series of otherwise discrete events, statements, and observations which they identify in the data (Charmaz, 1983: 112). At first the data may appear to be a mass of confusing, unrelated, accounts. But by studying and coding (often I code the same materials several times just after collecting them), the researcher begins to create order (Charmaz, 1983: 114). A concrete example of this processes occurs in Freidson’s (1975) Doctoring Together. This passage shows how the process moves back and forth between the noticing and collecting parts of the process. I have “coded” this example to highlight this movement. Noticing: ...we had carried out some 200 separate interviews...and had them transcribed....Each interview was read, and sections of them which seemed to be distinct incidents, anecdotes, or stated opinions about discrete topics....were then typed on 5 x 7 McBee-Keysort cards on which were printed general topical categories to guide coding. Collecting: Buford Rhea then read all the cards and tentatively classified them into the simple content categories we had decided upon in advance. Noticing: He then read them again so as to test, revise, and refine the initial gross classification.... Collecting: . .all cards bearing on some general substantive topic such as “patient relations” were removed from the total set of cards and put together in a pack. Noticing: All the cards in that large pack of between 800 and 1,200 were then read one by one.... Collecting: ...as they were read, the cards were sorted into preliminary topical piles. (Freidson, 1975: 270-271). Analysis is More than Coding, Sorting and Sifting The previous section suggests that disassembling, coding, and then sorting and sifting through your data, is the primary path to analysis. But as Michael Agar (1991) rightly cautions, intensive data coding, disassembly, sorting, and sifting, is neither the only way to analyze your data, nor is it necessarily the most appropriate strategy. I agree with this point. Later I will discuss Agar’s alternatives and suggest that they also fit the notice, collect, and think process. Qualitative Data Analysis 5 3. Thinking about Things In the thinking process you examine the things that you have collected. Your goals are: 1) to make some type of sense out of each collection, 2) look for patterns and relationships both within a collection, and also across collections, and 3) to make general discoveries about the phenomena you are researching. Examining the Pieces of a Puzzle Returning to the jigsaw puzzle analogy, after you sort the puzzle pieces into groups you inspect individual pieces to determine how they fit together and form smaller parts of the picture (e.g., the tree part or the house part). This is a labor intensive process that usually involves a lot of trial and error and frustration. A similar process takes place in the analysis of qualitative data. You compare and contrast each of the things you have noticed in order to discover similarities and differences, build typologies, or find sequences and patterns. In the process you might also stumble across both wholes and holes in the data. Problems with the Jigsaw Puzzle Analogy While the jigsaw puzzle approach to analyzing data can be productive and fruitful, it also entails some risks and problems. Experienced qualitative social scientists have always been aware of the potential problems, and organize their work to minimize the adverse effects. For example, Wiseman, who does code data, points out that the simple act of breaking down data into its constituent parts can distort and mislead the analyst. ...a serious problem is sometimes created by the very fact of organizing the material through coding or breaking it up into segments in that this destroys the totality of philosophy as expressed by the interviewee--which is closely related to the major goal of the study (Wiseman, 1979: 278). Part of the solution to this problem is as follows: To circumvent this problem, taped interviews were typed in duplicate. One copy was cut apart and affixed, by subject matter, to hand sort cards and then further cross-coded by coders....A second copy of the interview was left intact to be read in its entirety (Wiseman, 1979:278, my emphasis). In short, Wiseman protects her analysis by working back and forth between the parts and the whole of her data. Alternatives to the Jigsaw Puzzle Analogy One general problem with the jigsaw puzzle analogy is that it assumes that the best way to proceed is by intensive and inclusive coding of the data. It assumes that analytic discoveries directly follow from the process of coding and then sorting and sifting coded data. As I have already noted, and will discuss later, while this can be a good way to proceed it is not always the most appropriate or useful approach to analyzing qualitative data. Examples of Noticing, Collecting, Thinking The general process of Noticing, Collecting, and Thinking about things is reflected in many works which describe and discuss the practice of analyzing qualitative data. Four examples are presented below. In each example I have coded the text by breaking it up and inserting the terms Noticing, Collecting and Thinking into the text. This is one way of creating a “topographic” map of the text. While the fits are not always perfect, each statement is consistent with the model. 6 Qualitative Data Analysis Example 1 The first example comes from a description of QDA by Danny Jorgenson (1989). While this example repeats a previously quoted passage, this time I specifically identify the parts of the quote that correspond to the parts of the QDA process. Noticing: Analysis is a breaking up, separating, or disassembling of research materials into pieces, parts, elements, or units. Collecting: With facts broken down into manageable pieces, the researcher sorts and sifts them, Thinking: searching for types, classes, sequences, processes, patterns, or wholes. The aim of this process is to assemble or reconstruct the data in meaningful or comprehensible fashion (Jorgenson, 1989: 107, my emphasis). Example 2 Another example comes from a discussion of grounded theory by Corbin and Strauss (1990). Noticing/ Collecting: Open Coding is the part of analysis that pertains specifically to the naming and categorizing of phenomena through close examination of the data. ...During open coding the data are broken down into discrete parts, Thinking: closely examined, compared for similarities and differences, and questions are asked about the phenomena as reflected in the data (Corbin and Strauss, 1990: 62, my emphasis). Example 3 A more concrete description of the process is provided by Schneider and Conrad (1983). They describe the analysis of interviews they had collected in an interview study of epilepsy. In this example the codes emerged from the data. Noticing: We began coding the interviews by reading carefully a sample of the transcripts to develop substantive and general topic codes....We then photocopied the original transcripts, marked each appropriate line or section with the code in the margin, Collecting: and cut up and filed the pieces of paper according to the codes.... Thinking: Fairly early in our project it became apparent that the medical perspective on epilepsy did very little to describe our respondents experience ( Schneider and Conrad, 1983:242, my emphasis). Example 4 Finally, Spradley (1979) sketches the traditional process of anthropological field work. In this example, the noticing process is presented both on the general level of gathering data, and on the particular level of examining the data. “Sorting through field notes” implies noticing something Qualitative Data Analysis 7 that can then be collected. Noticing: And so the ethnographer started hanging around, watching, listening, and writing things down...In a few months, the stack of field notes about what people said and did grew quite large.... Noticing/ Collecting: The field work period drew to a close and the ethnographer returned home with notebooks filled with observations and interpretations. Sorting through field notes in the months that followed.... Thinking: the ethnographer compared, contrasted, analyzed, synthesized, and wrote (Spradley, 1979: 227, my emphasis). Alternatives to the Jigsaw Puzzle Analogy The jigsaw puzzle analogy assumes that analysis simply emerges out of coded, sorted and sifted data. But this is not always the case. Many times your analytical discoveries are only facilitated by, rather than transparently derived from, the way in which you have coded your data. The risk in following the jigsaw puzzle analogy too closely is that you might get deeply into the pieces and end up finding the codes but losing the phenomena. For example, if you just have the names of streets in a city, you know something about the city. For example, in some cities many streets are named after presidents. In others many streets are named after trees. But simply knowing the names of the streets doesn’t necessarily tell you much about the layout of the city, or how to get around in the city. For this you need a concrete representation of the streets in relationship to each other. Further, you need to be able to distinguish between neighborhood streets versus main traffic streets. Similarly, just having a collection of code words, or collections of coded segments of data, does not tell you everything you want and need to know about your data, and how the pieces of your data fit together. “A little bit of data and a lot of right brain” Michael Agar (1991) argues that while coding, segmenting, sorting and sifting data can be productive and useful strategies, there are other equally important strategies. The first alternative is to start by intensively examining a small bit of data, rather than intensively coding data. My point at the moment is just that this critical micro-level work requires looking at a few detailed passages, over and over again, doing the dialectic dance between an idea about how text is organized and a couple of examples, figuring out what I was looking at, how to look at it, and why (Agar, 1991: 190). That critical way of seeing, in my experience at least, comes out of numerous cycles through a little bit of data, massive amounts of thinking about that data, and slippery things like intuition and serendipity (Agar, 1991: 193). For that, you need a little bit of data, and a lot of right brain (Agar, 1991: 194). The question is, how do you come up with that “little bit of data?” Obviously you start by reading and rereading the data record. In the process you notice a few interesting things. You then collect one or more of these things and intensively think about them. So you are still within the basic model. I would like to carry this a step further and claim that when you identify and extract the 8 Qualitative Data Analysis segment with which you want to work, you are in fact coding the data. The difference is that you are not intensively coding, nor are you consumed by the sorting and sifting process. Parts in Context, “Patterns Among the Patterns” Agar’s second alternative is to look at coded, but unsorted, passages of data. This alternative seems to be consistent with intensively coding data. But it bypasses the sorting and sifting process. It goes directly from coding to discovery. The analysis is not built on sorting and sifting. Agar describes the process in the following way. I need to lay out a couple of stretches of transcript on a table so I can look at it all at once. Then I need to mark different parts in different ways to find the pattern that holds the text together and ties it to whatever external frame I’m developing. The software problem here would be simple to solve. You’d need to be able to quickly insert different colored marks of different kinds at different points so you could see the multiple connections across the text all at once, sort of a multi-threaded DNA laid on the text so you could look at the patterns that each thread revealed and then the patterns among the patterns (Agar, 1991:193). Here Agar is describing a process where you read and notice many things in the data record. Then you focus your attention on one part of the “coded” data record. This part can be chosen at random or deliberately. Intensively Analyzing a Small Piece of the Data Agar argues, in general, that QDA computer programs, such as The Ethnograph, are primarily oriented toward segmenting and sorting data, breaking down wholes into parts, and focusing attention on the collections of parts, at the expense of the wholes from which they come. Consequently QDA software biases the analyst toward segmenting and sorting, and away from intensive analysis of small bits of data, and away from viewing the parts in context. I would argue that The Ethnograph, at least, does in fact facilitate the two analytic alternatives proposed by Agar. In fact, The Ethnograph is unique in its ability to approximate Agar’s “multi- threaded DNA” model. In regard to the “little bit of data, lot of right brain” strategy, the coding and collecting of segments of data can provide the foundation for the process of intensive analysis of a small bit of data. For example, in order to find a piece of data to intensively analyze, Agar is still going through the process of noticing and collecting a piece of the data. When using QDA software the preliminary coding, and preliminary sorting and sifting, can generate pieces that become candidates for the intensive analysis described by Agar. The trick is to avoid intensive coding early in the analytic process. But even if you have done intensive coding you can always change the analytical direction, and shift your attention to a single piece of data for intensive analysis. In short, one approach does not preclude the other. In fact they can complement each other, and software can facilitate the shift to and from intensive analysis. An example comes from my own experience. A group of colleagues and I were analyzing data collected during a study of interactions between nurses and women during the process of giving birth. One interesting thing we noticed in our data was that the labor room nurse periodically talked about making “progress” during the birth process. We collected instances of “progress” talk and scheduled an analysis meeting on the topic. Qualitative Data Analysis 9 Figure 2. Example of Intensively Analyzed Data During an analysis meeting a team member would present one or two data segments. We also had access to the original coded transcript and the video from which it was transcribed. We would spend several hours analyzing and thinking about those segments. Each team member had a printout of the segment and would cover it with notes, thoughts, and scribbles. An example printout is shown in Figure 2. At the end of the session we would write up a preliminary memo summarizing our work. During analysis our attention was not restricted to the particular segment. For example, we might also examine and compare other “progress” segments with the segment we were analyzing. We would also look at the transcript from which the segment came so that we could place our analysis in the larger context from which the segment came. This context was not simply the immediately adjacent text within which the segment was embedded, but the entire event of which it was a part. This type of analytic process was not focused on gross analysis and summarization of a category of the data. Rather, it emerged out of preliminary coding and followed Agar’s prescription of working with “a little bit of data, and a lot of right brain” (Agar, 1991: 194). Sometimes the process took us beyond the topic of the segment. Sometimes it took us deeper into the topic. Threads and Patterns in the Data; Mapping the Data Landscape Agar’s second analysis alternative is the analogy of the “multi-threaded DNA laid on the text so you [can] look at the patterns that each thread reveal[s], and the patterns among the patterns”(Agar, 1991: 193). I argue that, because The Ethnograph lets you see your code words embedded in your 10 Qualitative Data Analysis data file, it directly facilitates this type of analysis. But before I address this I will offer two similar analogies: topographical maps and ad hoc maps. A topographical map is a way of coding the landscape so that it shows you the physical features of the landscape. It gives you a very different picture of the physical landscape compared to a standard road map. It shows you the hills and valleys, forests and clearings, and other features and details of the landscape in relationship to each other. This makes it easier for you to navigate through unfamiliar territory, especially off the roads. In a similar manner your “codes” can highlight features and details of your data landscape. The display of code words embedded in the data file (which The Ethnograph does) produces something resembling a “topographical” map of your data. Just as a real topographical map can help you discover and chart a path through the countryside, your codes can help you discover and chart patterns through the “landscape” of your data. These patterns are not reducible to code words, and are not discoverable from a simple examination of collections of coded segments. Yet these patterns can only be discovered because of the way in which you coded your data. An ad hoc map is the kind of map that you draw to tell people how to get to your house. When you draw this map you highlight (i.e., code) certain features of the landscape as reference points. For example, you might emphasize major intersections, stop lights, stores, etc. There are many intersections, stop lights, and stores in the area, but a particular combination of them mark the path to your house. In order to draw the map you have to know some general things about intersections, stop lights, and stores. But this general knowledge does not reveal the path to your house. Knowing and describing the path requires a knowledge of specific intersections, stoplights, and stores. Thus, describing the path depends on coded features of the landscape, but the path is not reducible to the coded features, nor is it revealed by studying collections of those features of the landscape. A practical example from my own work illustrates the “threading” or “map” metaphors. It comes from another analysis session on the second stage labor project. One day my colleagues and I focused on a data fragment where the nurse displayed the “three push rule” to the laboring woman. The plan was to intensively analyze this small piece of data. While two of us attended to this data fragment the third member of the team drifted away from the discussion and started looking at the fully coded transcript from which the segment came. She noticed the co-occurrence of a “praise” utterance with the “three push rule” display. She also noticed that this came at the end of a uterine contraction. Going back through many pages of the transcript she noticed the absence of a “praise” utterance during all the previous contractions. After she brought this to our attention we followed the patterns backwards to another pivotal event, and then forward to the “three push rule” display. In this way we discovered a new phenomenon, a “progress crisis,” which cut across, and transcended, the coded segments on the transcript. This discovery depended on the fact that we had coded our transcript in a particular way, but the discovery was not reducible to the codes, nor could it have been derived by simply inspecting collections of coded segments. Because of the way we had coded the data, the features of the data landscape were highlighted on the transcript, the “threads” were visible. Through the combination of: 1) focused attention, 2) intensive analysis of a small part of the data, and 3) the ability to see the how several “threads” or “features” of the data came together over several pages in the transcript, we were able to make the “progress crisis” discovery. Qualitative Data Analysis 11 Figure 3. A Model of Qualitative Data Analysis A Complex Model of the QDA Process The diagram in Figure 3 is a model of the features of the QDA process, and how a computer program … QUALITATIVE RESEARCH METHODOLOGIES Qualitative Research Chapter Nine Qualitative Research  There are several different approaches to qualitative research  The researcher is the instrument  There are multiple perspectives, with each having equal validity, or truth  Is found in many academic disciplines  Many researchers believe that all inquiry starts out in qualitative form  Questions at the beginning tend to be open-ended  Requires considerable preparation and planning When to Choose a Qualitative Approach Qualitative research studies typically serve one or more or the following functions:  Description  Interpretation  Verification  Evaluation Generally, qualitative studies do not allow the researcher to identify cause-and-effect relationships. Five Common Research Designs  Case Study  Ethnography  Phenomenological Study  Grounded Theory  Content Analysis Case Study  In a case study (sometimes called idiographic research) a particular individual, program, or event is studied in depth for a defined period of time.  Can focus on an individual case, or two or more cases (multiple case study)  A single case study cannot result in generalizing to other situations  See text pages 141-142 for study details Ethnography  An ethnography looks in depth at an entire group (usually one that shares a common culture)  The group is studied in its natural setting for a lengthy time period  The focus is on everyday behaviors or people in the group  The researcher identifies explicit and implicit patterns  Useful for gaining an understanding of the complexities of a particular group  See Text pages 142-144 for study details Phenomenological Study  Refers to a person’s perception of the meaning of an event  Attempts to understand people’s perceptions, perspectives, and understandings of a particular situation  By looking at multiple perspectives on the same situation, the researcher can then make some generalizations of what something is like from an insider’s perspective  See Text pages 145-146 for study details Grounded Theory Study  The grounded theory study is the one (study) least likely to begin with a theoretical framework  The major purpose is to begin with the data and use them to develop a theory  Typically the focus is on a process related to a particular topic  This approach has its roots in sociology but is used in other fields  See Text pages 256-257 for study details Content Analysis  This is a detailed and systematic examination of the contents of a particular body of material for the purpose of identifying patterns, themes, or biases  Usually performed on forms of communication  This approach requires the greatest amount of planning at the front end of the project  These are not necessarily stand-alone designs  See Text pages 257-258 for study details Data Collection Qualitative studies are characterized by an emerging design  Sampling  Observations  Interviews Sampling  Draw data from many sources  The particular entities (of data) selected for analysis comprise the sample, and the process of selecting them is called sampling  The sample chosen is dependent on the questions to be answered  Random selection  Theoretical sampling  The sample should provide information not only about how things are on average but also about how much variability exists in the phenomenon under investigation Observations  May make observations either as a relative outsider or, especially in the case of an ethnography, as a participant observer  Observations are intentionally unstructured and free- flowing  Flexible  Sometimes difficult to know what to look for  Drawback – the presence of an observer may influence people  It is essential that the actual observations are not confused with the interpretations of them Interviews  Questions can be asked about: facts, beliefs and perspectives, feelings, motives, present and past behaviors, standards for behavior, and conscious reasons for actions or feelings  May be open-ended or semi-structured  May be individual or focus groups Organizing and Analyzing Data No single correct way to analyze data  Data analysis spiral (text page 297-298)  Raw data  Organization  Perusal  Classification  Synthesis  Final Report Criteria for Evaluating Qualitative Research A variety of standards are used  Purposefulness  Explicitness of assumptions and biases  Rigor  Open-mindedness  Completeness  Coherence  Persuasiveness  Consensus  Usefulness