Fluent Essays

Prior to beginning work on this discussion, read Chapters 1 and 2 of the Hesse-Biber e-book, Mixed Methods Research: Merging Theory with Practice, and the two required articles for this week.  Mixed methods is a current popular methodology. While this type of methodology is useful for some studies, because of its dual nature as both quantitative and qualitative, it is not effective or appropriate for all research. For this discussion, you will consider the use of mixed methods for the topic chosen for the Research Proposal which is “Employment Issues Amongst Mentally Disabled Veterans”. For this PLEASE ALL ADDRESS THE FOLLOWING POINTS. apply the scientific method to your research topic by defining your research question and determining the method(s) necessary to answer that question.  Compare the characteristics and appropriate uses of the different methods and explain if your research question could best be answered through qualitative or quantitative methods, or a mix of both.  Identify the dominant method (quantitative or qualitative) for your proposed study.  Explain whether or not a mixed methods approach is the best way to study the topic, demonstrating that the second method is not added as an afterthought or merely to impress journal editors who favor mixed methods.  Justify your design choice and support your position with scholarly sources.  Include a discussion explaining how you would apply ethical principles to your design to address concerns which may impact your research. NOTE: PLEASE SEE THE ATTACHMENT Research Question Research Design Selected for the research proposal question and research design selected. Citation for two documents attached that can be used for this. Bryman, A. (2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6(1), 97–113. https://doi-org.proxy-library.ashford.edu/10.1177/1468794106058877 Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013, December 1). Achieving integration in mixed methods designs--principles and practices. Health Services Research, 48(6). https://doi-org.proxy-library.ashford.edu/10.1111/1475-6773.12117 Research question and Hypothesis       The topic I have selected for my research is “Employment Issues Amongst Mentally Disabled Veterans” as mental issues are the most rising issues and have a serious impact on society. The aim and objective of this research are to provide better opportunities to mentally disabled veterans so that they could engage themselves with decent jobs. The research question for this study is why employment issues are very common among mentally disabled veterans apart from the presence of specific organizations for them? As their main role is to keep them employed. My hypothesis for this study is due to increased modernization, negative stereotypes and stigmas about mentally disabled veterans and increased demands of jobs; many employers usually prefer younger working employees that do not have military transitional to civilian work life issues especially mentally disabled veterans who are compromised in their proper job functioning. According to Cook (2006), research studies show that many individuals which includes veterans with disabling mental disorders desires paid employment, consider themselves able to work, and strongly expresses the need for job training programs, vocational rehabilitation services, and other support services (Cook, J. A. (2006). Hence this study will help create awareness especially amongst the populace who have negative stigmas against mentally disabled veterans in civilian workplaces (Carden-Coyne, 2007). Additionally, this study will also help with available strategies for employment programs that will assist with the hiring of mentally disabled veterans by providing them with better learning skills which will increase the chances of long-term employment (Deahl et al;2011)                                       Research Methodology selection       For this research work, I will select the mixed research method with the triangular-based framework design as it will be helpful in the development of theory then testing it as well.  This method helps in developing reasoning and suggests effective ways that how that reasoning can be answered effectively. This methodology includes a combination of various techniques which will be helpful in the continuation of my research work. After the development of the hypothesis, a theory will be developed, and subsequently, the collection of data and attestation of the theory that was proposed will be done. Achieving Integration in Mixed Methods Designs—Principles and Practices Michael D. Fetters, Leslie A. Curry, and John W. Creswell Abstract. Mixed methods research offers powerful tools for investigating complex processes and systems in health and health care. This article describes integration prin- ciples and practices at three levels in mixed methods research and provides illustrative examples. Integration at the study design level occurs through three basic mixed method designs—exploratory sequential, explanatory sequential, and convergent— and through four advanced frameworks—multistage, intervention, case study, and par- ticipatory. Integration at the methods level occurs through four approaches. In con- necting, one database links to the other through sampling. With building, one database informs the data collection approach of the other. When merging, the two databases are brought together for analysis. With embedding, data collection and analysis link at multiple points. Integration at the interpretation and reporting level occurs through narrative, data transformation, and joint display. The fit of integration describes the extent the qualitative and quantitative findings cohere. Understanding these principles and practices of integration can help health services researchers leverage the strengths of mixed methods. Key Words. Qualitative research, survey, sampling, focus groups, biostatistical methods, epidemiology, program evaluation, research methodology This article examines key integration principles and practices in mixed meth- ods research. It begins with the role of mixed methods in health services research and the rationale for integration. Next, a series of principles describe how integration occurs at the study design level, the method level, and the interpretation and reporting level. After considering the “fit” of integrated qualitative and quantitative data, the article ends with two examples of mixed methods investigations to illustrate integration practices. Research Questions and Mixed Methods in Health Services Research Health services research includes investigation of complex, multilevel pro- cesses, and systems that may require both quantitative and qualitative forms © Health Research and Educational Trust DOI: 10.1111/1475-6773.12117 INTEGRATING MIXED METHODS IN HEALTH SERVICES AND DELIVERY SYSTEM RESEARCH 2134 Health Services Research of data (Creswell, Fetters, and Ivankova 2004; Curry et al. 2013). The nature of the research question drives the choice of methods. Health services researchers use quantitative methodologies to address research questions about causality, generalizability, or magnitude of effects. Qualitative method- ologies are applied to research questions to explore why or how a phenome- non occurs, to develop a theory, or to describe the nature of an individual’s experience. Mixed methods research studies draw upon the strengths of both quantitative and qualitative approaches and provides an innovative approach for addressing contemporary issues in health services. As one indication of the growing interest in mixed methods research, the Office of Behavioral and Social Sciences at the National Institutes of Health recently developed for researchers and grant reviewers the first best practices guideline on mixed methods research from the National Institutes of Health (Creswell et al. 2011). Rationale for Integration The integration of quantitative and qualitative data can dramatically enhance the value of mixed methods research (Bryman 2006; Creswell and Plano Clark 2011). Several advantages can accrue from integrating the two forms of data. The qualitative data can be used to assess the validity of quantitative findings. Quantitative data can also be used to help generate the qualitative sample or explain findings from the qualitative data. Qualitative inquiry can inform development or refinement of quantitative instruments or interven- tions, or generate hypotheses in the qualitative component for testing in the quantitative component (O’Cathain, Murphy, and Nicholl 2010). Although there are many potential gains from data integration, the extent to which mixed methods studies implement integration remains limited (Bryman 2006; Lewin, Glenton, and Oxman 2009). Nevertheless, there are specific approaches to integrate qualitative and quantitative research procedures and data (O’Cathain, Murphy, and Nicholl 2010; Creswell and Plano Clark 2011). These approaches can be implemented at the design, methods, and interpretation and reporting levels of research (see Table 1). Address correspondence to Michael D. Fetters, M.D., M.P.H., M.A., Family Medicine, University of Michigan, 1018 Fuller St., Ann Arbor, MI 48104-1213; e-mail: [email protected] Leslie A. Curry, Ph.D., M.P.H., is with the Yale School of Public Health (Health Policy), New Haven, CT. John W. Creswell, Ph.D., is with Educational Psychology, University of Nebraska-Lincoln, Lin- coln, NE. Achieving Integration in Mixed Methods Designs 2135 Integration at the Study Design Level Integration at the design level—the conceptualization of a study—can be accomplished through three basic designs and four advanced mixed methods frameworks that incorporate one of the basic designs. Basic designs include (1) exploratory sequential; (2) explanatory sequential; and (3) convergent designs. In sequential designs, the intent is to have one phase of the mixed methods study build on the other, whereas in the convergent designs the intent is to merge the phases in order that the quantitative and qualitative results can be compared. In an exploratory sequential design, the researcher first collects and analyzes qualitative data, and these findings inform subsequent quantitative data collec- tion (Onwuegbuzie, Bustamante, and Nelson 2010). For example, Wallace and colleagues conducted semistructured interviews with medical students, residents, and faculty about computing devices in medical education and used the qualitative data to identify key concepts subsequently measured in an online survey (Wallace, Clark, and White 2012). In an explanatory sequential design, the researcher first collects and ana- lyzes quantitative data, then the findings inform qualitative data collection and analysis (Ivankova, Creswell, and Stick 2006). For example, Carr explored the impact of pain on patient outcomes following surgery by conducting initial Table 1: Levels of Integration in Mixed Methods Research Integration Level Approaches Design 3 Basic designs Exploratory sequential Explanatory sequential Convergent 4 Advanced frameworks Multistage Intervention Case study Participatory—Community-based participatory research, and transformative Methods Connecting Building Merging Embedding Interpretation and Reporting Narrative—Weaving, contiguous and staged Data transformation Joint display 2136 HSR: Health Services Research 48:6, Part II (December 2013) surveys about anxiety, depression, and pain that were followed by semistruc- tured interviews to explore further these concepts (Carr 2000). In a convergent design (sometimes referred to as a concurrent design), the qualitative and quantitative data are collected and analyzed during a similar timeframe. During this timeframe, an interactive approach may be used where iteratively data collection and analysis drives changes in the data collection procedures. For example, initial quantitative findings may influence the focus and kinds of qualitative data that are being collected or vice versa. For exam- ple, in one study Crabtree and colleagues used qualitative findings and quanti- tative findings iteratively in multiple phases such that the data were interacting to inform the final results (Crabtree et al. 2005). In the more common and technically simpler variation, qualitative and quantitative data collection occurs in parallel and analysis for integration begins well after the data collec- tion process has proceeded or has been completed. Frequently, the two forms of data are analyzed separately and then merged. For example, Saint Arnault and colleagues conducted multiple surveys using standardized and culturally adapted instruments as well as ethnographic qualitative interviews to investi- gate how the illness experience, cultural interpretations, and social structural factors interact to influence help-seeking among Japanese women (Saint Arna- ult and Fetters 2011). Advanced frameworks encompass adding to one of the three basic designs a larger framework that incorporates the basic design. The larger framework may involve (1) a multistage; (2) an intervention; (3) a case study; or (4) a participatory research framework. In a multistage mixed methods framework, researchers use multiple stages of data collection that may include various combinations of exploratory sequential, explanatory sequential, and convergent approaches (Nastasi et al. 2007). By definition, such investigations will have multiple stages, defined here as three or more stages when there is a sequential component, or two or more stages when there is a convergent component; these differences distin- guishes the multistage framework from the basic mixed methods designs. This type of framework may be used in longitudinal studies focused on eval- uating the design, implementation, and assessment of a program or interven- tion. Krumholz and colleagues have used this design in large-scale outcomes research studies (Krumholz, Curry, and Bradley 2011). For example, a study by their team examining quality of hospital care for patients after heart attacks consisted of three phases: first, a quantitative analysis of risk-stan- dardized mortality rates for patients with heart attacks to identify high and low performing hospitals; second, a qualitative phase to understand the pro- Achieving Integration in Mixed Methods Designs 2137 cesses, structures, and organizational environments of a purposeful sample of low and high performers and to generate hypotheses about factors associated with performance; and third, primary data collection through surveys of a nationally representative sample of hospitals to test these hypotheses quanti- tatively (Curry et al. 2011; Bradley et al. 2012). Ruffin and colleagues con- ducted a multistage mixed methods study to develop and test in a randomized controlled trial (RCT) a website to help users choose a screening approach to colorectal cancer. In the first stage, the authors employed a con- vergent design using focus groups and a survey (Ruffin et al. 2009). In the second stage, they developed the website based on multiple qualitative approaches (Fetters et al. 2004). In the third stage, the authors tested the website in an RCT to assess its effectiveness (Ruffin, Fetters, and Jimbo 2007). The multistage framework is the most general framework among advanced designs. The additional three frameworks frequently involve multi- ple stages or phases but differ from multistage by having a particular focus. In an intervention mixed methods framework, the focus is on conducting a mixed methods intervention. Qualitative data are collected primarily to sup- port the development of the intervention, to understand contextual factors during the intervention that could affect the outcome, and/or explain results after the intervention is completed (Creswell et al. 2009; Lewin, Glenton, and Oxman 2009). For example, Plano Clark and colleagues utilized data from a pretrial qualitative study to inform the design of a trial developed to compare a low dose and high dose behavioral intervention to improve cancer pain management—the trial also included prospective qualitative data collection during the trial (Plano Clark et al. 2013). The methodological approach for inte- grating qualitative data into an intervention pretrial, during the trial, or post- trial is called embedding (see below), and some authors refer to such trials as embedded designs (Creswell et al. 2009; Lewin, Glenton, and Oxman 2009). In a case study framework, both qualitative and quantitative data are col- lected to build a comprehensive understanding of a case, the focus of the study (Yin 1984; Stake 1995). Case study involves intensive and detailed qualitative and quantitative data collection about the case (Luck, Jackson, and Usher 2006). The types of qualitative and quantitative data collected are chosen based on the nature of the case, feasibility issues, and the research question(s). In one mixed methods case study, Luck and colleagues utilized qualitative data from participant observation, semistructured interviews, informal field interviews and journaling, and quantitative data about violent events collected through structured observations to understand why nurses under-report vio- lence in the workplace and describe how they handle it (Luck, Jackson, and 2138 HSR: Health Services Research 48:6, Part II (December 2013) Usher 2008). Comparative case studies are an extension of this framework and can be formulated in various ways. For example, Crabtree and colleagues used a comparative case approach to examine the delivery of clinical preven- tive services in family medicine offices (Crabtree et al. 2005). In a participatory framework, the focus is on involving the voices of the tar- geted population in the research to inform the direction of the research. Often researchers specifically seek to address inequity, health disparities, or a social injustice through empowering marginalized or underrepresented populations. The distinguishing feature of a participatory framework is the strong emphasis on using mixed methods data collection through combinations of basic mixed methods designs or even another advanced design, for example, an interven- tion framework such as an RCT. Community-based participatory research (CBPR) is a participatory framework that focuses on social, structural, and physical environmental inequities and engages community members, organizational representatives, and researchers in all aspects of the research process (Macau- lay et al. 1999; Israel et al. 2001, 2013; Minkler and Wallerstein 2008). In one CBPR project, Johnson and colleagues used a mixed methods CBPR approach to collaborate with the Somali community to explore how attitudes, perceptions, and cultural practices such as female genital cutting influence their use of reproductive health services—this informed the development of interventional programs to improve culturally competent care (Johnson, Ali, and Shipp 2009). A similar variation involving an emerging participatory approach that Mertens refers to as transformative specifically focuses on pro- moting social justice (Mertens 2009, 2012) and has been used with Laotian refugees (Silka 2009). Integration at the Methods Level Creswell and Plano Clark conceptualize integration to occur through linking the methods of data collection and analysis (Creswell et al. 2011). Linking occurs in several ways: (1) connecting; (2) building; (3) merging; and (4) embedding (Table 2). In a single line of inquiry, integration may occur through one or more of these approaches. Integration through connecting occurs when one type of data links with the other through the sampling frame. For example, consider a study with a sur- vey and qualitative interviews. The interview participants are selected from the population of participants who responded to the survey. Connecting can occur through sampling regardless of whether the design is explanatory sequential or convergent. That is, if the baseline survey data are analyzed, and Achieving Integration in Mixed Methods Designs 2139 then the participants sampled based on findings from the analysis, then the design is explanatory sequential. In contrast, the design is convergent if the data collection and analyses occur at the same time for the baseline survey and interviews of all or a subsample of the participants of the survey. A key defin- ing factor in sequential or convergent is how the analysis occurs, either through building or merging, respectively. Integration through building occurs when results from one data collec- tion procedure informs the data collection approach of the other procedure, the lat- ter building on the former. Items for inclusion in a survey are built upon previously collected qualitative data that generate hypotheses or identify con- structs or language used by research participants. For example, in a project involving the cultural adaptation of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey for use in the Arabian Gulf (Hammoud et al. 2012), baseline qualitative interviews identified new domains of importance such as gender relations, diet, and interpreter use not found in the existing CAHPS instrument. In addition, phrases participants used during the interviews informed the wording of individual items. Integration through merging of data occurs when researchers bring the two databases together for analysis and for comparison. Ideally, at the design phase, researchers develop a plan for collecting both forms of data in a way that will be conducive to merging the databases. For example, if quantitative data are collected with an instrument with a series of scales, qualitative data can be collected using parallel or similar questions (Castro et al. 2010). Merging typically occurs after the statistical analysis of the numerical data and qualitative analysis of the textual data. For example, in a multistage mixed methods study, Tomoaia-Cortisel and colleagues used multiple sources of existing quantitative and qualitative data as well as newly col- lected quantitative and qualitative data (Tomoaia-Cortisel et al. 2013). The researchers examined the relationship between quality of care according to key patient-centered medical home (PCMH) measures, and quantity of care using a productivity measure. By merging both scores of quality and Table 2: Integration through Methods Approach Description Connecting One database links to the other through sampling Building One database informs the data collection approach of the other Merging The two databases are brought together for analysis Embedding Data collection and analysis link at multiple points 2140 HSR: Health Services Research 48:6, Part II (December 2013) quantity, with qualitative data from interviews, the authors illuminated the difficulty of achieving highly on both PCMH quality measures and produc- tivity. The authors extended this understanding further by merging staff satisfaction scores and staff interview data to illustrate the greater work com- plexity but lower satisfaction for staff achieving measures for high-quality care (Tomoaia-Cortisel et al. 2013). Integration through embedding occurs when data collection and analysis are being linked at multiple points and is especially important in interventional advanced designs, but it can also occur in other designs. Embedding may involve any combination of connecting, building, or merging, but the hall- mark is recurrently linking qualitative data collection to quantitative data collection at multiple points. Embedding may occur in the pretrial period, when qualitative (or even a combination of qualitative and quantitative) data can be used in various ways such as clarifying outcome measures, under- standing contextual factors that could lead to bias and should be controlled for, or for developing measurement tools to be utilized during the trial. Dur- ing the trial, qualitative data collection can be used to understand contextual factors that could influence the trial results or provide detailed information about the nature of the experience of subjects. Post-trial qualitative data col- lection can be used to explain outliers, debrief subjects or researchers about events or experiences that occurred during the trial, or develop hypotheses about changes that might be necessary for widespread implementation out- side of a controlled research environment. Such studies require caution to avoid threatening the validity of the trial design. In a site-level controlled trial of a quality improvement approach for implementing evidence-based employment services for patients at specialty mental health clinics, Hamil- ton and colleagues collected semistructured interview data before, during, and after implementation (Hamilton et al. 2013). In another interesting example, Jaen and colleagues used an embedded approach for evaluating practice change in a trial comparing facilitated and self-directed implementa- tion strategies for PCMH. The authors use both embedded quantitative and qualitative evaluation procedures including medical record audit, patient and staff surveys, direct observation, interviews, and text review (Jaen et al. 2010). Method level integration commonly relates to the type of design used in a study. For example, connecting follows naturally in sequential designs, while merging can occur in any design. Embedding generally occurs in an interven- tional design. Thus, the design sets parameters for what methodological inte- gration choices can be made. Achieving Integration in Mixed Methods Designs 2141 Integration at the Interpretation and Reporting Level Integration of qualitative and quantitative data at the interpretation and reporting level occurs through three approaches: (1) integrating through nar- rative; (2) integrating through data transformation; and (3) integrating through joint displays. A variety of strategies have been offered for publishing that incorporate these approaches (Stange, Crabtree, and Miller 2006; Creswell and Tashakkori 2007). When integrating through narrative, researchers describe the qualitative and quantitative findings in a single or series of reports. There are three approaches to integration through narrative in research reports. The weaving approach involves writing both qualitative and quantitative findings together on a theme-by-theme or concept-by-concept basis. For example, in their work on vehicle crashes among the elderly, Classen and colleagues used a weaving approach to integrate results from a national crash dataset and perspectives of stakeholders to summarize causative factors of vehicle crashes and develop empirical guidelines for public health interventions (Classen et al. 2007). The contiguous approach to integration involves the presentation of findings within a single report, but the qualitative and quantitative findings are reported in different sections. For example, Carr and colleagues reported survey findings in the first half of the results section and the qualitative results about contextual factors in a subsequent part of the report (Carr 2000). In their study of a quality improvement approach for implementing evidence-based employment services at specialty mental health clinics, Hamilton and colleagues used this approach but differ by presenting the qualitative results first and the quantita- tive results second (Hamilton et al. 2013). The staged approach to integration often occurs in multistage mixed methods studies when the results of each step are reported in stages as the data are analyzed and published separately. For example, Wilson and colleagues used an intervention mixed methods frame- work involving a clinical trial of usual care, nicotine gum, and gum plus coun- seling on smoking cessation (Wilson et al. 1988). They also used interviews to find the meaning patients attributed to their stopping smoking (Willms 1991). The authors published the papers separately but in the second published paper, the interview paper, they only briefly mention the original clinical trial paper. Integration through data transformation happens in two steps. First, one type of data must be converted into the other type of data (i.e., qualitative into quantitative or quantitative into qualitative). Second, the transformed data are then integrated with the data that have not been transformed. In qualitative 2142 HSR: Health Services Research 48:6, Part II (December 2013) studies, researchers sometimes code the qualitative data and then count the frequency of codes or domains identified, a process known also as content analysis (Krippendorff 2013). Data transformation in the mixed methods con- text refers to transforming the qualitative data into numeric counts and vari- ables using content analysis so that the data can be integrated with a quantitative database. Merging in mixed methods goes beyond content analy- sis by comparing the transformed qualitative data with a quantitative database. Zickmund and colleagues used qualitatively elicited patient views of self trans- formed to a numerical variable, and mortality data to conduct hierarchical multivariable logistical modeling (Zickmund et al. 2013). Researchers have used additional variations. Qualitative data can be transformed to quantitative data, then integrated with illustrative examples from the original qualitative dataset. For example, Ruffin and colleagues trans- formed qualitative responses from focus group data about colorectal cancer (CRC) screening preferences into quantitative variables, and then integrated these findings with representative quotations from three different constituen- cies (Ruffin et al. 2009). Quantitative data can also be transformed into a quali- tative format that could be used for comparison with qualitatively accessed data. For example, Pluye and colleagues examined a series of study outcomes with variable strengths of association that were converted into qualitative levels and compared across the studies based on patterns found (Pluye et al. 2005). When integrating through joint displays, researchers integrate the data by bringing the data together through a visual means to draw out new insights beyond the information gained from the separate quantitative and qualitative results. This can occur through organizing related data in a figure, table, matrix, or graph. In their quality improvement study to enhance colorectal cancer screening in practices, Shaw and colleagues collocated a series of quali- tatively identified factors with CRC screening rates at baseline and 12 months later (Shaw et al. 2013). “Fit” of Data Integration When using any of these analytical and representation procedures, a potential question of coherence of the quantitative and qualitative findings may occur. The “fit” of data integration refers to coherence of the quantitative and qualita- tive findings. The assessment of fit of integration leads to three possible out- comes. Confirmation occurs when the findings from both types of data confirm the results of the other. As the two data sources provide similar conclusions, the results have greater credibility. Expansion occurs when the findings from Achieving Integration in Mixed Methods Designs 2143 the two sources of data diverge and expand insights of the phenomenon of interest by addressing different aspects of a single phenomenon or by describ- ing complementary aspects of a central phenomenon of interest. For example, quantitative data may speak to the strength of associations while qualitative data may speak to the nature of those associations. Discordance occurs if the qualitative and quantitative findings are inconsistent, incongruous, contradict, conflict, or disagree with each other. Options for reporting the findings include looking for potential sources of bias, and examining methodological assump- tions and procedures. Investigators may handle discordant results in different ways such as gathering additional data, re-analyzing existing databases to resolve differences, seeking explanations from theory, or challenging the validity of the constructs. Further analysis may occur with the existing data- bases or in follow-up studies. Authors deal with this conundrum by discussing reasons for the conflicting results, identifying potential explanations from the- ory, and laying out future research options (Pluye et al. 2005; Moffatt et al. 2006). Examples Illustrating Integration Below, two examples of mixed methods illustrate the integration practices. The first study used an exploratory sequential mixed methods design (Curry et al. 2011) and the second used a convergent mixed methods design (Meurer et al. 2012). Example 1. Integration in an Exploratory Sequential Mixed Methods Study—The Survival after Acute Myocardial Infarction Study (American College of Cardiology 2013). Despite more than a decade of efforts to improve care for patients with acute myocardial infarction (AMI), there remains substantial variation across hospitals in mortality rates for patients with AMI (Krumholz et al. 2009; Popescu et al. 2009). Yet the vast majority of this variation remains unex- plained (Bradley et al. 2012), and little is known about how hospitals achieve reductions in risk-standardized mortality rates (RSMRs) for patients with AMI. This study sought to understand diverse and complex aspects of AMI care including hospital structures (e.g., emergency department space), pro- cesses (e.g., emergency response protocols, coordination within hospital units), and hospital internal environments (e.g., organizational culture). Integration through design. An exploratory sequential mixed methods design using both qualitative and quantitative approaches was best suited to 2144 HSR: Health Services Research 48:6, Part II (December 2013) gain a comprehensive understanding of how these features may be related to quality of AMI care as reflected in RSMRs. The 4-year investigation aimed to first generate and then empirically test hypotheses concerning hospital-based efforts that may be associated with lower RSMRs (Figure 1). Integration through methods. The … A L A N B RY M A N University of Leicester A B S T R A C T This article seeks to move beyond typologies of the ways in which quantitative and qualitative research are integrated to an examination of the ways that they are combined in practice. The article is based on a content analysis of 232 social science articles in which the two were combined. An examination of the research methods and research designs employed suggests that on the quantitative side structured interview and questionnaire research within a cross-sectional design tends to predominate, while on the qualitative side the semi-structured interview within a cross-sectional design tends to predominate. An examination of the rationales that are given for employing a mixed-methods research approach and the ways it is used in practice indicates that the two do not always correspond. The implications of this finding for how we think about mixed-methods research are outlined. K E Y W O R D S : qualitative research, quantitative research, mixed-methods research, multi-strategy research, typologies There can be little doubt that research that involves the integration of quanti- tative and qualitative research has become increasingly common in recent years. While some writers express unease about the ‘whatever works’ position that underpins it (e.g. Buchanan, 1992; Pawson and Tilly, 1997), so far as research practice is concerned, combining quantitative and qualitative research has become unexceptional and unremarkable in recent years. Indeed, for some writers it has come to be seen as a distinctive research approach in its own right that warrants comparison with each of quantitative and qualitative research. In this sense, we end up with three distinct approaches to research: quantitative; qualitative; and what is variously called multi-methods (Brannen, 1992), multi-strategy (Bryman, 2004), mixed methods (Creswell, A R T I C L E 97 DOI: 10.1177/1468794106058877 Integrating quantitative and qualitative research: how is it done? Q R Qualitative Research Copyright © 2006 SAGE Publications (London, Thousand Oaks, CA and New Delhi) vol. 6(1) 97–113. 2003; Tashakkori and Teddlie, 2003), or mixed methodology (Tashakkori and Teddlie, 1998) research. In the field of evaluation research, and indeed in several other applied fields, the case for a multi-strategy research approach seems to have acquired especially strong support (Tashakkori and Teddlie, 2003). Typologies of mixed-methods research The discussion of the integration of quantitative and qualitative research has increasingly been taken over by a formalized approach which is especially apparent in the discussion and proliferation of typologies of integration. This has been a particular emphasis among North American contributors to the field. Creswell et al. (2003) argue that giving types of mixed-methods research names has certain advantages. It conveys a sense of the rigour of the research and provides guidance to others about what researchers intend to do or have done (for example, funding bodies and journal editors). To that extent, the typologies of mixed-methods or multi-strategy research can be helpful to researchers and writers in clarifying the nature of their intentions or of their accomplishments. However, the variety and range of typologies has reached the point where these exercises have become almost too refined, bearing in mind that the range of concrete examples of multi-strategy research is not great. Indeed, most of the typologies have been constructed in largely theoretical terms and have not been apparently influenced in a systematic way by examples of multi-strategy research. To a large extent, they are exercises in logically possible types of inte- gration, rather than being built up out of examples. However, the dimensions out of which the typologies are constructed are instructive, in that they draw attention to the different aspects of multi- strategy research: 1. Are the quantitative and qualitative data collected simultaneously or sequentially? (Morgan, 1998; Morse, 1991). 2. Which has priority – the quantitative or the qualitative data? (Morgan, 1998; Morse, 1991). 3. What is the function of the integration – for example, triangulation, explanation, or exploration? (Creswell, 2003; Creswell et al., 2003; Greene et al., 1989). 4. At what stage(s) in the research process does multi-strategy research occur? (Tashakkori and Teddlie, 1998). It may be at stages of research question formulation, data collection, data analysis, or data interpret- ation. 5. Is there more than one data strand? (Tashakkori and Teddlie, 2003). With a multi-strand study, there is more than one research method and hence source of data. With a mono-strand study, there is one research method Qualitative Research 6(1)98 and hence one source of data. However, whether a mono-strand study can genuinely be regarded as a form of mixing methods is debatable. A further issue with the use of these typologies is that they imply some forward commitment to a type of design, much like a decision to employ an experimental research design entails a commitment to uncovering data of a particular type. However, as some authors observe (e.g. Erzberger and Kelle, 2003), the outcomes of multi-strategy research are not always predictable. While a decision about design issues may be made in advance and for good reasons, when the data are generated, surprising findings or unrealized poten- tial in the data may suggest unanticipated consequences of combining them. What do we know about mixing quantitative and qualitative research? The exercise of specifying typologies co-exists with unease among some authors about what we actually know about the ways in which quantitative and qualitative research are combined in practice. For example, it has been suggested that there are relatively few guidelines about ‘how, when and why different research methods might be combined’ (Bryman, 1988: 155). Maxwell has suggested that ‘the theoretical debate about combining methods has prevented us from seeing the different ways in which researchers are actually combining methods’ (Maxwell, 1990: 507, cited in Maxwell and Loomis, 2003: 251). He and Loomis have argued further that: Uncovering the actual integration of qualitative and quantitative approaches in any particular study is a considerably more complex undertaking than simply classifying the study into a particular category on the basis of a few broad dimen- sions or characteristics. (Maxwell and Loomis, 2003: 256) Remarks such as these suggest that the formalization of approaches to multi- strategy research through typologies has moved too far ahead of a systematic appreciation of how quantitative and qualitative research are combined in practice. The writers who adopt a formalized strategy use many examples to illustrate their ‘types’ but we have relatively little understanding of the prevalence of different combinations, though there are some exceptions to this statement (e.g. Greene et al., 1989; Niglas, 2004). An investigation of articles combining quantitative and qualitative research With these kinds of consideration in mind, an investigation was undertaken of the ways that quantitative and qualitative research are combined in published journal articles. The findings reported in this article derive from only one phase of this research project, albeit a major component of it – namely, a content Bryman: Integrating quantitative and qualitative research 99 analysis of articles based on multi-strategy research. Journal articles do not encapsulate all possible contexts in which projects reporting multi-strategy research might be found. Conference papers and books are other possible sites. However, journal articles are a major form of reporting findings and have the advantage that, in most cases, the peer review process provides a quality control mechanism. By contrast, conference papers and books are sometimes not peer reviewed. The approach to gleaning a sample was to search the Social Sciences Citation Index (SSCI) for articles in which relevant key words or phrases such as ‘quantitative’ and ‘qualitative’, or ‘multi(-)method’, or ‘mixed method’, or ‘triangulation’ appeared in the title, key words, or abstract. This means that the sample comprises articles which to some degree foreground the fact that the study is based on both quantitative and qualitative research. Searches using other kinds of key words, such as ‘survey’ and ‘ethnography/ic’, produce a far larger sample of articles than could be dealt with within the purview of this investigation. In conducting the search, the emphasis was on uncovering articles in five fields: sociology; social psychology; human, social and cultural geography; management and organizational behaviour; and media and cultural studies. The analysis was restricted to the 10-year period of 1994–2003. The fact that the findings are based on a large corpus of articles suggests that the sample is unlikely to be overly atypical, although claims of representativeness would be impossible to sustain. Judgments about whether articles fell within the purview of the investigation, in terms of whether they could be regarded as deriving from the five fields, were made on the basis of the journal title or information supplied in abstracts. In this way, a total of 232 articles was generated and content analyzed. What was and was not an example of the combination of quantitative and qualitative research was occasionally problematic. The most notable of these occasions had to do with cases in which the researcher claimed to have used a qualitative approach or to be using qualitative data, but in fact the ‘qualitative data’ were based on a quantitative analysis of unstructured data – for example, of responses to open questions. Articles in which this occurred and where such data were the only source of the qualitative component were not included in the sample, because it is very debatable whether they can be regarded as indica- tive of a qualitative approach. This kind of quantification of qualitative data is more properly regarded as indicative of a quantitative research approach. Indeed, in some articles that were included in the sample, this kind of process was depicted by authors as indicative of a quantitative research approach rather than a qualitative one. There is clearly some confusion concerning whether the quantification of qualitative, unstructured data is indicative of a quantitative or a qualitative research approach. For the purposes of sample selection, it was taken to be the former, regardless of authors’ claims. However, this was not a very common occurrence; although a log was not kept of these cases, they number no more than five or six articles. Qualitative Research 6(1)100 The sample is likely to be biased in the sense that by no means all authors of articles reporting multi-strategy research foreground the fact that the findings reported derive from a combination of quantitative and qualitative research, or do not do so in terms of the key words that drove the online search strategy. An alternative search strategy is to select a sample of journals and to search for articles exhibiting multi-strategy research. This tactic was employed by Niglas (2004) in her investigation of multi-strategy research in education. Her sample of 145 articles derived from 1156 articles in 15 journals. This is a very good way of generating a sample but, in the context of a study that is meant to cover five fields of study, it is difficult to implement and also results in a lot of redundancy because a large number of articles have to be read in order to establish whether they are based on both quantitative and qualitative research (only 12.5 percent of articles read for Niglas’s study were relevant to the main focus of her investigation). Moreover, foregrounding that a study is based on multi-strategy research is interesting because it implies that the fact that the different sources of data were employed is important and significant to the author(s) concerned. Since a major focus of the research was the kinds of purposes to which multi-strategy research is put, the online search strategy that was used for the study reported here was very relevant, because we might anticipate that researchers who choose to emphasize this aspect of their studies will have given greater consideration to the issues involved in combin- ing quantitative and qualitative research. In this sense, the articles from which the findings derive constitute a purposive sample. A further issue that suggests some advantages to the sampling approach taken for this article is that it allows articles in a wide variety of journals to be uncovered. Thus, while it is certainly possible to trawl through sociology journals for instances of multi-strategy research articles in sociology, such a process risks neglecting many relevant sociology articles appearing in specialist journals. Several writers have pointed out that quantitative and qualitative research can be combined at different stages of the research process: formulation of research questions; sampling; data collection; and data analysis. Articles for this study were chosen in terms of data collection and data analysis and then content analyzed in relation to these aspects of the research process. Issues of sampling did materialize in the study, as the findings below will indicate. Data collection and analysis were emphasized because these are arguably defining features of quantitative and qualitative research. Moreover, multi-strategy research articles nearly always entail the collection and analysis of both quan- titative and qualitative data (Niglas, 2004). B A C KG RO U N D F I N D I N G S First, a small number of background features of the articles analyzed thus far will be mentioned. When the primary discipline of each article is examined, we find that the major contributing discipline is sociology with 36 percent of all articles. This is followed by social psychology (27%); management and Bryman: Integrating quantitative and qualitative research 101 organizational behaviour (23%); geography (8%); and media and cultural studies (7%). These findings strongly suggest that multi-strategy research is more commonly practised in some disciplines than others. A further interesting background characteristic is the nation of the insti- tutional affiliation of the author or first author of each article. North America is the major contributor with 49 percent of all articles; the UK comes second with 27 percent; followed by Europe and Australia (8% and 7%); Middle East (4%); with Asia, Africa and Latin America contributing 3 percent between them. These figures are obviously significantly affected by the fact that only English language publications were sought and read. R E S E A RC H M E T H O D S A N D R E S E A RC H D E S I G N S U S E D The first issue to be addressed in this article is: what research methods and research designs were employed in the articles? Each article was coded in terms of the research methods that were employed. Some of the research methods are perhaps better thought of as methods of data analysis, but they are frequently portrayed as research methods because of their distinctive approaches to sampling or capturing data (for example, content analysis, discourse analysis and conversation analysis). Table 1 presents the main methods used. The analysis presented derives from a multiple response analysis using SPSS. A striking feature is that a small number of methods account for the vast majority of all methods employed. Survey methods and qualitative interviews account for the vast majority of methods employed in the articles. If we aggregate self-administered questionnaire, structured interview and questionnaire/structured interview (a category used when it was unclear Qualitative Research 6(1)102 T A B L E 1 . Research methods employed Number of articles using Self-administered questionnaire 121 Structured interview 52 Structured observation 3 Content analysis 18 Quantification of qualitative interview questions 15 Questionnaire/structured interview 18 Semi-structured interview 159 Participant observation/ethnography 14 Unstructured interview 6 Qualitative analysis of documents 28 Answers to open questions in questionnaire 48 Focus groups 33 Language-based analysis 5 Other method 55 how survey instruments were administered), 82.4 percent of all articles coded used a survey instrument. If we aggregate semi-structured interview and unstructured interview, we find that data for 71.1 percent of articles derived from either of these two ways of conducting qualitative interviews. Further, 57.3 percent of all articles are based on a combination of a survey instrument and qualitative interviewing. In other words, one combination of research methods predominates in this data set – that is, one in which data are collected by either structured interview or questionnaire on the quantita- tive side along with either a semi-structured or unstructured interview on the qualitative side. A further feature is that with 6.5 percent of articles, the quantitative data derive from an individual qualitative or focus group interview and that, in 20.7 percent of articles, the qualitative data derive from open questions in a struc- tured interview or self-administered questionnaire. In the former case, the quantitative data derive from a research instrument associated with qualitative data collection while, in the second case, the qualitative data derive from a research instrument associated with quantitative data collection. In other words, for around 27 percent of articles, the collection of quantitative and qualitative data was not based on the administration of separate research instruments. This finding is interesting because some methodologists might argue that a combination of quantitative and qualitative data based on the administration of one research instrument does not represent a true integration of quantita- tive and qualitative research because one will tend to be subordinate to the other. Thus, when multi-strategy research derives from the administration of a semi-structured interview, some of whose questions are quantified, an argument might be levelled that this does not represent a genuine form of quantitative research since the data have not been gathered in line with its underlying principles. Similarly, it might be argued that asking a small number of open questions in the course of a structured interview does not really provide an instance of multi-strategy research because the qualitative data have been collected in the course of administering a research instrument that has been devised in terms of survey principles. Moreover, such a situation requires a modification of approach to answering questions on the part of respondents in the course of responding to a research instrument. However, articles adopting an approach in which quantitative and qualitative data derived from the same research instrument were included. When we turn to research designs, the aim of the analysis was to code articles in terms of the design employed for the quantitative data and the design employed for the qualitative data. In a small number of cases (4), because of the complexity of the data, a third research design was coded. Table 2 presents the data on this aspect of the investigation using a classifi- cation that follows Bryman’s (2004) categorization of research designs. In this classification, a study is treated as a case study if it involves just a single case. Bryman: Integrating quantitative and qualitative research 103 If it was a multiple case study, involving two or more cases, it was treated as a comparative design. Again, the analysis derived from a multiple response analysis using SPSS. As one might expect from the findings in Table 2, the bulk of the studies employed a cross-sectional design for the collection of both the quantitative and the qualitative data. Experimental and quasi-experimental designs barely figure in the findings. Employing a cross-sectional design for the collection of both quantitative and qualitative data is by far the most common design combi- nation (62.9% of all articles). When we put the data relating to research methods and research designs together, we find that 41.8 percent of all articles included both a survey instrument and personal qualitative interviewing within a cross-sectional design for the collection of both sets of data. Some- times, although rarely, this format will have been accompanied by other sources of data. J U S T I F I C AT I O N S F O R C O M B I N I N G Q UA N T I TAT I V E A N D Q UA L I TAT I V E R E S E A RC H A major focus of the content analysis was on the rationales proffered for combining quantitative and qualitative research. This aspect of the investi- gation was approached in several ways. First, the rationale given by authors for combining the two approaches to data collection and/or analysis was coded. For this exercise, the reasons that were given before the findings were presented were typically examined. Then, the ways in which quantitative and qualitative research were actually combined were coded. In doing so, the coding reflected authors’ reflections on what they felt had been gleaned from combining quan- titative and qualitative research, and any ways in which the two were combined which were not reflected in authors’ accounts. The purpose of discriminating between these two ways of thinking about the justification for multi-strategy research was that authors’ accounts of why they intended to combine quanti- tative and qualitative research might differ from how they actually combined Qualitative Research 6(1)104 T A B L E 2 . Research designs employed Number of articles using Cross-sectional design 1 169 Cross-sectional design 2 148 Case study 1 24 Case study 2 16 Longitudinal 1 28 Longitudinal 2 19 Experimental 1 (includes quasi-experimental) 9 Experimental 2 (includes quasi-experimental) 5 Comparative 1 (includes multiple case study) 30 Comparative 2 (includes multiple case study) 19 them in practice. If a difference was sometimes found between the two accounts (that is, between the rationale and practice), this would be interesting because the scientific paper is often perceived among sociologists of science as an ex post facto reconstruction that rationalizes and injects coherence into the different elements of the research process (e.g. Gilbert and Mulkay, 1984). In coding the justifications for combining quantitative and qualitative research, two different schemes were employed. First, the influential scheme devised in the context of evaluation research by Greene et al. (1989) was used. This scheme isolates five justifications for combining quantitative and quali- tative research: 1. Triangulation: convergence, corroboration, correspondence or results from different methods. In coding triangulation, the emphasis was placed on seeking corroboration between quantitative and qualitative data. 2. Complementarity: ‘seeks elaboration, enhancement, illustration, clarifi- cation of the results from one method with the results from another’ (Greene et al., 1989: 259). 3. Development: ‘seeks to use the results from one method to help develop or inform the other method, where development is broadly construed to include sampling and implementation, as well as measurement decisions’ (Greene et al., 1989: 259). 4. Initiation: ‘seeks the discovery of paradox and contradiction, new perspec- tives of [sic] frameworks, the recasting of questions or results from one method with questions or results from the other method’ (Greene et al., 1989: 259). 5. Expansion: ‘seeks to extend the breadth and range of enquiry by using different methods for different inquiry components’ (Greene et al., 1989: 259). This scheme has been quite influential and was employed by Niglas (2004) in her examination of education research articles. In their analysis of evalu- ation research articles, Greene et al. (1989) coded each article in terms of a primary and a secondary rationale, a procedure that was also employed by Niglas (2004). An advantage of the Greene et al. scheme is its parsimony, in that it boils down the possible reasons for conducting multi-strategy research to just five reasons, although the authors’ analysis revealed that initiation was uncommon. A disadvantage is that it only allows two rationales to be coded (primary and secondary). Accordingly, a more detailed but considerably less parsimonious scheme was devised. It was based on an extensive review of the kinds of reasons that are frequently given in both methodological writings and research articles for combining quantitative and qualitative research. The scheme provided for the following rationales: a) Triangulation or greater validity – refers to the traditional view that quan- titative and qualitative research might be combined to triangulate Bryman: Integrating quantitative and qualitative research 105 findings in order that they may be mutually corroborated. If the term was used as a synonym for integrating quantitative and qualitative research, it was not coded as triangulation. b) Offset – refers to the suggestion that the research methods associated with both quantitative and qualitative research have their own strengths and weaknesses so that combining them allows the researcher to offset their weaknesses to draw on the strengths of both. c) Completeness – refers to the notion that the researcher can bring together a more comprehensive account of the area of enquiry in which he or she is interested if both quantitative and qualitative research are employed. d) Process – quantitative research provides an account of structures in social life but qualitative research provides sense of process. e) Different research questions – this is the argument that quantitative and qualitative research can each answer different research questions but this item was coded only if authors explicitly stated that they were doing this. f) Explanation – one is used to help explain findings generated by the other. g) Unexpected results – refers to the suggestion that quantitative and quali- tative research can be fruitfully combined when one generates surprising results that can be understood by employing the other. h) Instrument development – refers to contexts in which qualitative research is employed to develop questionnaire and scale items – for example, so that better wording or more comprehensive closed answers can be generated. i) Sampling – refers to situations in which one approach is used to facilitate the sampling of respondents or cases. j) Credibility – refers to suggestions that employing both approaches enhances the integrity of findings. k) Context – refers to cases in which the combination is rationalized in terms of qualitative research providing contextual understanding coupled with either generalizable, externally valid findings or broad relationships among variables uncovered through a survey. l) Illustration – refers to the use of qualitative data to illustrate quantitative findings, often referred to as putting ‘meat on the bones’ of ‘dry’ quanti- tative findings. m) Utility or improving the usefulness of findings – refers to a suggestion, which is more likely to be prominent among articles with an applied focus, that combining the two approaches will be more useful to practitioners and others. n) Confirm and discover – this entails using qualitative data to generate hypotheses and using quantitative research to test them within a single project. o) Diversity of views – this includes two slightly different rationales – namely, combining researchers’ and participants’ perspectives through quantitative and qualitative research respectively, and uncovering relationships between variables through quantitative research while also Qualitative Research 6(1)106 revealing meanings among research participants through qualitative research. p) Enhancement or building upon quantitative/qualitative findings – this entails a reference to making more of or augmenting either quantitative or qualitative findings by gathering data using a qualitative or quantita- tive research approach. q) Other/unclear. r) Not stated. This classification includes a larger number of categories than other schemes and as such is meant to capture in finer detail the range of reasons that are given for conducting multi-strategy research. There are clearly symmetries between the Greene et al. scheme and the more fine-grained approach just outlined. For example, ‘development of a research instrument’ and ‘for sampling/case study selection reasons’ correspond to ‘development’, while ‘to enhance or build upon quantitative/qualitative findings’ corresponds to ‘complementarity’. Table 3 shows the distribution of articles in terms of just the primary rationale using the Greene et al. scheme (see column for ‘Rationale’). In just over a quarter of all articles, no rationale was provided. Complementarity and expansion were the most frequently cited primary rationales with 29 percent and 25 percent …