Research Methods Literature Review PLEASE SEE DOCUMENTS NEEDED ATTACHED!!! - Psychology
See the outline of the topic selected for the Final Research Proposal attached. Apply the scientific method by suggesting both a specific research question and a hypothesis for the topic in the outline.
Evaluate the chosen four peer-reviewed articles attached by summarizing each and explaining how the research design described in each article could be useful for designing original research on the topic in the outline. Compare and contrast the paradigms or world views inherent in the methodology associated with each research design. Apply professional standards and situate yourself as a researcher by identifying which of these approaches best fits with your worldview.
Basically this is a literature review of the four articles attached about research designs for each research methods. In the literature review, explain the design described in each article, and how it would be applied to the research topic in the outline attached.In those descriptions, identify which research design from a research method would best fit the topic in the outline attached and then explain why you would use that particular one for the topic. Please see the Research Designs for each Research Methods that were selected for the four articles for the literature review in the reference page attached.
Data/PresetImageFill3-27.jpg
Data/PresetImageFill1-25.jpg
Data/PresetImageFill2-26.jpg
Data/PresetImageFill4-28.jpg
Data/PresetImageFill5-29.jpg
Data/PresetImageFill0-24.jpg
Data/bullet_gbutton_gray-30.png
Index/Document.iwa
Index/ViewState.iwa
Index/CalculationEngine.iwa
Index/DocumentStylesheet.iwa
Index/AnnotationAuthorStorage.iwa
Index/DocumentMetadata.iwa
Index/Metadata.iwa
Metadata/Properties.plist
Metadata/DocumentIdentifier
06AEF591-B65C-46DB-B900-0E9608A9E04F
Metadata/BuildVersionHistory.plist
Template: Blank (11.0)
T11.1 (7356.0.96)
preview.jpg
preview-micro.jpg
preview-web.jpg
Ethography Design is one of Qualitative Research Methods
Buscatto, M. (2018). Doing ethnography: ways and reasons. In The sage handbook of qualitative data collection (pp. 327-343). SAGE Publications Ltd,https://www-doi-org.proxy-library.ashford.edu/10.4135/9781526416070
Posttest-only control group Design is one of the Quantitative experimental Research Methods
Frey, B. (2018). Posttest-Only Control Group Design. In The SAGE encyclopedia of educational research, measurement, and evaluation (Vols. 1-4). Thousand Oaks,, CA: SAGE Publications, Inc.https://dx-doi-org.proxy-library.ashford.edu/10.4135/9781506326139.n530
Archival Design is one of Non-experimental Research Method
Mills, A. & Mills, J. (2018). Archival research. In The sage handbook of qualitative business and management research methods (pp. 32-45). SAGE Publications Ltd,https://www-doi-org.proxy-library.ashford.edu/10.4135/9781526430236
Triangulation Design is one of the Mixed Methods Research
Turner, Scott & Cardinal, Laura & Burton, Richard. (2015). Research Design for Mixed Methods: A Triangulation-based Framework and Roadmap. Organizational Research Methods. 20. 10.1177/1094428115610808.
Doing Ethnography: Ways and Reasons
In: The SAGE Handbook of Qualitative Data Collection
By: Marie Buscatto
Edited by: Uwe Flick
Pub. Date: 2018
Access Date: August 16, 2021
Publishing Company: SAGE Publications Ltd
City: London
Print ISBN: 9781473952133
Online ISBN: 9781526416070
DOI: https://dx.doi.org/10.4135/9781526416070
Print pages: 327-343
© 2018 SAGE Publications Ltd All Rights Reserved.
This PDF has been generated from SAGE Research Methods. Please note that the pagination of the
online version will vary from the pagination of the print book.
https://dx.doi.org/10.4135/9781526416070
Doing Ethnography: Ways and Reasons
Marie Buscatto
Introduction
More than a century ago, anthropologists adopted ethnography as their main method (Taylor, 2002). Early
on, they defined general scientific principles guiding the study of ‘foreign’ societies (Gobo, 2008). But
ethnography has, also very early on, been adopted by other disciplines, such as sociology, opening new ways
to study contemporary societies (Madge, 1963). For instance, at the end of the nineteenth century, Frederick
W. Taylor founded his ‘scientific method’ based on his personal observation of workers (see Wästerfors,
Chapter 20, this volume). In the 1920s Elton Mayo and his colleagues founded the ‘Human Resources School’
following the ‘Hawthorne experiment, which was mainly based on observations led among workers in a
plant.1 After World War II, prominent researchers in Chicago, such as Erving Goffman, Howard S. Becker or
Donald Roy based some of their most renowned works – respectively dealing with mental illness, marijuana
smokers, or plant workers – on long-term systematic observations.
This method has now become quite legitimate in sociology. What, then, is defined as ethnography? How
may the use of ethnography enrich ones scientific results? How is an ethnographic study to be conducted
over time? What are the issues to be dealt with to avoid ethnographys potential limits? What are the new
developments? These are some of the questions to be dealt with in this chapter, based on current and past
literature and my reflexive experience as an ethnographer.
What is ethnography?
Ethnography is mainly defined as long-term observation (see Wästerfors, Chapter 20, this volume) personally
conducted in situ by the researcher and aimed at producing specific data:
In its most characteristic form it involves the ethnographer participating, overtly or covertly, in
peoples daily lives for an extended period of time, watching what happens, listening to what is said,
asking questions – in fact, collecting whatever data are available to throw light on the issues that are
the focus of the research. (Hammersley and Atkinson, 1995 [1983], p. 1)
Ethnography may take several forms – participant, semi-participant, or external. It may be conducted in
various ways – for example, following the same people through different places (see Kusenbach, Chapter 22,
this volume) or being focused on one or several specific settings. It may be implemented in foreign countries
over long periods of time (as in most anthropological works) or in the researchers own country over shorter
periods of time (as in most sociological works). It may be done overtly – those being observed know about it
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 2 of 20 Doing Ethnography: Ways and Reasons
– or covertly. Ethnography can also be used to study society, adopting quite differing approaches. While Gary
Alan Fine promotes a ‘peopled ethnography, to be distinguished from ‘postulated’ or ‘personal ethnographies’
(2003), Loïc Wacquant defends the need for ‘carnal sociology’ (2005). Whereas James Clifford and George
Marcus argue for the development of ‘postmodern ethnographies’ (1986), Howard Garfinkel is in favour of
an ‘ethnomethodological approach’ (1967). If Michael Burawoy develops the ‘extended case method’ (1998),
Barney Glaser and Anselm Strauss earlier expose the virtues of ‘grounded theory’ (1968).
In other words, even if one tends to associate ethnographic sociology with the interactionist paradigm, due to
the powerful influence of the interactionist ‘School of Chicago’ researchers – Hughes, Goffman, Becker most
notably – ethnography can also be used to develop any theoretical framework, be it Marxism, postmodern
theory, ethnomethodology, phenomenology, conversation analysis (see Jackson, Chapter 18, this volume),
pragmatism, or social fields. Those different uses are not necessarily compatible with each other since
ways to collect, to interpret, and to present data may vary strongly among those sociologists, as powerfully
discussed in Katzs (2004) controversial article. But whatever researchers’ ‘ethnographic styles’ (Wacquant,
2003) and main theoretical framework, the two central features of ethnography remain the effective presence
of the ethnographer in the world she studies and its long-term systematic observation:
‘We characterize ethnography as sustained, explicit, methodical observation and paraphrasing of
social situations in relation to their naturally occurring contexts.’ (Weick, 1985, p. 568) (…) It
excludes when the fieldworker does not remain in the field (for months or for years) to become
saturated with first-hand knowledge of the setting. (Glaser and Strauss, 1967) (in Morrill and Fine
1997, p. 425)
Ethnographic data are mainly constituted from the researchers personal in situ systematic observations
carried out over time. But they also include any other data collected in the field during the course of
observation, such as informal conversations, formal interviews (see Roulston and Choi, Chapter 15, this
volume), documents (see Rapley and Rees, Chapter 24, this volume) or objects which are part of those
observations (Gobo, 2008).
What are the main contributions of ethnography to the study of social
phenomena?
One may summarize ethnographic main contributions to the study of social phenomena around three main
ideas.
First of all, ethnography provides a privileged access to ‘invisible’ or difficult to access social phenomena.
Observation, as opposed to questionnaires or interviews, gives access to peoples practices, and not to their
oral justifications or representations (LaPiere, 1934), that is, to all those ‘natural, hidden, taboos or difficult to
express practices which people have difficulty in describing (or would not like to describe even if they were
aware of them). While interviews or questionnaires enable people to express opinions or to describe practices
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 3 of 20 Doing Ethnography: Ways and Reasons
as they remember them, direct observation gives access to practices while they are performed. This has been
key in unveiling dance musicians’ or marijuana smokers’ practices (Becker, 1963), the ways mental illness
is daily fabricated through ritual practices in mental institutions (Goffman, 1961) or plant workers’ resisting
practices (Roy, 1952; see next section).
Moreover, since ethnography is led over time, the researcher is also able to discover the ways that practices
are socially produced through action and interaction, how they are legitimated, how they get transformed
through negotiation, conflict, and confrontation between actors. One may then better grasp key social logics,
even if sometimes contradictory, which underline peoples practices through time. As stated by Michael Rosen
about organizational ethnography, ‘by definition, ethnography is a longitudinal method, geared towards a
process-based understanding of organizational life’ (Rosen, 1991, p. 12). Following this argument, specific
organizations can then be thoroughly defined as collective worlds which get produced, legitimated and
transformed through organizational members’ actions and interactions such as the specific neighbourhood
investigated by Whyte (1943). Thanks to long-term personal observations, the researcher is able to produce
a complete understanding of how a specific observed world (such as a hospital, a school, a neighbourhood
or a restaurant) or a specific group of people (such as artists, sick people, drug dealers or homeless people)
collectively produce their lives over time.
One last, but not the least of the advantages of ethnography, is to get access to multiple perspectives
and practices as developed by observed people over time. In all social activities, logics of action may vary
depending on the time of the day, the week, or the year. Actors’ rationalities may also vary depending on
several factors, which may change And since the ethnographer is to observe peoples activities in the long
run, she will be in a good position to distinguish exceptional facts from recurring activities, general rules and
norms from disrupting behaviours… and see how those recurring and exceptional events work together to
produce, legitimate and transform the observed world, group of people, or actions.
Machine workers’ resisting strategies: a historical illustration
Lets take a famous historical example presented by the Chicagoan interactionist Donald Roy (1911–1980),
which has highly influenced the development of the sociology of work. Based on the analysis of covert
participant observations made during eleven months of work as a radial-drill operator in the machine
shop of a steel-processing plant in 1944 and 1945, he showed how workers were fighting a rational and
collective war against management through what he called ‘restriction’ practices. In his 1952 article, he
describes the active strategies collectively shaped by machine workers in order to defend their economic
and personal interests against management rules and orders: his ethnographic work unveiled two specific
workers’ resisting strategies, ‘quota restriction’ and ‘goldbricking, as will be shown now.
Based on his own productive output, he first found that his behaviours were quite simple to describe: either
he was getting a $1.25 to $1.34 an hour pay, or he was getting a salary far below the 0.85 cents official basic
salary per hour. And what he called ‘this bimodal pattern of hourly earnings for the ten-month period’ (Roy,
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 4 of 20 Doing Ethnography: Ways and Reasons
1952, p. 428) appeared to be very close to those of his main fellow workers. It ‘was the rule of the shop’ and,
based on his daily observations, this rule was clearly produced by two specific resisting strategies: ‘quota
restriction’ and ‘goldbricking.
On the one hand, the concentration of the $1.25 to $1.34 level was due to ‘quota restriction’ behaviours: the
fixed quota being considered as feasible to earn at least 1 dollar, workers did produce more than expected,
but avoided producing too much above a 1.25 dollars quota, so that the set quota would remain as such (and
would not be downgraded by the Methods Department). On the other, the concentration of hourly payments
far below the normal pay of 0.85 cents was due to ‘goldbricking’ behaviours: when the quota was considered
as too demanding to earn at least 1 dollar an hour, workers preferred to ‘hold back’ and to earn much less, so
that the Methods Department would have to improve the quota. In both cases, workers then had ‘free time,
sometimes hours to spend, in order to respect both strategies and ensure the quotas remain feasible and
acceptable to them. As observed by Roy, ‘I reached the peak in quota restriction on June 27, with but three
and a half hours of productive work out of the eight’ (Roy, 1952, p. 433).
If those strategies were economically sound for those machine workers, they were also collectively
developed. As regularly detailed by Roy, workers were spending a lot of time assessing jobs collectively,
making sure that new hires did not work too fast and did not threaten their resisting strategies, and explaining
to them all the gains to be drawn from such strategies (protect feasible quotas, avoid working too hard, having
‘free time’ on the job). Workers led a collective, even if hidden, war against management and the Methods
Department.
Ethnography has been key in unveiling those practices since workers tend to hide them. Long-term
observations also enabled Roy to discover the bimodal pattern of restriction and the ways it was collectively
built-up over time. He also showed that despite the multiple jobs to be dealt with in this plant, this pattern did
not change over time and was driven by machine workers’ economic and personal interest. Roys results have
been key, right from the 1950s, in developing a better understanding of how first-level workers are able to
collectively fight to defend their working conditions – including their level of earning – using quite rational and
well-organized strategies as is now regularly demonstrated in all organizations (for example, see Buscatto,
2008).
Better grasp ‘GENDER AT WORK: a contemporary illustration
Following this first historical example, I will now show how contemporary ethnography has been quite
instrumental in better grasping gendered differences in work settings or situations. Overall, ethnography
does enrich the identification and analysis of gender in work settings or situations precisely by inducing the
researcher to focus on ‘doing gender’ (West and Zimmerman, 1987).2 It is in daily interaction that ‘sex-class
makes itself felt, here in the organization of face-to-face interaction, for here understanding about sex-based
dominance can be employed as a means of deciding who decides, who leads, and who follows’ (Goffman,
1997, p. 208). Ethnography enables the researcher to unveil how gender segregation is produced by actors
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 5 of 20 Doing Ethnography: Ways and Reasons
in the work context. Gender segregation, a complex, dynamic phenomenon operative at different points in the
social order, is actually enacted by actors in the work situation. To deepen this initial conclusion, I will now
describe the three specific contributions that ethnography can make to the study of gender in work situations
in enabling researchers to spot disparaging gendered stereotypes, occupational networks, and norms (for
detailed explanations, see Buscatto, 2010).
First of all, direct observation proves extremely effective not only in identifying existing gender stereotypes but
also to grasp how they are constructed, how they operate in work situations and help produce professional
legitimacy. For example, in my study of French jazz singers (Buscatto, 2007a, 2007b), through observing
singers at work – rehearsing, performing on stage, jamming, – and in leisure situations – drinking, partying,
going out, joking in informal situations, I progressively came to not only unveil two key feminine pejorative
stereotypes associated with seduction and lack of technicality, but also the ways those pejorative stereotypes
favoured a strong devaluation of those female singers, and their marginalization in the French jazz world. Due
to those two stereotypes, female singers tended to be considered as potential sexual objects while their voice
was not viewed as a ‘real instrument, which negatively affected their reputation as professional musicians. By
focusing my attention on daily interactions between musicians, ethnography helped identify this tie between
specific negative gender stereotypes and their effects on mens and womens professional reputations.
Second of all, ethnography is useful not only in identifying the difficulty women have in getting into and
remaining a part of ‘masculine’ social networks, but also and above all, in spotting the social ‘reasons’ for this,
as shown in my study of women jazz instrumentalists (Buscatto, 2007a). Social networks are crucial when
it comes to keeping ones place in fluid art worlds and constructing a relatively good professional reputation
in them (Becker, 1982). I was able to show that French women jazz instrumentalists over the age of 30,
whatever their musical reputation, could not make a living primarily from jazz, partly because they had such a
hard time getting into stable social networks; in this regard they were extremely dependent on their male jazz
musician (programmer, agent, or producer) husbands or partners (when they had one). It was by comparing
my observations with statements from these men and women, collected in interviews and informal situations,
that I realized that, except for friendly moments spent with their husbands’ or partners’ musician friends, these
women instrumentalists were not likely to have any prolonged friendships with male jazz musicians. One
reason appeared to be that, when playing music and sharing moments outside of musical work, jazzmen felt
more comfortable with other jazzmen. Masculine socializations had prepared them to share more topics of
interest, to behave in more similar ways on a daily basis. A second reason explaining female marginalization
was their high seductive power – as for female singers – which made them difficult to envision as regular
colleagues. Interviews did enable us to unveil this phenomenon: when musicians were asked to describe their
professional connections (groups and invitations), female musicians were not often mentioned by their male
counterparts as regular colleagues and female musicians had fewer opportunities to work on a regular basis
as compared to their male colleagues. But observations were key in analysing why female musicians tended
not to be treated as regular colleagues. Male-only interactions appeared much more relaxed, ‘natural’ and
easy to handle for male musicians than mixed interactions, both in work situations – including dealing with
conflicts and fights over music, – and in leisure situations – when partying, drinking, travelling. Male norms
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 6 of 20 Doing Ethnography: Ways and Reasons
made it easier for men to work with other men in this jazz world, partly explaining female instrumentalists’
marginalization over the age of 30.
Third, women may become ‘fed up’ with working in male milieus that they consider unlikely to include them
professionally or where they find it difficult to gain recognition for their professional abilities. This difficulty
is explained in part by the maleness of work norms that make it harder for women to participate than men.
Here again, ethnography helps in identifying these gender norms and their possible effect on the construction
of occupational work positions, as shown in my study about unionists (Buscatto, 2009). The very way that
union activity operated was strongly marked by norms that are socially constructed as ‘masculine’ – topics of
conversations, valued public behaviours, flexibility outside of work. All this made it difficult for women to gain
a foothold in union worlds. As this example clearly showed, it was by comparing data collected in interviews
– concerning areas of interest, modes of acceding to union positions, ideas about union activity – with data
collected through observation – union member gatherings, social behaviour, the mutual inviting that union
members do – that I could spot which social norms were valued and/or activated by sex and what their
gender-related effects were on unionists’ trajectories. Unionism was a ‘male’ activity, and as such it was less
likely to attract women than men. When it did attract them ‘anyway, it afforded them a position that they were
then hard-pressed to construct in a ‘masculine’ way.
Reflexivity as a scientific model
But then, how does one conduct an ethnographic study scientifically? The anthropologists who invented field
studies in the late nineteenth century, most notably Bronislaw Malinowski and Edward E. Evans Pritchard,
defined their way of proceeding to be scientific (Gobo, 2008). The stated aim was to collect ethnographic data
in an objective way. But anthropology was destabilizing the positivist scientific model and ethnography soon
appeared as incomplete, lacking and imperfect (Buscatto, 2016).3 It thus could not be practised with a blind
respect for the positivist method based on the presumed superiority of the hypothetico-deductive model and
the understanding that it would ensure objectivity for the collected ‘data.
In the positivist method, researchers construct hypotheses, indicators and questions in order to ensure that
they will not be caught up in, or tripped up by, preconceptions, which may conflict with the objectivity of the
experimental method. But very soon, ethnographic researchers could not be considered as being protected
from their theoretical or personal biases. They were potentially considered as systematically falling victim to
them. And being integrated into the community they were studying offered no external guarantee that they
would be able to break free of their ideologies, assumptions, or preconceptions. It was considered, very early
on, that researchers’ past experience, personality, and cultural orientations necessarily influence what they
see and how they interpret it. Realizing this, anthropologists were quick to construct a reflexive scientific
approach.
Reflexivity implies reconciling the opposites of involvement and detachment (Elias, 1956; Hughes, 1971).
The intention of constant and rigorous adherence to this principle is to provide a solid basis for research
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 7 of 20 Doing Ethnography: Ways and Reasons
without eroding its primary quality: the researchers ability to adapt to the experience and realities of the
people she is observing. Though involvement does imply constant flexibility and adaptability on the part
of ethnographic researchers, they must still be vigilant about maintaining distance, remaining detached.
Involvement is a construction that the researcher needs to analyse and negotiate with actors in connection
with the rules structuring the social system in which they operate. Detachment is a continuous activity and
guides the researchers choices in all phases of his involvement: determining how to behave, what strategies
to implement, the limits to impose on her action and analysing collected observations in relation to the
roles played. But in her involvement, the researcher may go through phases of intense identification. The
processes underlying that kind of identification then have to be analysed, deciphered, and contextualized
so as to apprehend, if only after the fact, the workings of the action structuring the negotiated relationship
between the researcher and the observed actors and to assess their implications for the study. These last
activities are what is meant by detachment. Detachment thus requires the researcher to describe and critically
question her way of viewing the world and the social conditions in which ‘data’ are produced in the research
act (for developments, see Gobo, Chapter 5, this volume).
How does one start an ethnographic research?
Following reflexivity as a scientific model, ethnography is supposedly an inductive method, which implies
that one is to start observations as openly as possible in order to let analyses emerge from observations
(as opposed to a deductive method which entices the researcher to start by developing strong theoretical-
driven hypotheses to be tested empirically (for a detailed discussion of induction, deduction, abduction, see
Kennedy and Thornberg, Chapter 4, this volume). As nicely stated by Atkinson ‘we create the fields we
study, in collaboration with hosts and informants, through the research acts we enact’ (Atkinson, 2015, p. 26).
However, as also asserted by Atkinson, even if ethnography does suggest that the observer is to let the field
‘speak’ through time, she is also to guide those observations whenever necessary. Indeed, sheer induction
never happens since researchers, as educated human beings, do hold preconceived ideas about the world.
So, one is to follow two main principles when entering his or her ‘field, in order not to get trapped in such a
sheer inductive illusion.
Before starting his field, the ethnographer is to wonder what to observe and how. Since she is far from being
a neutral and fully open-minded human being, what is best here is to build up and define specific ‘theoretical
orientations’ and ‘target-question’ in order to avoid, as much as possible, being influenced by preconceptions
without being aware of them. Whether he is aware of them or not, ‘theoretical orientations’ shape the way
the ethnographer defines his discipline scope, society as a whole, the ways individuals are shaped and how
they participate in shaping society collectively. Ethnography being an inductive method which implies that the
ethnographer is guided by what she observes in order to progressively fabricate sociological results, she does
start conducting her observations as openly as possible, so that theoretical orientations are not to be confused
with a specific school of thought. But theoretical orientations do participate in shaping how the ethnographer
starts observing and it is important that she is aware of them so as to be able, if needed, to transform
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 8 of 20 Doing Ethnography: Ways and Reasons
them over time. They also enable the ethnographer to define a ‘target-question’ being as adequate as it can
be, right from the beginning. Indeed, the ‘target-question’ is directly linked to the ‘theoretical orientations,
while specifying the topic to be investigated over time, even in quite general terms (the ‘target-question’
is to be enriched, transformed, confirmed throughout the research). Thanks to this ‘target-question’ – for
instance social mobility, illness trajectories, organizational socialization, gender at work, or urban dynamics
– the ethnographer will be able to decide what is to be observed and how. The ethnographer can then more
easily define where to start observations and how to lead them over time. It will guide the ethnographer
in deciding where to observe, how and in which conditions, and can also be transformed over time based
on a first analysis of initial observations (for instance if the target-question appears too narrow-minded or
misconstructed or biased, based on a first analysis of data as detailed below).
If right from the beginning the observer is as open, fluid and available as possible and tries to be as close to
and familiar with the people she observes, ‘theoretical orientations’ and ‘target-questions’ do enable her to be
as efficient as she can be in developing rigorous sociological findings. Glaser and Strauss, while defending
in their 1968 seminal book, ethnography as an inductive method – the ‘grounded theory’ – did define such
guiding principles when illustrating their use of ethnography empirically:
The general problem to be investigated was whether different ideologies did exist among the various
psychiatric professionals. […] Field work was begun first at the private hospital and directed initially
only by the frameworks of ideas known as the ‘sociology of work’ and ‘symbolic interactionism.
(Glaser and Strauss, 1968, p. 157)
In this example, they not only identify clear ‘theoretical orientations’ – ‘sociology of work’ and ‘symbolic
interactionism’ – but also a well-defined ‘target-question’ – ‘whether different ideologies did exist among the
various psychiatric professionals. Those choices led them to define one specific field – a psychiatric hospital,
people to be observed, that is, all the people involved in treating psychiatric patients – and a long-term
observation strategy in order to identify ways psychiatric professionals were collectively defining psychiatric
illnesses and negotiating their roles in treating them through time.
As clearly stated by …
Archival Research
In: The SAGE Handbook of Qualitative Business and
Management Research Methods: Methods and Challenges
By: Albert J. Mills & Jean Helms Mills
Pub. Date: 2019
Access Date: August 16, 2021
Publishing Company: SAGE Publications Ltd
City: 55 City Road
Print ISBN: 9781526429278
Online ISBN: 9781526430236
DOI: https://dx.doi.org/10.4135/9781526430236
Print pages: 32-45
© 2018 SAGE Publications Ltd All Rights Reserved.
This PDF has been generated from SAGE Research Methods. Please note that the pagination of the
online version will vary from the pagination of the print book.
https://dx.doi.org/10.4135/9781526430236
Archival Research
Albert J. Mills Jean Helms Mills
Introduction
At its simplest, archival research can be said to involve the study of collections of documents (e.g.
newsletters, annual reports, memoranda) and artifacts (e.g. uniforms, photographs) to gain an understanding
of a selected organization (e.g. British Airways), leader (e.g. Sir Colin Marshall), or professional group (e.g.
flight attendants). However, this ignores the fact that, like other research methods, archival research is
informed by underlying research philosophies. In this chapter, we focus on three major research philosophies
– modernist, postmodernist, and amodernist – and how they inform and shape archival research. Where
possible we draw on all aspects of our own research because over the years we have moved from an initial
modernist approach, to a postmodernist approach and, quite recently, an amodernist approach.1
To begin with our three major approaches: in brief, the modernist approach views the past as having an
ontological basis in fact, whose factual traces can be found through – among other means – a methodical
search through archival documents. The resulting narrative (or storied account) is an attempt to provide a
history of what really or actually happened in the past (Elton, 1967). The postmodernist approach2 views the
past as ontologically unavailable. People and events in the past were real at the time but we can only re-
present the past through narratives that rely on the dominant understanding of those people and events at
the time but also in the present. In other words, there is a disconnect between the past and history (Munslow,
2010): the past is what happened before now and history is the attempt to reproduce the past through
narrative (a story of events) or chronicles (a description of events). Finally, the amodernist approach focuses
on how ‘knowledge of the past’ is socially constructed through a series of human (e.g. historians) and non-
human (e.g. archives) actors to create a sense of history (Durepos & Mills, 2012; see also Chapter 26 of this
volume).
For respective modernist, postmodernist, and amodernist accounts, see Elton (1969, 1991), Jenkins (1991),
and Durepos and Mills (this volume). For useful overviews of the field see Bryman et al. (2011), Durepos
(2015), Jenkins (1995), and Rowlinson et al. (2014).
• Before doing archival research, you should become familiar with some of the key ontological and
epistemological debates surrounding the understanding of history and the past.
What do we mean by archival research?
Before you consider archival research, you need to have an understanding not only of what you are looking
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 2 of 21 Archival Research
for but why and how you are looking for it. You need to know something about recent debates in management
and organizational history about the nature of the past and history. A good place to start is Booth and
Rowlinsons (2006) discussion of the ‘historic’ turn in management and organizational history. This is useful
not only for understanding debates around what constitutes the past and history but also in discussing the
range of methods and methodological approaches associated with different approaches. See also the debate
between Toms and Wilson (2010) and Taylor et al. (2009).
Different approaches to history and the past shape not only what to look for ‘in the’ archives but also how
to study archival materials and where to look for archives. For example, modernist historians tend to view
archives as ‘an empirical data corpus’ of artifacts and documents that are usually associated with a physical
location (McHoul & Grace, 2003, p. 30). These locations are often referred to as an archive where the
purpose is to collect together a number of artifacts around a specific topic. It is a place where elements,
or traces, of the past can be studied through a search of documents (e.g. in-house newsletters) and other
artifacts (e.g. airline uniforms). Postmodernists, on the other hand, are not simply looking for documents and
other artifacts from the past but also the complex system of practices that underlie the way of collecting
and ordering materials in an archive. Arguably, these practices ‘govern ones manner of perceiving, judging,
imagining and acting’ (Flynn, 1996, p. 30) in ways that constitute an ‘unconscious structuring of discourse’
(May, 2006, pp. 38–9). This can influence what you come to view as (legitimate) knowledge and what counts
as evidence of that knowledge. The amodernist focus is on gaining an understanding of the relationships
between human (e.g. flight attendants), non-human (e.g. uniforms), and non-corporeal (e.g. airline stories)
actants in the creation of a particular phenomenon (e.g. the association of females with flight attending
and males as pilots).3 The essential idea here is to understand the production of knowledge (for example,
discriminatory notions of womens roles at work) by tracing the relationships that come to constitute and
maintain that ‘knowledge.
These three approaches generate different views of what constitutes an archive. From a modernist
perspective, it can be argued that:
Archival records are an invaluable tool of data gathering … that is focused on the past and its impact
on the present. The careful analysis of archival records can provide valuable information of the life,
concerns, and aspirations of individuals and groups, as well as on the activity, structure, mission,
and goals of associations, organizations, and institutions. (Stan, 2010, p. 29, emphasis added)
This is a clear modernist account of the value of archival research. The postmodernist will contend that
this type of account lures us into thinking that archival records, in themselves, embody some sort of fixed
elements of empirical truth that can be uncovered through methodical techniques of collection and analysis
of information-bearing data.
While modernists tend to talk about archives as physical locations that house a particular collection of
documents and artifacts (e.g. the Pan American World Airways Collection at the Otto Richter Library of the
University of Miami), postmodernists view ‘the archive’ as something broader. Thus, for example, Rostis
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 3 of 21 Archival Research
(2015, p. 33), in his Foucauldian study of the discourse of humanitarianism, notes that:
The archive in its broadest sense is everywhere and not restricted to the four walls of an archival
office of any one organization. Therefore, traces of constructs such as madness, discipline, or
sexuality should be found in multiple places; Foucaults genealogies draw upon this feature and
sweep quite broadly across sources, historical periods, and people.
In contrast, the amodernist views the archive – be it a specific building or broader social phenomena – as a
potential nexus for the production of knowledge, a site of networked relations to be explored and uncovered.
This provides us with at least three forms of archive. The first is a formally constituted collection of documents
and other artifacts at and as an ‘archive. The second refers to a number of sites, including physically
designated archives, which are thought to constitute the traces of a pattern of ideas and practices that
influence our understandings of the past. The third refers to a site of outcomes of networked relationships.
These various sites are thought to constitute a complex set of underlying rules or an archive. Following Mills
and Helms Mills (2011), we will use the term archive to refer to the modernist notion and archive to refer to
the postmodernist notion: either can be the starting point of a site of networked outcomes for the amodernist
depending on where they focus their attention.
Despite the fundamental differences in approach you will still find modernist, postmodernist, and amodernist
researchers in formally designated archives undertaking analyses of documents and other artifacts. Rostis
(2015), for example, included in his archival research, analysis of documents at the Museum of the
International Red Cross in Geneva.
For simplicity sake, the focus of the rest of this chapter will be on formally constituted archives, defined as a
collection of materials that serve to provide traces of the past of a particular organization (e.g. Air Canada4),
organizational leader and other members (e.g. Juan Trippe, the former CEO of Pan Am, and female members
of the same airline), or social phenomenon (e.g. Aviation and Space). This will allow us to focus on the value
of documents, which is an important focus of various forms of archival research.
Why do archival research?
The question of the value of archival research is often tied up with questions about the value of history and
the past. If history and the past are important for business studies then, the argument goes, archives are an
important source for tracing what happened before now.
Modernist
First, let us briefly review the argument for the importance of history and the past from the modernist
perspective. Professor Alfred Kieser (1994), who has played an important role in encouraging an ‘historic turn’
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 4 of 21 Archival Research
in Management and Organization Studies (MOS), makes four key points for the value of history and the past:
1. ‘In order to understand contemporary institutions it is important to know something of their
historical development’ (p. 609).
2. Historical analysis can reduce the ideological biases embedded in current ‘fashionable trends’
in MOS (p. 610).
3. We need to ‘interpret existing organizational structures not as determined by [objective] laws
but as a result of decisions in past choice[s] … whether intentional or implicit’ (p. 611).
4. ‘By confronting theories of organizational change with historical development, these theories
can be subjected to a more radical test than they have to pass when merely being confronted
with data on short-run changes’ (p. 612).
Each of these points relies in large part on the availability of archives or accounts based on archival research.
British Airways, for example, has been in operation, in one form or another, since 1919.5 While it is possible
to conduct interviews with current and former employees of the airline, this approach would be limited to the
number of people you could interview in a certain time span; the availability of people prepared to talk to you;
the span of time that their experience would cover (most of those who worked for the airline before World War
II would now be dead); and the memories of those interviewed. At the very least you would need to access
company documents and other artifacts to verify and expand on what interviewees tell you. Existing published
histories (e.g. Penrose, 1980) could also help to provide other insights into the company. However, the various
documents to be found in an archive, including associated airline histories, were written for a specific purpose
that may not address or reveal other purposes. For example, feminist studies of the airline industry (Barry,
2007; Yano, 2011) are more likely than not to be confronted by archival materials (and histories) that largely
focus on men and machines.
Modernist accounts of management and organizational history include Urwick (1938), George (1968), Kieser
(1994), Wren (2005), and Bowden and Lamond (2015).
Postmodernist
Postmodernists see value in archives for various reasons other than proof of any particular truth. Their interest
is in the role that archives, as ‘complex systems of rules, play in the construction of ‘knowledge. There are
at least two levels to this. The first one involves analysis of the materials themselves to reveal how corporate
thinking and practices helped to socially construct particular discourses. For example, in our study of British
Airways, we examined the airlines hiring and imaging of women to gain a sense of how discriminatory
practices develop, are maintained and change over time (Mills, 2006). To do this we used the framework of
the juncture (or ‘a concurrence of events in time in which a series of images, impressions, and experiences
come together, giving the appearance of a coherent whole that influences how an organization is understood’
Mills, 2010, p. 509). This helped us to identity changing dominant corporate attitudes to the role of women
over time and factors associated with the development, maintenance and change in selected practices.6 The
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 5 of 21 Archival Research
second level is to examine how the archive itself serves to shape our understanding of the past. In a plea for ‘a
fuller reflection of the uses and limitations of the archive in business historical research, Schwarzkopf (2012,
p. 1) cites Foucaults contention that ‘archives are not just sites for the passive collection of records from
the past. Rather, archives are very active in both enabling and limiting what we see, know, understand, and
accept as real’ (p. 9). To return to our study of British Airways, we found that much of what was collected and
saved focused primarily on activities associated with men, such as piloting, engineering, and management
at all levels. We note that the association between men and particular activities not only reflected dominant
practices in the airline but also served to reinforce those associations. As our research went on to argue,
a powerful discourse of gender relations played an important role in the development of British Airways,
determining who was hired and to what position; who was privileged and who was marginalized in the stories
told (e.g. detailed accounts of male airline pilots and managers) and the stories not told (e.g. the role of female
pilots during World War II). For a considerable period of time, a large number of corporate stories about
female employees centered on sexuality and the female flight attendant as the leading, albeit marginalized,
image of women in the airline industry (Helms Mills & Mills, 2008; Mills, 2006). Similarly, a discourse of
colonialism is evident for much of British Airways’ first forty or more years of operation. Here the archive
neglects the impact of the airline on non-white members of the British Empire, both in terms of the airlines
restricted hiring practices of non-white Africans and their imaging as irrational, menial labour, and basically
savage (Mills, 1995).
Postmodernist accounts of management and organizational history include Jacques (1996), Rowlinson
(2004), and McKinlay (2013).
Amodernist
Amodernist social scientists have come lately to archival research7 and are particularly interested in how
histories are produced as an outcome of a series of actor-networked relations. Here the archive is of interest
as a punctuated actor, i.e. something that is spoken of as if it is a single, albeit powerful, entity that offers
a potential treasure trove of facts about a particular person, company, event, or period. Here the focus is
not so much on developing histories out of available archival materials but rather analysis of how specific
histories come to be developed. Myrick et al. (2013), for example, analyzed archival materials of the Academy
of Management (AoM). The analysis was not to develop a new history of that association but rather to make
sense of how the existing history of the association (see http://aom.org/About-AOM/History.aspx; Wrege,
1986) came into being, and what the implications are. We were particularly interested to see what coherence
of actors worked together to create the idea of an association that has been continuously in operation since
1936. While this point of longevity lent credence to the authority and legitimacy of the AoM, it papered over
contrary details, such as the fact that the supposedly first two meetings (1936, 1937) were dinner meetings
not conferences; that the name Academy of Management was retrospectively applied to the association at a
later date; and that between 1941 and 1947 the association was basically defunct. Yet the 2011 conference
was heralded as the 75th year of the AoM – dating the association back to 1936, while the 2015 conference
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 6 of 21 Archival Research
http://aom.org/About-AOM/History.aspx
was deemed the 75th conference – contradictorily dating the conferences back to 1941.
Amodernist accounts of management and organizational history include Bruce (2006), Bruce and Nyland
(2011), Coller et al. (2015), Hartt et al. (2014), Myrick et al. (2013), and Shenhav and Weitz (2000).
• Archival research is useful for understanding relationships between the past and history in
understanding management and organizations over time. Those insights can be in the form of
establishing factual details of what really happened and their impact on current organizational reality
(modernist); revealing powerful discourses and their influence on extant knowledge (postmodernist);
and gaining insights into how history is produced as knowledge (amodernist).
Thoroughly Modern Mission
In this and the next section we will explore the methods and challenges faced by, respectively, modernists
and then postmodernists and amodernists.
It would be doing a gross injustice to modernist researchers to suggest that they are unaware of the multitude
of problems surrounding the gathering and analysis of archival material. Indeed, much of the discussion of
archival research from a modernist perspective involves questions about focus, access, and the evaluation;
collection and storage; evidentiary value and triangulation. Lets look at each in turn.
Focus
You may want to study some aspect of the history of a particular person, organization, phenomenon, etc. A
couple of good places to start are lists of archival sources and histories of your chosen topic.
Archival source lists:
There are numerous online sources to available archives and their accessibility these days. A search by
archives and then within the archive may provide useful clues to what you are looking for. For example, when,
in an earlier modernist phase, we were interested in studying how organizational cultures develop, maintain,
and change discriminatory practices over time, we looked around for a business that (a) had an existing and
well-developed archive – to allow us to track different cultural artifacts; (b) had been in existence for at least
50 years – to allow us to study changes over relatively long time periods; (c) was internationally well-known
– to provide us with an example that was of interest to a large number of potential readers/ researchers, as
well as being socio-economically powerful to have an influence on social thinking; (d) where possible, had
a powerful socio-cultural association with gendered practices; and (e) had a number of published histories
on the company. A search through a guide to British archives (Foster & Sheppard, 1989) led us to British
Airways, which proved to be a treasure trove of documentation, artifact collections, documentary films, and
photographs. The Archives and Museum collection included in-house newsletters8 (useful for tracking people
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 7 of 21 Archival Research
and events over time), annual reports9 (providing clues to leading members of the airline and their strategic
thinking), corporate memoranda10 (interesting for insights into factors that were of particular interest to the
company), advertisements11 (showing how the company imaged itself and its employees), and corporate
films and a vast photographic library12 (for gaining insights into who is privileged and who is marginalized by
corporate leaders and editors).
Histories:
The existence of published histories can help the researcher to gain insights into the chosen organization,
people, and or/phenomena but can also provide extensive information into the available sources of
information to be found. Our research into histories of Pan American Airways not only led us to the Pan
American World Airways Collection at the University of Miami but also to the Smithsonian National Air and
Space Museum as well as the British Airways Archives and Museum, which contain a number of documents
on relationships between Pan Am and British Airways. We would note that while this collection of ‘information’
is similar to a modernist approach, the ultimate difference lies in what material is being collected (e.g.
evidence of dominant discourses versus fact) and how it is analyzed. Our framework at the time drew on
postmodernism.
These sources will not only help you to decide what it is possible to focus on but in all likelihood, they will
shape the direction you go in. Esterberg (2002, p.130) suggests that such histories and debates around the
subject can help you ‘to conceptualize your study and figure out what the main controversies are. On the
other hand, existing histories may appear to limit what you can study. British Airways histories, for example,
say little or nothing on gender and colonial relations. That may be an indication of what is potentially missing
from the archives and/or the time needed to be spent searching the archives. This will shape the focus on
your study and the types of documents you will need to spend more time on. In our study of British Airways,
we had to spend a considerable time looking for accounts of female members of staff and the role they played
in the airline. The problem is exacerbated where there are no available archival sources or they are spread
out across a number of places.
• Consult specific companies and organizational members, archival source lists, and existing histories
to assess your ability to study the topic in hand.
• Begin your archival research by reading ‘what other historians and social scientists have said about
the topic’ (Esterberg, 2002, p. 130).
Access
These sources are only the beginning of the process. Access is a crucial issue. You may know where to
look, but you need to check what is accessible and what isnt. Not all archives are open to the public, and
where they are, some of the materials may not be accessible (Berg, 1989; Lipartito, 2014). An example of
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 8 of 21 Archival Research
the former is where we attempted to gain access to the archives of United Airlines (UA). Our study of Air
Canada indicated that UA had played a role in training Air Canadas flight attendants in 1937. In our request
to visit the UA archives, we were referred to the online part of the archive, which contained very limited
(and carefully crafted) material, and to a recent corporate history (Garvey & Fisher, 2002), which gave no
clues to the material we were seeking. An example of the latter problem is detailed by Rostis (2015) who
was attempting to study the role of the Biafran War (1967–70) in the development and role of humanitarian
agencies in Africa, only to find that the International Red Cross restricted access to those files. Rostis resorted
to newspaper reports of the time, but we have been unable as yet to gain information on the early role of UA
in the development of Air Canada.
• Check access to selected archives and the availability of related histories in advance of developing
a specific focus.
Collection and Storage
Having found your way to an archive, there are a number of considerations to take into account that affect
what you find and how you make sense of it. Berg (1989) distinguishes between those archives (usually
public) where the documents have been developed for ‘the express purpose of the examination of others’
(p. 86) and those (private) archives that were not developed for public engagement. While the latter may
take on a range of peculiar forms according to the needs of the person(s) establishing them, the former have
usually been ordered in a particular way to assist researchers to find certain materials. This may be quite
systematic as in the case of the National Archives of Canada or they may be placed in a rough order such
as the Pan American collection, where materials are boxed together in broad categories (e.g. ‘interviews and
biographies, ‘China, etc.). Either way, what is collected and how it is ordered can greatly affect the material
that you manage to find and what sense you make of it. Webb et al. (1984) contend that there are at least ‘two
major sources of bias in archival records – how they are collected and how they survive’ (p. 114). According
to Stan (2010, p. 29) researchers need to ‘be aware that not all archives were systematically collected [and
may not] include many relevant documents, either because these documents were destroyed intentionally or
accidentally or because they were scattered among other archives collections. The Pan Am collection, for
example, for many years consisted of just over 600 boxes of material, but this changed dramatically on, what
was to be, our last day of research at the collection when another 900 boxes were added to the collection. The
additional boxes, we were informed, were culled from tens of thousands of boxes discovered in a warehouse
in New York.13 Addressing the issue of what does and does not get collected, Schwarzkopf (2012) argues
that ‘archives reinforce survivorship bias [with large] and dominant companies often [having] better archives’
than smaller companies (p. 7). He also goes on to argue that there is a European bias in the establishment
of archives ‘because it is only within the global north and the west that this institutional support is so widely
available that local and national authorities and firms themselves can afford to preserve company records’
(p. 5). In summary, Schwarzkopf (2012) argues that ‘archives are not just sites for the passive collection
of records from the past. Rather, archives are very active in both enabling and limiting what we see, know,
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 9 of 21 Archival Research
understand, and accept as real’ (p. 9). He goes on not only to warn researchers about the in-built biases of
archives but also for ‘an open theoretical debate about the epistemological status of the archive in business
history’ (p. 2).
• Researchers need to be aware of the influence of archives through the establishment, selection, and
ordering processes involved.
• A process of reflection of the influence of the archive should be encouraged.
Evaluation
Scott (1990) argues that four criteria should be used to evaluate documents – authenticity, credibility,
representativeness, and meaning (cited in Myers, 2009, p. 159).
Authenticity refers to whether the document is what it claims to be. The most high profile example is a
document called The Protocols of the Elders of Zion, published in the early part of the twentieth century,
which was claimed to be a Jewish plan for world domination. The document was eventually proven to be a
forgery but not before carmaker Henry Ford published half a million copies in 1920 in an attempt to spread
anti-Semitism.
Credibility refers to the extent to which the author of a document is believable in what they are claiming.
For example, in 1950, United States Senator Joseph McCarthy constantly referred to documents in his
possession that proved that there were over 200 known communists working in the State Department. He was
later to claim that there were communists in the U.S. military. None of these claims was ever authenticated
and McCarthy became known in some quarters as self-serving and prone to exaggeration (Caute, 1979).
Representativeness refers to the extent to which any single document can be taken …
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/284790946
Research Design for Mixed Methods: A Triangulation-based Framework and
Roadmap
Article in Organizational Research Methods · November 2015
DOI: 10.1177/1094428115610808
CITATIONS
98
READS
13,587
3 authors, including:
Some of the authors of this publication are also working on these related projects:
Justification shift and uncertainty: Why are low-probability near misses underrated against organizational routines? View project
Scott F. Turner
University of South Carolina
14 PUBLICATIONS 659 CITATIONS
SEE PROFILE
Richard M. Burton
Duke University
175 PUBLICATIONS 3,261 CITATIONS
SEE PROFILE
All content following this page was uploaded by Scott F. Turner on 21 December 2015.
The user has requested enhancement of the downloaded file.
https://www.researchgate.net/publication/284790946_Research_Design_for_Mixed_Methods_A_Triangulation-based_Framework_and_Roadmap?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_2&_esc=publicationCoverPdf
https://www.researchgate.net/publication/284790946_Research_Design_for_Mixed_Methods_A_Triangulation-based_Framework_and_Roadmap?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_3&_esc=publicationCoverPdf
https://www.researchgate.net/project/Justification-shift-and-uncertainty-Why-are-low-probability-near-misses-underrated-against-organizational-routines?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_9&_esc=publicationCoverPdf
https://www.researchgate.net/?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_1&_esc=publicationCoverPdf
https://www.researchgate.net/profile/Scott-Turner-8?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_4&_esc=publicationCoverPdf
https://www.researchgate.net/profile/Scott-Turner-8?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_5&_esc=publicationCoverPdf
https://www.researchgate.net/institution/University-of-South-Carolina?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_6&_esc=publicationCoverPdf
https://www.researchgate.net/profile/Scott-Turner-8?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_7&_esc=publicationCoverPdf
https://www.researchgate.net/profile/Richard-Burton-12?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_4&_esc=publicationCoverPdf
https://www.researchgate.net/profile/Richard-Burton-12?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_5&_esc=publicationCoverPdf
https://www.researchgate.net/institution/Duke-University?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_6&_esc=publicationCoverPdf
https://www.researchgate.net/profile/Richard-Burton-12?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_7&_esc=publicationCoverPdf
https://www.researchgate.net/profile/Scott-Turner-8?enrichId=rgreq-0e2b20f47b062aa77d04532f0219ec89-XXX&enrichSource=Y292ZXJQYWdlOzI4NDc5MDk0NjtBUzozMDg4OTcyOTI3MTgwODBAMTQ1MDY1ODI2MTE0Ng\%3D\%3D&el=1_x_10&_esc=publicationCoverPdf
Research Design for Mixed Methods: A Triangulation-Based Framework and Roadmap
Scott F. Turner
Darla Moore School of Business
University of South Carolina
Columbia SC 29208
803-777-5973
[email protected]
Laura B. Cardinal
C. T. Bauer College of Business
University of Houston
Houston, TX 77204
713-743-2559
[email protected]
Richard M. Burton
Fuqua School of Business
Duke University
Durham, NC 27708
919-660-7700
[email protected]
Organizational Research Methods
November 25 2015
Published online before print
Research Design for Mixed Methods: A Triangulation-Based Framework and Roadmap
Abstract
All methods individually are flawed, but these limitations can be mitigated through mixed methods
research, which combines methodologies to provide better answers to our research questions. In
this study, we develop a research design framework for mixed methods work that is based on the
principles of triangulation. Core elements for the research design framework include theoretical
purpose, i.e., theory development and/or theory testing; and methodological purpose, i.e.,
prioritizing generalizability, precision in control and measurement, and authenticity of context.
From this foundation, we consider how the multiple methodologies are linked together to
accomplish the theoretical purpose, focusing on three types of linking processes: convergent
triangulation, holistic triangulation, and convergent and holistic triangulation. We then consider
the implications of these linking processes for the theory at hand, taking into account the following
theoretical attributes: generality/specificity, simplicity/complexity, and accuracy/inaccuracy.
Based on this research design framework, we develop a roadmap that can serve as a design guide
for organizational scholars conducting mixed methods research studies.
Keywords
research methods, mixed methods, triangulation, validity, research strategies, theory
development, theory testing
Acknowledgements
The authors wish to thank Associate Editor David Ketchen and three anonymous reviewers for
their detailed and thoughtful feedback. Portions of this research were presented at the 75th Annual
Meeting of the Academy of Management and the 35th Annual International Conference of the
Strategic Management Society.
1
Research Design for Mixed Methods: A Triangulation-Based Framework and Roadmap
As a basic concept in the social sciences, triangulation refers to using multiple, different
approaches to generate better understanding of a given theory or phenomenon (Burton & Obel,
2011; Singleton & Straits, 1999). While early work in the social sciences focused on the role of
triangulation in measurement, i.e., using different methods to assess construct validity (Campbell
& Fiske, 1959), more recent attention has emphasized how triangulation can contribute to our
understanding more broadly. Mixed methods research is based on the idea of heightened
understanding through methodological triangulation (Denzin, 1970, 2012; Molina-Azorin, 2007;
Torrance, 2012). For this form of triangulation, McGrath (1982) describes the use of multiple
methodological strategies, which represent generic classes of research strategies for gaining
knowledge about a research question; examples include laboratory experiments and field studies.
McGrath (1982) discusses the importance of methodological triangulation from the view that
individually “all research strategies and methods are seriously flawed” (p. 70), but in
combination, the use of multiple methods that do not share the same failings can enhance what is
known about a given research question. In similar fashion, Weick (1969) states, “we typically
need multiple methods, or techniques which are imperfect in different ways. When multiple
methods are applied, the imperfections in each method tend to cancel one another” (p. 21).
Although triangulation can take place within a given methodology, such as using different
subject populations for a lab experiment (e.g., Bateman & Zeithaml, 1989) or using different
platforms for computer simulation (Axtell, Axelrod, Epstein, & Cohen, 1996), mixed methods
research focuses on triangulation that spans multiple methodologies (Denzin, 1970, 2012;
Singleton & Straits, 1999). Jick (1979) argues that methods-spanning forms of triangulation are
on the complex end of a continuum of triangulation design while within-method forms are on the
simple end. For triangulation that spans methodologies, much attention has been directed to
convergent triangulation, which is based on the idea that knowledge develops by obtaining
convergence in substantive findings across a diverse set of methodologies (McGrath, Martin, &
Kulka, 1982). This reflects the view that using multiple methods produces more valid results as
the strengths of one method can offset the limitations of another method (Jick, 1979; Scandura &
Williams, 2000).
While scholarly advice has been offered for many forms of convergent triangulation
(McGrath et al., 1982; Pedhazur & Schmelkin, 1991), less attention has been directed to the
holistic form of triangulation. This represents an important omission because holistic
triangulation offers the promise of capturing “more complete, holistic, and contextual portrayal
of the unit(s) under study” and enriching “our understanding by allowing for new or deeper
dimensions to emerge” (Jick, 1979, pp. 603-604). Relative to convergent triangulation, which
emphasizes knowledge development in the search for agreement, the holistic form asserts that
divergence can be just as important as convergence in the accrual of knowledge. This is based in
part on the idea that certain methods are uniquely capable of “seeing” particular aspects of a
phenomenon or “judging” particular attributes of a theory.1 Thus, through holistic triangulation,
1 A parallel discussion of convergence versus divergence can be found with respect to performance management in
the organizational behavior literature. For example, we might not expect to find convergence when performance of
an individual’s work is evaluated by peers versus a supervisor because these sources have different vantage points
and can provide qualitatively different types of information. We wish to thank a reviewer for highlighting this
observation.
2
scholars can generate more insight for a theory or develop more complete understandings of a
given phenomenon through the use of multiple methodologies (Denzin, 2012; Gibbert &
Ruigrok, 2010; Shah & Corley, 2006).
While scholars have made the case for the benefits and promise of methodological
triangulation in the organizational sciences (e.g., Jick, 1979; McGrath et al., 1982), in a related
review, Scandura and Williams (2000) found that triangulating across methods is still rare in the
organizational sciences. There are a number of reasons for the scarcity of studies employing
methodological triangulation. One core reason is that there is limited guidance available for
organizational scholars regarding how to design mixed methods research studies and particularly
for how to design holistic triangulation studies. Other important challenges include the time-
consuming nature of the work, the need to assemble and coordinate researcher expertise across
different methodological areas, and the demands associated with publishing mixed methods
research.
Thus, the objective of this article is two-fold: the first objective is to develop a research
design framework for mixed methods work that is based on the principles of triangulation, and
then based on that framework, the second objective is to provide a roadmap to guide
organizational scholars interested in conducting mixed methods research studies. Our emphasis
is on research that combines methods that stand more or less on equal footing, i.e., equivalent
methods, rather than situations in which one method is dominant and another is in a supporting
role (Molina-Azorin, 2007). While triangulation in research methodology can encompass a
number of aspects, such as using multiple sources of data and examining multiple empirical
settings (Scandura & Williams, 2000), as a methods foundation, our study draws from the
research strategies presented in McGrath (1982). In this study, we focus on the following
research strategies: archival records, 2 case studies, computer simulations, experimental
simulations, field experiments, formal theory (mathematical), interviews, lab experiments, and
surveys. In the article, we often use the research strategy and method terms interchangeably.
For the first objective, our research design framework is based on several core elements,
which are captured in Figure 1. The first two elements focus on theoretical and methodological
purpose – that is, the theoretical purpose for the research, and the purposes for the methods that
are utilized in that research. The third element focuses on how the multiple methodologies are
linked together given theoretical and methodological purpose and considers the implications of
the linking process for the theory at hand.3 For our second objective, we draw from this research
design framework to develop a roadmap that provides guidance for organizational scholars
regarding how to design and implement triangulation-based mixed methods research studies. We
conclude with further examination of two key issues involved in the design and implementation
2 Some scholars have placed emphasis on archival records as a distinct research strategy that involves the use of
archival sources of data (Austin, Scherbaum, & Mahlman, 2002; Scandura & Williams, 2000; Singleton & Straits,
1999). While Scandura and Williams (2000) incorporated this approach under the label of field studies that use
secondary data, we have adopted the archival records label used in Singleton and Straits (1999), given its distinction
from many field studies.
3 Given our emphasis on methods at the level of research strategies (McGrath, 1982) and their relationship with
theoretical purpose, our framework focuses more on design and to a lesser extent on measurement and analysis
among the core methodological topics of design, measurement, and analysis (e.g., Aguinis, Pierce, Bosco, & Muslin,
2009).
3
of mixed methods studies: how to select a set of methodologies that increases validity by
offsetting the limitations inherent to the individual methodologies, and how to overcome
practical challenges involved in conducting mixed methods research. This discussion considers
important assumptions and challenges associated with triangulation-based mixed methods
research that influence whether and how this research can produce heightened understanding.
*** Insert Figure 1 Here ***
The Foundation for Triangulation: Theoretical Purpose and Methodological Purpose
Theoretical purpose. To develop a research design framework for mixed methods research
involving triangulation, we first need to consider the theoretical purpose for the research study,
which often ties closely to the research question (Burton & Obel, 1995; Molina-Azorin, 2007).
Theoretical purpose refers to what researchers aim to accomplish in the study as it pertains to
theory; such aims include theory generation, theory elaboration, and theory testing (Lee,
Mitchell, & Sablynski, 1999). For mixed methods studies that focus on convergent triangulation,
the theoretical purpose may involve assessing the validity of an explanation by conducting
multiple tests using methodologies that are flawed in different ways, or it may involve starting
with one method for theory development,4 and concluding with another method for theory
testing. Thus, theory testing often plays an important role in convergent triangulation research.
By contrast, for studies focused on holistic triangulation, the theoretical purpose often centers on
the development of theory.
Methodological purpose. Following theoretical purpose, scholars pursuing mixed methods
research need to give careful attention to the purposes for which they are utilizing particular
methods. McGrath (1982, 1995) suggests that scholars select research strategies based on the
extent to which a given method can accomplish several objectives: (1) its ability to maximize
generalizability with respect to populations (e.g., actors), (2) its precision in control and
measurement of variables related to behaviors of interest, and (3) its ability to provide
authenticity of context for the observed behaviors. Accomplishment of these respective
objectives enables researchers to contend with threats to different types of validity. For example,
a method that offers precision in control and measurement of variables is well-suited for
addressing threats to internal validity, which refers to the validity of inferences regarding
whether covariation between an independent and dependent variable results from a causal
relationship, as opposed to the possibility that other extraneous variables are responsible for the
outcome. Alternatively, a method that can maximize generalizability with respect to populations
can contend with threats to external validity, which refers to the validity of inferences about
whether the causal relationship is generalizable to other subject populations, settings, and time
periods (Shadish, Cook, & Campbell, 2002). As such, the selection of research methodologies is
important because it influences the answers to our research questions in that the types of
evidence, nature of claims, and confidence in inferences depend considerably on which methods
are being employed.
4 For parsimony in presentation, we use “theory development” to encompass theory generation and theory
elaboration. In the former case, researchers ideally begin with no theory, while in the latter case, the objective is to
advance preexisting theory; in both cases, though, the objective is development of theory (Eisenhardt, 1989; Lee et
al., 1999).
4
According to McGrath (1982, 1995), certain research strategies are often well-suited to one
of these purposes; for example, lab experiments enable precision in control and measurement of
variables, while field studies (i.e., case studies) provide an authentic context within which the
behaviors of actors are observed. While this view emphasizes that individual methods tend to
have particular strengths and weaknesses, it is also important to keep in mind that certain
strategies can be utilized to accomplish different methodological purposes. For example,
computer simulations can be used to maximize generalizability with respect to populations, or
they can be used as laboratories for control and manipulation of variables (Burton, 2003; Davis,
Eisenhardt, & Bingham, 2007; Harrison, Lin, Carroll, & Carley, 2007). As another example,
archival records may be used to enhance precision in control and measurement of variables (e.g.,
focusing on measures well-suited for a particular group, firm, or industry), maximize
generalizability with respect to populations (e.g., seeking breadth across groups, firms, or
industries), or capture behaviors of interest that have taken place within an authentic context.
Table 1 highlights which methodological purposes are commonly fulfilled by particular research
strategies.
*** Insert Table 1 Here ***
Overall theoretical purpose (i.e., theory development, theory testing) and methodological
purpose (i.e., generalizability, precision in control/measurement, authenticity of context) provide
an important foundation for triangulation-based research design.
Linking Methods in Light of Theoretical and Methodological Purposes
In the previous section, we described how theoretical and methodological purposes are at the
foundation of triangulation-based research. In this section, we focus on the process of linking,
which is a central way in which scholarly understanding in a given area can be extended through
the use of mixed methods.
In the design of triangulation-based studies, linking refers to the process by which multiple
research strategies are brought together within a research study to realize the theoretical purpose
for that work. We argue that there are three core processes for linking research strategies in
triangulation-based research: (a) linking processes focused on convergent triangulation, (b)
linking processes focused on holistic triangulation, and (c) linking processes focused on
convergent and holistic triangulation. As described next, what distinguishes these processes is
how the set of methodologies relate to the theoretical purpose for the study and how the
researcher leverages methodological purpose in determining the complement of methodologies
utilized in the study.
Linking processes focused on convergent triangulation are commonly of two basic types; one
type focuses on theory testing, with two research strategies being used for the purpose of testing
the same theory, while the other type spans both categories of theoretical purpose, with one
research strategy focused on theory development and one focused on theory testing.5 For both
types of convergent triangulation, the researcher utilizes multiple methods to assess whether
5 For ease of explanation, we typically refer to linking processes as if two methods were being utilized, although
researchers could clearly use more than two methods in a mixed methods research study.
5
convergent results are observed across the methods.
Linking processes for holistic triangulation are focused on theory development and span at
least two categories of methodological purpose (e.g., prioritizing generalizability, prioritizing
control/measurement, prioritizing authenticity of context). For holistic triangulation, the
researcher utilizes multiple research strategies because he or she expects to learn different and
unique things from them.6
Linking processes focused on convergent and holistic triangulation span both categories of
theoretical purpose (i.e., theory development, theory testing) and involve at least two categories
of methodological purpose. For convergent and holistic triangulation, the researcher utilizes
multiple research strategies because he or she expects to obtain better understanding through
opportunities for the methods to show convergent results in certain areas and through unique
perspectives/angles that one or more of the individual methods can provide in other areas. Figure
2 provides a number of examples of these linking processes.
*** Insert Figure 2 Here ***
Linking of methodologies in convergent triangulation studies. For studies focused on
convergent triangulation, the objective for the researcher is to demonstrate more valid results by
finding agreement across different research strategies (Jick, 1979; McGrath, 1982). In Figure 2,
linking processes focusing on convergent triangulation are reflected in horizontal arrows within
theory testing (i.e., testing theory using two complementary methods) and in arrows that begin in
theory development and conclude in theory testing (i.e., developing and testing theory).
Good examples of convergent triangulation studies that test theory using two complementary
methods can be found in Bardolet, Fox, and Lovallo (2011) and Morris and Moore (2000).
Bardolet et al. begins with the observation that strategy scholars commonly explain subsidization
across business units in multi-business firms with agency theory (e.g., principal-agent
conflicts/negotiations). The researchers then propose that the cross-subsidization phenomenon
may be explained by a simpler argument that corporate executives have cognitive biases in favor
of even resource allocations. To test this argument, they initially turn to regressions using
archival data, specifically using Compustat data to generate a large dataset that spans many
industries. The researchers found support for their argument with regression analyses of the
archival data, but they were also aware that the archival-based analyses did not allow for
sufficient controls to rule out common explanations based on agency theory. Thus, they next
turned to a series of lab experiments as a complementary method for theory testing, which
offered greater capacity to control for such possibilities and provided consistent results, which
offered greater support for their argument.
As another example of convergent triangulation, Morris and Moore (2000) examined how
learning was influenced by counterfactual thinking, and how organizational accountability
influenced this process. The researchers distinguished between upward counterfactual
6 An alternative process could be one of holistic triangulation with the aim of theory testing. For example, multiple
methods might be used for their unique ability to test different aspects of the same theory (e.g., Flynn & Staw,
2004). We do not consider such a process here because of the emphasis on convergence with theory testing in mixed
methods research (i.e., multiple methods providing different tests for the same question).
6
comparisons, which compare actual events to better possible alternatives, and downward ones,
which foster thinking relative to worse possible alternatives. They argue that upward
counterfactual comparisons promote learning because they focus attention on factors preceding
outcomes that might be changed in order to produce better outcomes, which provides a frame for
learning how to act in similar situations in the future, and they argue that organizational
accountability inhibits upward counterfactual comparisons because it restricts complex and self-
critical thinking. In the study, the researchers tested these predictions through two
complementary methods: (a) statistical analyses using archival data of experienced pilots’
assessments of near-miss aviation incidents and (b) an experimental simulation in which
undergraduate students operate a flight simulator.7 The analyses initially focused on the archival
data from experienced pilots because it offered an authentic context as described in Table 1,
whereas the researchers described tests of similar processes in prior studies as limited to
academic and lab settings. While these analyses supported their predictions, the regressions
based on archival data could only rule out certain alternative explanations, so the researchers
next turned to an experimental simulation using a flight simulator that afforded greater precision
in control and measurement.
As a good example of the alternative form of convergent triangulation, i.e., developing and
testing theory, consider Cohen and Klepper (1996). The study examines the effect of
organizational size on the allocation of R&D effort between process and product innovation. The
theoretical purpose of the study was first to develop theory through a mathematical model that
explains the influence of size on the allocation of R&D effort and then to test the predictions
derived from the model with regression analyses using archival data. From the view of
methodological purpose, both methods prioritize generalizability with respect to populations of
organizations. Specifically, the researchers develop a theoretical model that they expect to hold
for organizations in a variety of industries, and they test the predictions using archival data from
the Federal Trade Commission that includes businesses in 36 different manufacturing industries.
As another example of this form of convergent triangulation, Grant, Berg, and Cable (2014)
examine how self-reflective job titles affect emotional exhaustion. Beginning with the purpose of
theory development, the researchers conducted a case study at the Make-A-Wish Foundation.
Based on their analyses of interviews, observations, and archival sources, the researchers
discovered that employees’ selecting their own job titles contributed to less emotional
exhaustion, and they proposed three mechanisms (self-verification, psychological safety, and
external rapport) to explain this effect. While the case study suited the methodological purpose
of authenticity of context, it was also considered to be an extreme case of employee burnout,
such that the researchers described selecting an alternative research strategy to test their
inductively-derived theory in a way that could support greater generalizability. Their choice of a
second research strategy – a field experiment – also contributed by offering greater precision in
control and measurement. Thus, they conducted a field experiment in a large health services
organization, which included pre- and post-intervention surveys, and the results supported their
proposition that choosing self-reflective …
Posttest-Only Control Group Design
In: The SAGE Encyclopedia of Educational Research,
Measurement, and Evaluation
By: Grant B. Morgan & Rachel L. Renbarger
Edited by: Bruce B. Frey
Book Title: The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation
Chapter Title: Posttest-Only Control Group Design
Pub. Date: 2018
Access Date: August 16, 2021
Publishing Company: SAGE Publications, Inc.
City: Thousand Oaks,
Print ISBN: 9781506326153
Online ISBN: 9781506326139
DOI: https://dx.doi.org/10.4135/9781506326139
Print pages: 1279-1281
© 2018 SAGE Publications, Inc. All Rights Reserved.
This PDF has been generated from SAGE Research Methods. Please note that the pagination of the
online version will vary from the pagination of the print book.
https://dx.doi.org/10.4135/9781506326139
The posttest-only control group design is a research design in which there are at least two groups, one of
which does not receive a treatment or intervention, and data are collected on the outcome measure after
the treatment or intervention. The group that does not receive the treatment or intervention of interest is
the control group. The general process for this design is that (a) two or more groups are formed; (b) the
treatment or intervention is administered; (c) data are collected after the treatment or intervention has been
administered, commonly using a behavioral, cognitive, or psychological assessment; and (d) the data are
compared between groups to determine whether the treatment or intervention was effective. The goal of this
design is often to make causal inferences; that is, to draw conclusions about whether or not a difference
between groups (i.e., the effect) is observed as the result receiving the intervention (i.e., the cause). This
entry presents considerations for using the posttest-only control group design for causal inference, discusses
the advantages and limitations of this method, and provides an example.
Considerations for Causal Inference
The three commonly referenced conditions that must be met in order to infer causality are (1) temporal
precedence of proposed cause and effect, (2) covariation of proposed cause and effect, and (3) elimination
of alternative explanations for the effect. For the first condition to be met, the treatment or intervention must
occur before differences between groups on the outcome variable (i.e., the effect) are observed. The posttest-
only control group design is able to meet this condition because the treatment or intervention is administered
before the potential group differences are observed. For the second condition to be met, it must be possible
for the researcher to determine what happens both when the treatment or intervention is present and when it
is absent. Again, the posttest-only control group design is able to meet this condition because it includes at
least one treatment or intervention group and one control group. For the third condition to be met, potential
alternative explanations for differences in group outcomes must be accounted for or ruled out. The posttest-
only control group design may, but does not necessarily, meet this condition.
To minimize such alternative explanations, groups are formed using random assignment, so that any
differences between groups that are observed before the administration of treatment or intervention will
be due to randomness. Ideally, there will be no preexisting group differences due to random assignment
and/or matching of some sort. The determining factor is how the groups are formed; if they are formed
using random assignment with matching (in which the researcher creates pairs of participants, one from the
treatment group and one from the control group, who have comparable important characteristics beyond
group membership) or without, then the posttest-only control group design is able to meet this condition.
If the groups are formed using a nonrandom mechanism (e.g., convenience, self-selection of participants
into groups, criterion-based inclusion, or already-existing groups), then the posttest-only control group design
cannot meet this condition. The way groups are formed for comparison is of crucial importance for inferring a
causal relationship between the treatment or intervention and the observed differences between groups. For
the types of research questions that lend themselves to posttest-only control group design, causal inference
is commonly the desired outcome, so random assignment with or without matching is recommended.
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 2 of 5 Posttest-Only Control Group Design
Advantages
The posttest-only control group design is commonly compared against the pretest–posttest control group
design. Because no data are collected before the administration of the treatment or intervention (i.e., no
pretest), the posttest-only control group design requires fewer resources (e.g., time, money, energy) for
data collection. In fact, collecting data prior to a treatment or intervention may not be possible or feasible.
Furthermore, the process of collecting pretest data may prepare participants or give them clues into the
intended effect of treatment or intervention (referred to as a testing threat to internal validity). This may affect
the way participants interact with the actual treatment or intervention compared with how they would have
acted without prior knowledge in a posttest-only design.
Limitations
There are a number of limitations of the posttest-only control group design. These limitations are primarily
related to not being able to rule out alternative explanations. First, regardless of how groups are formed, it
is possible that the treatment group(s) differ from the control group before any treatment or intervention is
ever administered. If this is the case, there is no mechanism in place to provide evidence of these differences
because no data were collected ahead of the treatment or intervention. Unknown preexisting differences may
well affect the effectiveness of the treatment or intervention in some way. Such preexisting differences may
result in the researcher reaching an incorrect conclusion about observed group differences being attributable
to the treatment or intervention.
A second limitation associated with the posttest-only control group design is maturation threat, which is a
more common threat to causal inferences in longer studies. Maturation refers to how people change over the
course of time, whether it be emotionally, physically, or mentally. Although the length of the study is usually
not long enough for people to naturally change in a meaningful way, the absence of a pretest prevents the
researcher from being able to collect data on the natural change of participants over the course of the study.
The expected growth of participants over the course of the study can only be assessed by examining the
change of the control group. As a result, the posttest-only control group may be subject to the maturation
threat on causal inference.
As noted earlier, the posttest-only control group design involves two or more groups, one of which is the
control group. As a result, the design is subject to certain limitations related to multiple groups. These
limitations have been referred to as social interaction threats to the inferring causal relationship(s), and many,
if not all, research designs involving multiple groups are subject to these same limitations.
The first of the multiple-group limitations is diffusion of treatment. This occurs when participants in the
control group receive the treatment or intervention from participants in the treatment or intervention group
directly or through some other means, and the control group adopts or tries to replicate the treatment or
intervention in their own group. The second of the multiple group limitations is rivalry. If the participants in
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 3 of 5 Posttest-Only Control Group Design
each group know which group(s) their scores are being compared against, the group members may begin to
compete with each other, thus affecting the results of the group comparisons. The third of the multiple-group
limitations is equalization of treatment, which occurs when there is some perceived desirability, value, or
inequity associated with being in the treatment or intervention group. If this is the case, there may be internal
or external pressure to reassign participants to the treatment or intervention or to administer it to everyone
to promote fairness. Obviously, this would interfere with the design’s ability to demonstrate the effectiveness
of the treatment or intervention. The last of the multiple-group limitations is resentful demoralization, which
occurs when participants in the control group become aware that they are not receiving the treatment
or intervention and, being demoralized by this discovery, attempt to sabotage the research. For example,
they may perform worse on the assessments than they would ordinarily, artificially increasing the observed
differences between groups.
Example
Two scenarios within the same context are presented here to demonstrate the posttest-only control group
design and its implications on how the treatment and control groups are formed. Suppose a fifth-grade
teacher hypothesizes that learning and practicing chess will increase math achievement in students who
participate in an afterschool program. She decides to collect data that will allow her to test her hypothesis.
In the first scenario, she randomly chooses half of the students in the afterschool program to learn and
practice chess, and she administers the program with fidelity for 4 weeks. At the end of the 4-week period,
she administers a math assessment with demonstrated reliability and validity evidence to all students in the
afterschool program—those who participated in the chess program and those who did not. The students who
participated in the chess program received higher math scores than those who did not. This scenario is an
example of a posttest-only control group design. In a second scenario, the teacher allows any student who is
interested to participate in the chess program. After students self-select into the program, she administers the
program with fidelity for 4 weeks. At the conclusion of the 4-week period, she administers a math assessment
to all students in the afterschool program—those who participated in the chess program and those who did
not. Those who participated in the chess program had higher math scores than those who did not participate
in the chess program. This is also an example of a posttest-only control group design.
With which scenario would you feel more comfortable concluding that the observed differences in math
achievement was attributable to the chess program? In the first instance, the groups were formed through a
random process, and any differences that existed between the group members were therefore due to chance.
In the second scenario, the groups self-selected into or out of the chess program, and as a result, there
are known differences between the group members, namely, interest in chess. Therefore, the program may
have a different effect for those students than it would have had for “typical” students. As a result, it would
be impossible to separate the interaction between interest in chess and effect of chess on the observed
difference in posttest math achievement.
For the purposes of this entry, the posttest-only control group design is exemplified through these two
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 4 of 5 Posttest-Only Control Group Design
scenarios. As demonstrated, it is important to consider the threats to internal validity in the use and decision
making associated with this research design.
See also Analysis of Variance; Causal Inference; Experimental Designs; Nonexperimental Designs;
Pretest–Posttest Designs; Random Assignment; t Tests
Grant B. Morgan & Rachel L. Renbarger
http://dx.doi.org/10.4135/9781506326139.n530
10.4135/9781506326139.n530
Further Readings
Kirk, R. E. (2013). Experimental design: Procedures for the behavioral sciences (4th ed.). Thousand Oaks,
CA: SAGE.
Maxwell, S. E., & Delaney, H. D. (2004). Designing experimental and analyzing data: A model comparison
perspective (2nd ed.). Mahwah, NJ: Erlbaum.
Trochim, W. M. K., & Donnelly, J. P. (2006). The research methods knowledge base (3rd ed.). Mason, OH:
Atomic Dog.
SAGE
2018 SAGE Publications, Ltd. All Rights Reserved.
SAGE Research Methods
Page 5 of 5 Posttest-Only Control Group Design
http://methods.sagepub.com/reference/the-sage-encyclopedia-of-educational-research-measurement-and-evaluation/i2602.xml
http://methods.sagepub.com/reference/the-sage-encyclopedia-of-educational-research-measurement-and-evaluation/i4418.xml
http://methods.sagepub.com/reference/the-sage-encyclopedia-of-educational-research-measurement-and-evaluation/i8788.xml
http://methods.sagepub.com/reference/the-sage-encyclopedia-of-educational-research-measurement-and-evaluation/i14629.xml
http://methods.sagepub.com/reference/the-sage-encyclopedia-of-educational-research-measurement-and-evaluation/i16443.xml
http://methods.sagepub.com/reference/the-sage-encyclopedia-of-educational-research-measurement-and-evaluation/i17241.xml
http://methods.sagepub.com/reference/the-sage-encyclopedia-of-educational-research-measurement-and-evaluation/i20520.xml
http://methods.sagepub.com/reference/the-sage-encyclopedia-of-educational-research-measurement-and-evaluation/i20520.xml
http://dx.doi.org/10.4135/9781506326139.n530
Posttest-Only Control Group Design
In: The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation
CATEGORIES
Economics
Nursing
Applied Sciences
Psychology
Science
Management
Computer Science
Human Resource Management
Accounting
Information Systems
English
Anatomy
Operations Management
Sociology
Literature
Education
Business & Finance
Marketing
Engineering
Statistics
Biology
Political Science
Reading
History
Financial markets
Philosophy
Mathematics
Law
Criminal
Architecture and Design
Government
Social Science
World history
Chemistry
Humanities
Business Finance
Writing
Programming
Telecommunications Engineering
Geography
Physics
Spanish
ach
e. Embedded Entrepreneurship
f. Three Social Entrepreneurship Models
g. Social-Founder Identity
h. Micros-enterprise Development
Outcomes
Subset 2. Indigenous Entrepreneurship Approaches (Outside of Canada)
a. Indigenous Australian Entrepreneurs Exami
Calculus
(people influence of
others) processes that you perceived occurs in this specific Institution Select one of the forms of stratification highlighted (focus on inter the intersectionalities
of these three) to reflect and analyze the potential ways these (
American history
Pharmacology
Ancient history
. Also
Numerical analysis
Environmental science
Electrical Engineering
Precalculus
Physiology
Civil Engineering
Electronic Engineering
ness Horizons
Algebra
Geology
Physical chemistry
nt
When considering both O
lassrooms
Civil
Probability
ions
Identify a specific consumer product that you or your family have used for quite some time. This might be a branded smartphone (if you have used several versions over the years)
or the court to consider in its deliberations. Locard’s exchange principle argues that during the commission of a crime
Chemical Engineering
Ecology
aragraphs (meaning 25 sentences or more). Your assignment may be more than 5 paragraphs but not less.
INSTRUCTIONS:
To access the FNU Online Library for journals and articles you can go the FNU library link here:
https://www.fnu.edu/library/
In order to
n that draws upon the theoretical reading to explain and contextualize the design choices. Be sure to directly quote or paraphrase the reading
ce to the vaccine. Your campaign must educate and inform the audience on the benefits but also create for safe and open dialogue. A key metric of your campaign will be the direct increase in numbers.
Key outcomes: The approach that you take must be clear
Mechanical Engineering
Organic chemistry
Geometry
nment
Topic
You will need to pick one topic for your project (5 pts)
Literature search
You will need to perform a literature search for your topic
Geophysics
you been involved with a company doing a redesign of business processes
Communication on Customer Relations. Discuss how two-way communication on social media channels impacts businesses both positively and negatively. Provide any personal examples from your experience
od pressure and hypertension via a community-wide intervention that targets the problem across the lifespan (i.e. includes all ages).
Develop a community-wide intervention to reduce elevated blood pressure and hypertension in the State of Alabama that in
in body of the report
Conclusions
References (8 References Minimum)
*** Words count = 2000 words.
*** In-Text Citations and References using Harvard style.
*** In Task section I’ve chose (Economic issues in overseas contracting)"
Electromagnetism
w or quality improvement; it was just all part of good nursing care. The goal for quality improvement is to monitor patient outcomes using statistics for comparison to standards of care for different diseases
e a 1 to 2 slide Microsoft PowerPoint presentation on the different models of case management. Include speaker notes... .....Describe three different models of case management.
visual representations of information. They can include numbers
SSAY
ame workbook for all 3 milestones. You do not need to download a new copy for Milestones 2 or 3. When you submit Milestone 3
pages):
Provide a description of an existing intervention in Canada
making the appropriate buying decisions in an ethical and professional manner.
Topic: Purchasing and Technology
You read about blockchain ledger technology. Now do some additional research out on the Internet and share your URL with the rest of the class
be aware of which features their competitors are opting to include so the product development teams can design similar or enhanced features to attract more of the market. The more unique
low (The Top Health Industry Trends to Watch in 2015) to assist you with this discussion.
https://youtu.be/fRym_jyuBc0
Next year the $2.8 trillion U.S. healthcare industry will finally begin to look and feel more like the rest of the business wo
evidence-based primary care curriculum. Throughout your nurse practitioner program
Vignette
Understanding Gender Fluidity
Providing Inclusive Quality Care
Affirming Clinical Encounters
Conclusion
References
Nurse Practitioner Knowledge
Mechanics
and word limit is unit as a guide only.
The assessment may be re-attempted on two further occasions (maximum three attempts in total). All assessments must be resubmitted 3 days within receiving your unsatisfactory grade. You must clearly indicate “Re-su
Trigonometry
Article writing
Other
5. June 29
After the components sending to the manufacturing house
1. In 1972 the Furman v. Georgia case resulted in a decision that would put action into motion. Furman was originally sentenced to death because of a murder he committed in Georgia but the court debated whether or not this was a violation of his 8th amend
One of the first conflicts that would need to be investigated would be whether the human service professional followed the responsibility to client ethical standard. While developing a relationship with client it is important to clarify that if danger or
Ethical behavior is a critical topic in the workplace because the impact of it can make or break a business
No matter which type of health care organization
With a direct sale
During the pandemic
Computers are being used to monitor the spread of outbreaks in different areas of the world and with this record
3. Furman v. Georgia is a U.S Supreme Court case that resolves around the Eighth Amendments ban on cruel and unsual punishment in death penalty cases. The Furman v. Georgia case was based on Furman being convicted of murder in Georgia. Furman was caught i
One major ethical conflict that may arise in my investigation is the Responsibility to Client in both Standard 3 and Standard 4 of the Ethical Standards for Human Service Professionals (2015). Making sure we do not disclose information without consent ev
4. Identify two examples of real world problems that you have observed in your personal
Summary & Evaluation: Reference & 188. Academic Search Ultimate
Ethics
We can mention at least one example of how the violation of ethical standards can be prevented. Many organizations promote ethical self-regulation by creating moral codes to help direct their business activities
*DDB is used for the first three years
For example
The inbound logistics for William Instrument refer to purchase components from various electronic firms. During the purchase process William need to consider the quality and price of the components. In this case
4. A U.S. Supreme Court case known as Furman v. Georgia (1972) is a landmark case that involved Eighth Amendment’s ban of unusual and cruel punishment in death penalty cases (Furman v. Georgia (1972)
With covid coming into place
In my opinion
with
Not necessarily all home buyers are the same! When you choose to work with we buy ugly houses Baltimore & nationwide USA
The ability to view ourselves from an unbiased perspective allows us to critically assess our personal strengths and weaknesses. This is an important step in the process of finding the right resources for our personal learning style. Ego and pride can be
· By Day 1 of this week
While you must form your answers to the questions below from our assigned reading material
CliftonLarsonAllen LLP (2013)
5 The family dynamic is awkward at first since the most outgoing and straight forward person in the family in Linda
Urien
The most important benefit of my statistical analysis would be the accuracy with which I interpret the data. The greatest obstacle
From a similar but larger point of view
4 In order to get the entire family to come back for another session I would suggest coming in on a day the restaurant is not open
When seeking to identify a patient’s health condition
After viewing the you tube videos on prayer
Your paper must be at least two pages in length (not counting the title and reference pages)
The word assimilate is negative to me. I believe everyone should learn about a country that they are going to live in. It doesnt mean that they have to believe that everything in America is better than where they came from. It means that they care enough
Data collection
Single Subject Chris is a social worker in a geriatric case management program located in a midsize Northeastern town. She has an MSW and is part of a team of case managers that likes to continuously improve on its practice. The team is currently using an
I would start off with Linda on repeating her options for the child and going over what she is feeling with each option. I would want to find out what she is afraid of. I would avoid asking her any “why” questions because I want her to be in the here an
Summarize the advantages and disadvantages of using an Internet site as means of collecting data for psychological research (Comp 2.1) 25.0\% Summarization of the advantages and disadvantages of using an Internet site as means of collecting data for psych
Identify the type of research used in a chosen study
Compose a 1
Optics
effect relationship becomes more difficult—as the researcher cannot enact total control of another person even in an experimental environment. Social workers serve clients in highly complex real-world environments. Clients often implement recommended inte
I think knowing more about you will allow you to be able to choose the right resources
Be 4 pages in length
soft MB-920 dumps review and documentation and high-quality listing pdf MB-920 braindumps also recommended and approved by Microsoft experts. The practical test
g
One thing you will need to do in college is learn how to find and use references. References support your ideas. College-level work must be supported by research. You are expected to do that for this paper. You will research
Elaborate on any potential confounds or ethical concerns while participating in the psychological study 20.0\% Elaboration on any potential confounds or ethical concerns while participating in the psychological study is missing. Elaboration on any potenti
3 The first thing I would do in the family’s first session is develop a genogram of the family to get an idea of all the individuals who play a major role in Linda’s life. After establishing where each member is in relation to the family
A Health in All Policies approach
Note: The requirements outlined below correspond to the grading criteria in the scoring guide. At a minimum
Chen
Read Connecting Communities and Complexity: A Case Study in Creating the Conditions for Transformational Change
Read Reflections on Cultural Humility
Read A Basic Guide to ABCD Community Organizing
Use the bolded black section and sub-section titles below to organize your paper. For each section
Losinski forwarded the article on a priority basis to Mary Scott
Losinksi wanted details on use of the ED at CGH. He asked the administrative resident