Fluent Essays

see peer. Peer 1 Analyze how you can maintain high standards and demonstrate high expectations for all ethnically, culturally, and linguistically diverse students in the classroom. When it comes to being a teacher, you want to make sure that you are showing the best communication skills that you can within the cultural background to all your students that you are teaching in the classroom. When you are a teacher, you want to first get to know all your students and their background and making sure that your classroom is a safe environment when the students are entering your classroom each morning to learn as well. The students will have a great relationship when it comes to teaching them within their trust and support. I believe that when it comes to the first thing with the classroom setting is to have a great standard for what the students will be learning in there as well. Also, you want to maintain a higher standard and demonstrating when it comes to the expectation within the ethnically, linguistically, and culturally diverse classroom for the students and the rules that they need to manage as well. There is a lot of different standards that the students have when it comes to the experts in the learning from the classroom. Since there is a lot of different cultural backgrounds in the classrooms, I will be instructing the students on the diversity learning within their styles, needs, and skills that they will be learning in the classroom. The students will be challenged when it comes to the differentiated instruction learning in the different forms that they need to learn. When it comes to doing this, the students will be able to learn easier and I will be able to have all the students come together in the form of their goals that are expected from each other. When it comes to the rules in the classroom, everyone will be following them within the regulations and the part of the rules are created within the rules as well. I will make sure that I am consistent when it comes to the standards that I have made in the classroom as well. Discuss how you would differentiate instruction for the inclusion of various learning styles. When it comes to being a teacher, you want to make sure that the students understand that they are learning in different ways within their brain work as well.  When it comes to the differentiating of the instruction within the inclusions, this means that I will be able to introduce the learning styles in different variations. Howard Gardner's multiple intelligence is the style that is what he has called this. Ever since this class is dealing with inclusion, then I have to make sure that the students that have a different style have a learning area within the multiple intelligence as well. When you are studying the students, this will make your job easier for the teaching styles better for you to teach them. When it comes to the students and their different styles of learning, this is where the compresses of intelligence begin.  There are different things that the students have in their environment and they are visual-spatial that are very aware within their classroom. Then, there is bodily-Kinesthetic that the students will be able to learn easier when it comes to their movement, making different things, and touching as well. When it comes to the students, they will be learning throughout the interaction-intrapersonal, but within the students, they will be tended to be a little shy when it comes to being away from the other classmates as well. Then, lastly, you have the higher development of the auditory skills that the students will be learning within the skills as well. I will be making sure that the students are using the learning styles in different ways within the instruction for the inclusion styles as well. Reflect on ways in which you would modify instruction for students in your area of specialization. When it comes to teaching the students in my area, we want to make sure that they are instructed within the specialization and making sure that the students are able to be modified by the instructions as well. As the special education students in the classroom, they need to make sure that I know the reason behind their disability and how I can help them develop their needs as much as possible too. The regular students would be able to introduce themselves within the 21st century when it comes to my area as I will be working with them called the 4cs. What are the 4cs you ask? They are communication, creativity, collaboration, and critical thinking. All four of these go a long way when it comes to the students in the classroom.  The 21st century has helped the students out when it comes to keeping the students in school at all times within the framework too. Reference Framework for 21st Century Learning Links to an external site. . (n.d.). Retrieved from http://www.p21.org/our-work/p21-framework tecweb.org/styles/gardner.html https://wvde.state.wv.us/counselors/links/initiatives/21st-century-framework.pdf (Links to an external site.) Peer 2   Analyze how you can maintain high standards and demonstrate high expectations for all ethnically, culturally, and linguistically diverse students in the classroom. As educators with students from culturally diverse backgrounds from minority communities in our classroom, we must offer a multicultural environment.  This will maintain a high standard in our classroom at all times. This might mean that all students need to feel comfortable in a classroom environment in which their culture is available around the class. There are many reasons children from culturally diverse backgrounds are discriminated against when they have disabilities. According to Taylor and Quintana (2003), "to teach successfully in culturally diverse classrooms, school teachers need to become more aware of their multicultural perceptions as their beliefs and behaviors affect the academic and social skill development of their students" (as cited in Koyama et al., 2012). By showing students of different backgrounds what other cultures are like, educators in the classroom not only provide a basis for quality learning but also teach students that everyone is the same no matter what culture they come from.   Reflect on ways in which you will choose culturally relevant curriculum and instructional materials that recognize, incorporate, and reflect students' heritage and the contributions of various ethnic groups. To provide students with an effective teaching method, the teacher must know well their cultural background. Students from different cultures learn differently and may need another way of teaching. Since students from diverse backgrounds speak various languages, promoting multiculturalism builds a multicultural library in the classroom. The Framework for 21st Century Learning website enables different learning and teaching methods to help all students, including those with diverse cultural backgrounds in the area they need help. They combine different outcomes with assisting students in mastering a skill like key subjects, media, technology, curriculum instruction, and professional development. This will help the students to feel comfortable and better understood because their culture is represented around their classroom. For example, including posters and flags depicting some of the countries of origin, including ethnic foods, and inviting families to share their culture in the classroom can improve students' understanding of the diverse, multicultural environment in which they learn.   Discuss how you would differentiate instruction for the inclusion of various learning styles. Differentiated instruction helps the teacher make changes in the classroom according to the different types of students. In addition, the teacher's goal is to make sure that the student wants to come to class every day to learn something new (Koschmeder, 2012).  For example, students need language, communication, reading comprehension, classroom management, and self-care skills. The special education teacher can find accommodation by creating small groups for class topics and discussions. Students usually participate more when they are in smaller groups versus larger groups. In my experience, when students are not in group activities, they rarely raise their hands. Small group discussions help them feel free to comment and raise their hands to answer questions or give their perspective. Flexible grouping promotes communication. As educators providing differentiated instruction, we offer individualized attention to work toward the uniqueness of our student's learning styles and abilities. We can also create differentiated lessons by giving a personalized packet to help the student understand the material. The use of technology enhances the lesson and provides a student a different way to present the information being taught, such as recording or text to talk. Provide options to show the end material and use various forms of assessments to accommodate students who might not understand the lesson in traditional assessment forms. References   Framework for 21st Century Learning. (n.d.). Retrieved August 30, 2021, from http://www.p21.org/our-work/p21-framework   Koschmeder, C. (2012). What is differentiated Instruction? Youtube. https://www.youtube.com/watch?v=YAWKxpCv1Fw   Koyama, C., Plash, S., & Davis, K. (2012). Comparing cross-cultural, multicultural self-awareness among K-12 in-service school teachers . SRATE Journal, 21(1), 29-37.  http://files.eric.ed.gov/fulltext/EJ959534.pdf Analyze how you can maintain high standards and demonstrate high expectations for all ethnically, culturally, and linguistically diverse students in the classroom. Being a twenty year educator and learning from older and younger teachers, I truly believe that teachers and myself must maintain high standards and expectations for all my students by first establishing classroom management and holding every student who walks into their class to the same expectations and never lowering those expectations for any particular individual. I  truly believe that all students have the ability to learn, whether they are special education or general education students. It's about tapping into their young minds and taking the time out to see what they truly interest them. The specific behaviors that teachers must communicate loud and clear with no grey lines regarding their expectations for all students. I attack different areas students differently to ensure they also have high expectations for themselves academically and socially in my classroom and helping them meet their goals and objectives. You have to differentiate instruction in this day and age because of the diversity individuals who sit in your classroom. Through differentiated instruction, all students learn the same lesson but all students learn different.  Discuss how you would differentiate instruction for the inclusion of various learning styles. All students are unique in their way, it's like a snowflake, it's never two students who are the same. Some children are visual learners, some are verbal learners and others are kinesthetic learners. Some different instruction should include art, music, videos and multiple visual aides. Reading, math , writing , spelling and listening skills for my students are very important for them to master in my classroom. Hands or activities and group projects is also an important part of my classroom in which all students must master. Music is another form in which they will listen to different types of genre while working, which actually helps a lot of students stay calm and relaxed while doing individual or group work. In making sure my classroom environment inclusion, all of my students will interact with one another regarding the learning process in what they like and dislike within the classroom.  Reflect on the ways in which you would modify instruction  for students in your area  of specialization. Looking back on my classroom during the 2020-21 school year, there were individuals whom needed modify instructions in which they had special needs related to a specific emotional, physical or behavioral challenge. Accommodations and modifications are necessary to allow those students the equal opportunity as their regular education peers. Moving the child closer to the front of the classroom and near me is one solution. I had one student who used a recorder and then go back to find answer or  main points what we were talking about as a whole group. i like to use big manipulatives around the room for visual sights. Modify instruction allows some students to learn at their present levels , rather than failing to comprehend what's being taught to them.  Reference(s)   Framework for 21st Century Learning (Links to an external site.) . (n.d.). Retrieved from http://www.p21.org/our-work/p21-framework   Mertler, C. A. (2017).  Action research: Improving schools and empowering educators  (5th ed.) Sage Publications. · Chapter 5: Collecting Data · EDU694 · Pages · Week 4 - Instructor Guidance 08/10/2021 - UAGC Graduate · Home · Announcements · Syllabus · Modules · Grades · Course Policies · Writing Center & Library · Course Resources · BigBlueButton (Formerly Conferences) · Student Support Center WEEK 4 INSTRUCTOR GUIDANCE Welcome to Week 4 of EDU 694: Capstone 1: Educational Research. Please be sure to review the Week 4 homepage for this course to see: · The specific learning outcomes for the week · The schedule overview · The required and recommended resources · The introduction to the week · A listing of the assessments Next, be sure to read this entire Instructor Guidance page. Overview In Week 3, you continued discussing and identifying the cycle of Action Research and how research applies to ethics. This week you will look at how Action Research can have an impact on 21st century teaching and learning as well as create a visual representation on the different types research data collection. Intellectual Elaboration     Thinkstock Image # 65341841 (Links to an external site.) 21st Century Teaching and Learning It will help if before we discuss diversity if you reflect on your knowledge about 21st Century Teaching and Learning skills. Although Week 1focuses on 21st Century Skills, the other significant focus is on diversity in the school context. The overarching question for the week is: How do we apply these 21st Century Teaching and Learning Skills to assist diverse students?  After viewing the two videos recommended below, you will explain how 21st century teaching and learning skills are applicable to both teacher and student. While watching the videos, ask yourself what you already use in the classroom to support 21st century teaching and learning and what more you could do to ensure students are obtaining the skills they need for future learning. Review this video from Teknolojileri (2013) (Links to an external site.)  discussing 21st century skills in approximately two minutes. Specifically, Teknolojileri discusses the skills of creativity, communication, collaboration, critical thinking, media literacy, information literacy, and problem solving for students, which are a part of the 21st century teaching and learning plan. Next, review this approximately three minute video from  Knowledge Delivery Systems (2013) (Links to an external site.)  that explores the fundamentals of 21st Century Skills and what they mean to the practice of educators. Diversity and Culturally Responsive Pedagogy The contemporary classroom requires teachers to educate students from a variety of backgrounds, including culture, language, and learning abilities (Gollnick & Chinn, 2002). Therefore, as these increasing numbers of students from diverse backgrounds populate 21st century classrooms in the United States, the need for pedagogical appropriate lessons that are also culturally responsive is magnified. Culturally responsive and pedagogically appropriate lessons are important to create because when diverse students come together in a classroom, students’ thoughts expressed verbally, graphically or in writing reveal a wealth of information about each student’s thinking. Furthermore, through exposure to the diversity of classmates, students tend to expand their thinking skills and move away from linear thinking patterns toward thinking about their world and their learning outside of their culture and previous education experiences.  On this point, view the George (2011) (Links to an external site.)  video, which is about three minutes in length, discussing how innovation occurs when diverse minds collaborate. As you watch the video, think about 21st Century Teaching and Learning and consider: How has the school shown in the video, incorporated 21 Century Teaching and Learning to include diversity in the classroom and school? Consider sharing your thoughts on the video as part of your discussions responses this week or start a conversation in the Global Campus Café! Next, consider how diversity, as it relates to balanced achievement, is a primary goal of every school. Yet, recall that diversity is only one aspect of a complex phenomenon. If we dive deeper into our learning of diversity, you will remember that culturally responsible pedagogy is a deeper aspect of learning and teaching with diverse student populations. Lynch (2011) describes culturally responsive pedagogy as a style of teaching that facilitates and supports the achievement of all students. In a culturally responsive classroom, effective teaching and learning occur in a culturally supported, learner-centered context, whereby the strengths students bring to school are identified, nurtured, and utilized to promote student achievement (Lynch, 2011, para. 3). Culturally responsive pedagogy comprises of three dimensions: (a) institutional, (b) personal, and (c) instructional. The institutional dimension reflects the administration and its policies and values. The personal and instructional dimension refers to the cognitive and emotional processes teachers must engage in to become culturally responsive (Ladson-Billings, 2012, p.65). Research Methodologies: Qualitative and Quantitative Research Research methodologies, like salt and pepper, or peanut butter and jelly, come in two distinct, recognizable, metaphorical flavors: Qualitative (Links to an external site.)  and Quantitative. Each comes with its own ideas and philosophies, strengths and weaknesses, proponents and detractors, heroes and knaves. The purpose of this Guidance is to outline these two types of methodologies and provide links to additional information. Qualitative Research Qualitative research emerged in the last half of the 20th century as a counterpoint to quantitative methodologies. Qualitative research is: · Non-Numeric—It is concerned with written description, rather numerical analysis;   · Inductive—Research questions emerge from the collected data; · Social—It is concerned with people in social situations; · Field Research Oriented—The research is conducted on-site, in the field where the social situation is taking place; · Observational—Data is collected largely through observations of the actions of others; · Participatory—The researcher is often a participant in the events he/she is observing/recording/researching; and · Coded—Coding (Links to an external site.)  is the chief analysis tool used by qualitative researchers.  Methods used by qualitative researchers include: · Action Research —A “learning by doing” approach to research (O’Brien, 1998, para 3); · Case Studies (Links to an external site.) —A focused study on a single case (individual or group) in a specific setting; · Ethnographies —Often conducted by participant observers; · Interviews (Links to an external site.) —Often seeks to find the story behind a particular set of experiences.  Incidentally, interviews were at the center of my doctoral dissertation study.   In sum, then, Qualitative research has clear value in capturing human experience onsite/in the field, recording, coding, and analyzing that experience, and reporting it in interesting ways that enhance our understanding of the human condition.   Quantitative Research Quantitative research emerged during the Scientific Revolution as a way to examine, observe, and analyze natural phenomena. Quantitative research is: · Numeric—Data is collected in numerical form, and is analyzed using statistical means; · Deductive—It begins with a particular point of view (hypothesis) and sets out to prove/disprove it.   · Classic—It mirrors the “Scientific Method.”  Methods used by quantitative researchers include: · Correlations (Links to an external site.) —Examines relationships between two or more phenomena—See Type 6 in linked material (p.  3) for more information; · Evaluations (Links to an external site.) —Seeks to make judgments about the value of particular events—See Type 8 in linked material (p.  3) for more information; · Experiments (Links to an external site.) —Common in scientific research using group research design; · Single-subject research (Links to an external site.)  ---A research design commonly used in low-incidence populations and applied research · Meta-Analysis —A research study of research studies—This seeks to draw a single set of conclusions through review and analysis of many similar research studies.  A particularly good example for education is Hattie, 2011, Visible Learning.   · Surveys (Links to an external site.) —Probably the most common quantitative research method—Used in everything from market research to political poling.   Quantitative research analysis relies on Descriptive (Links to an external site.)  and Inferential (Links to an external site.)  statistical methods.    In sum, then, Quantitative research has clear value in numerically and scientifically examining phenomenon that lead to ways that enhance our further understanding of the human condition.   Closing Remarks When teachers are aware of diversity in the classroom and are culturally responsive, while incorporating 21st Century Skills into everyday curriculum, it creates a more effective and supporting learning environment for all students. Assessment Guidance This section includes additional specific assistance for excelling in the discussions for Week 4 beyond what is given with the instructions for the assessments. If you have questions about what is expected on any assessment for Week 4, contact your instructor before the due date. Discussion 1: Diversity Through 21st – Century  Teaching and Learning This discussion provides you the opportunity to demonstrate your knowledge of 21st Century Learning as it relates to diverse student populations. In this discussion you will  select three of the five prompts provided , and then discuss how the Framework for 21st century learning can be applied to each prompt using specific examples of the actions you would take to apply the framework. Assignment: Visual Interpretation This discussion is an opportunity for you to get creative by creating a visual representation of the different types of data collection tools that can be used in Action Research. Have fun with this assignment, get creative and use a digital tool that is new to you. This assignment is also an opportunity to work on your Week 6 Assignment, you will skim your Action Research report, chosen in Week 2, for the types of data used and describe in at least one page the different types of data chosen and report out on the findings.   References Bloomfield, J., & Fisher, M. J. (2019). Quantitative research design. Journal of the Australasian Rehabilitation Nurses’ Association (JARNA), 22(2), 27–30. https://doi-org.proxy-library.ashford.edu/10.33235/jarna.22.2.27-30 m California State University, Long Beach. (n.d.). Data collection strategies II: Qualitative research (Links to an external site.) . Retrieved from http://www.csulb.edu/~msaintg/ppa696/696quali.htm Embury, D. C., Parenti, M., & Childers-McKee, C. (2020). A charge to educational action researchers. Action Research, 18(2), 127–135. https://doi-org.proxy-library.ashford.edu/10.1177/1476750320919189 Gay, G. 2003. The Importance of multicultural education. Educational Leadership, 61(4), 30-35. Genzuk, M. (1999). A synthesis of ethnographic research (Links to an external site.) . Retrieved from http://www-bcf.usc.edu/~genzuk/Ethnographic_Research.html George, J. ( 2011, May 11). Innovation through diversity. [Video file]. Retrieved from http://www.youtube.com/watch?v=FjrjiSecZv0&feature=youtu.be Gollnick, D., & Chinn, P. (2002). Multicultural education in a pluralistic society. NJ: Pearson. Hattie, J. (2011). Visible learning: A synthesis of over 800 meta-analyses related to achievement. New York: Routledge Heffner, C. (2016). Chapter 4: Single subject design (Links to an external site.) . Research Methods. FL: AllPsych. Retrieved from http://allpsych.com/researchmethods/singlesubjectdesign/ Hess, K., Carlock, D., Jones, B., & Walkup, J. (2009). What exactly do “fewer, clearer, and higher standards” really look like n the classroom? Using a cognitive rigor matrix to analyze curriculum, plan lessons, and implement assessments. Retrieved from http://www.sde.idaho.gov/site/common/webinars/Cognitive%20Rigor%20Matrix%20Article_Hess,%20Carlock,%20Jones,%20and%20Walkup.pd Ismail, M. (2005). Quantitative research methodology (Links to an external site.) . Scribd. Retrieved from http://www.scribd.com/doc/98295/Quantitative-Research-Methodology Johnson, R., & Christiansen, L. (2007). Table 14.1: Strengths and weaknesses of quantitative research (Links to an external site.) . University of South Alabama. Retrieved from http://www.southalabama.edu/coe/bset/johnson/oh_master/Ch14/Tab14-01.pdf Johnson, R., & Christiansen, L. (2007.). Table 14.2: Strengths and weaknesses of qualitative research (Links to an external site.) . University of South Alabama. Retrieved from http://www.southalabama.edu/coe/bset/johnson/oh_master/Ch14/Tab14-02.pdf Knowledge Delivery Systems (2013, June 5). 21st century skills: Rethinking how students learn. [Video file]. Retrieved from http://www.youtube.com/watch?v=qMG5dvhEzyo Kozol, J. 2012. Savage inequalities: Children in America's schools (Reprint ed.). Portland, OR: Broadway Books. Ladson-Billings, G. (2012). The dreamkeepers. San Francisco: Jossey-Bass. Lynch, M. (2011, December). What is cultural responsive pedagogy? Retrieved from http://www.huffingtonpost.com/matthew-lynch-edd/culturally-responsive-pedagogy_b_1147364.html Boslaugh, S. E. . M. (2019). Meta-Analysis. Salem Press Encyclopedia. http://search.ebscohost.com.proxy-library.ashford.edu/login.aspx?direct=true&db=ers&AN=89677588&site=eds-live&scope=site O'Brien, R. (1998). An overview of the methodological approach of Action Research. University of Toronto. Retrieved from http://www.web.ca/robrien/papers/arfinal.html O’Shaughnessy, K., McDonald, C., Maher, H., and Dobie, A. (2002, Fall). Who, what, when, and where of writing rituals. The Quarterly 24 (4) Proud to be Green. (2011, March 9). Using the scientific method while going green (Links to an external site.) . Retrieved from http://proud2bgreen.wordpress.com/2011/03/09/year-to-year-energy-usage-comparison/ Reardon, S. F. (2013). "The widening income achievement gap". Educational Leadership, (8): 10-16. Ryan, M. (2009). Making visible the coding process: Using qualitative data software in a post-structural study (Links to an external site.) . Issues In Educational Research,19(2). Retrieved from http://www.iier.org.au/iier19/ryan.html Teknolojileri, B. (2013, May).  21st century skills (Links to an external site.)  [Video file]. Retrieved from http://www.youtube.com/watch?v=qwJIhZcAd0I Trochim, W. (2006). Descriptive statistics (Links to an external site.) . Research Methods Knowledge Base. Retrieved from http://www.socialresearchmethods.net/kb/statdesc.php Trochim, W. (2006). Inferential statistics (Links to an external site.) . Research Methods Knowledge Base. Retrieved from http://www.socialresearchmethods.net/kb/statinf.php Trochim, W. (2006). Survey research (Links to an external site.) . Research Methods Knowledge Base. Retrieved from http://www.socialresearchmethods.net/kb/survey.php Valenzuela, D., & Shrivastava, P. (n.d.).  Interview as a method for qualitative research (Links to an external site.)  [Presentation slides]. Retrieved from http://www.public.asu.edu/~kroel/www500/Interview%20Fri.pdf [email protected] (n.d.). Case studies (Links to an external site.) . Colorado State University. Retrieved from http://writing.colostate.edu/guides/guide.cfm?guideid=60 Required Resources Text Mertler, C. A. (2017).  Action research: Improving schools and empowering educators  (5th ed.) Sage Publications. · Chapter 5: Collecting Data Web Page Framework for 21st Century Learning (Links to an external site.) . (n.d.). Retrieved from http://www.p21.org/our-work/p21-framework · This website provides a comprehensive review of 21st-century teaching and learning and combines a focus on student outcomes with support systems that help students master skills they will need in the 21st century. This resource will be used to support your work on the discussions this week (i.e., Diversity Through 21st-Century Teaching and Learning) and this week’s assignment (i.e. Visual Interpretation). Accessibility Statement does not exist. Privacy Policy does not exist.   Recommended Resources Websites Edraw Max Pro (Links to an external site.) . (https://www.edrawsoft.com/EDrawMax.php) · A web based tool, Edraw Max enables students, teachers, and business professionals to reliably create and publish various kinds of diagrams to represent any ideas. This resource will be used to support your work on the discussions this week’s assignment (i.e. Visual Interpretation). Accessibility Statement does not exist. Privacy Policy (Links to an external site.) Smore (Links to an external site.) . (https://www.smore.com/) · A web-based tool to help create newsletters and flyers. This resource will be used to support your work on the discussions this week’s assignment (i.e. Visual Interpretation). Accessibility Statement does not exist. Privacy Policy (Links to an external site.) Go to top of page Action Research Improving Schools and Empowering Educators Fifth Edition Craig A. Mertler Arizona State University FOR INFORMATION: SAGE Publications, Inc. 2455 Teller Road Thousand Oaks, California 91320 E-mail: [email protected] SAGE Publications Ltd. 1 Oliver’s Yard 55 City Road London, EC1Y 1SP United Kingdom SAGE Publications India Pvt. Ltd. B 1/I 1 Mohan Cooperative Industrial Area Mathura Road, New Delhi 110 044 India SAGE Publications Asia-Pacific Pte. Ltd. 3 Church Street #10-04 Samsung Hub Singapore 049483 Copyright © 2017 by SAGE Publications, Inc. All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. Printed in the United States of America Names: Mertler, Craig A., author. Title: Action research : improving schools and empowering educators / Craig A. Mertler. Description: Fifth edition. | Thousand Oaks, California : SAGE Publications, [2016] | Includes bibliographical references and index. Identifiers: LCCN 2016007234 | ISBN 978-1-4833-8905-9 (pbk. : alk. paper) Subjects: LCSH: Action research in education. Classification: LCC LB1028.24 .M47 2016 | DDC 370.72—dc23 LC record available at http://lccn.loc.gov/2016007234 978-1-5063-8745-1 This book is printed on acid-free paper. Acquisitions Editor: Terri Accomazzo Development Editor: Jessica Miller eLearning Editor: Robert Higgins Editorial Assistant: Erik Helton Production Editor: Olivia Weber-Stenis Copy Editor: Erin Livingston Typesetter: C&M Digitals (P) Ltd. Proofreader: Ellen Brink Indexer: Jean Casalegno Cover Designer: Gail Buschman Marketing Manager: Ashlee Blunk Preface Purpose of the Text Most, if not all, graduate students in education—and, in particular, in-service teachers seeking graduate degrees—are required to complete a course in educational research methods. The majority of these methods courses provide a broad overview of educational research methods, designs, and techniques. However, I would argue that graduate-level research methods courses taught to in-service teachers could be more suitable provided the appropriate instructional materials were available to instructors and students, such that they focus on a classroom-based approach to research. Most research methods courses—and, therefore, their appropriate textbooks—follow the description that I offer above, in that they are “survey” courses (i.e., those that provide an overview of a variety of research methods). There are numerous texts on the market that meet this description. In contrast, there are relatively few books that focus specifically on action research as a methodology, and there are even fewer that do so with the target audience of practicing educators in mind. The purpose of this book is to introduce educators to the process of conducting their own classroom-based or school-based action research. Detailed but practical information describing each step of the cyclical, iterative process is presented in a sequential manner. Educators are provided with an overview of traditional educational research prior to examining action research as a mechanism for designing and conducting their own applied research projects. The focus is not on the theoretical aspects of educational research but rather on the practical facets of conducting applied classroom and/or school research. As presented in this textbook, action research is not simply a means of conducting applied research. It is also a mechanism for engaging educators in reflective practice and customizing professional development opportunities in order to capitalize on the unique interests of individual educators or teams of educators. The reason behind my desire to write a textbook on this topic is fairly straightforward. I have taught educational research methods for more than 20 years. The vast majority of the students enrolled in this course are in-service teachers seeking master’s degrees in various fields, including curriculum, teaching, administration, and counseling. This course is intended to serve as an overview of research methods used to conduct research in the broad field of education, focusing primarily on quantitative methods. The focus is on very formal methodological approaches, such as descriptive, correlational, causal-comparative, experimental, and quasiexperimental research methods. The educators enrolled in a research methods course—especially those who intend on remaining in the K–12 setting—typically experience substantial difficulty in being able to see the application of these formal methodological approaches in their educational settings. The bottom line is that they do not really need to understand the application of these approaches to conducting research, since the majority will likely never design or conduct such formal methodological procedures. It is my belief that this is the case for many graduate programs in education across the country. On the other hand, when we reach the topic of action research, the discussion typically piques student interest. The students can actually see how this methodological approach could be used in their schools, in their classrooms, with their students, and so on. Since action research is conducted by practitioners—yet still incorporates a good degree of rigor—students begin to see themselves designing and carrying out action research studies. By focusing our attention on a broad overview of research methods, I feel that we are doing an injustice to these practicing educators. We are not providing them with the tools necessary to design and conduct research studies that provide meaningful and immediate solutions to local-level problems. In other words, we are not adequately preparing them to investigate problems and seek solutions in their local settings and in a professional manner. The practical nature of the book stems from the fact that it focuses on research methods and procedures that teachers, administrators, counselors, intervention specialists, and so forth, can use in conjunction with their everyday instructional practices and activities in schools and classrooms. Educators are shown how to design and conduct school-based research in order to make their instructional practices more effective. The numerous examples—many of which are supplied by me, while others come from published action research studies—of the principles, procedures, and techniques discussed in the narrative make it easy for students to understand the material in this book. Theoretical aspects of research as well as highly technical concepts and procedures, which are unlikely to be used by practicing educators, are de-emphasized—producing a textbook that provides comprehensive coverage of action research methods for practicing educational professionals without being unnecessarily technical; that is, it is a practical book for educators. This book provides them with the knowledge and skills necessary to design research studies that seek solutions to local-level problems, conduct those studies, and communicate the results to local stakeholders and other interested parties. Although it is based on the research literature, the book takes a very practical approach, never losing sight of its intended audience—the practicing educator. Text Targets Graduate Students, Educators This book was written with graduate students as the primary target audience. Specifically, this audience includes but of course is not limited to K–12 classroom teachers, administrators, counselors, special educators, and intervention specialists. In all likelihood, this text would be used as the primary book for a graduate course in action research, although it could also serve as a supplemental text for other graduate-level courses not focusing on research methods (e.g., courses in curriculum, supervision). The book is appropriate for educators in all areas of education (e.g., elementary and secondary mathematics, science, social studies, languages, music, art, physical education, special education, administration, counseling, and special education); examples as well as sample articles throughout the book come from a variety of settings and situations. Text Organized Sequentially, Like an Action Research Study The main topics covered in the book pertain most closely to designing and conducting classroom-based applied research. These general topic areas—and the chapters where they are addressed in the book—include the following: · An overview of educational research (Chapter 1) · An overview of action research (Chapter 1) · The characteristics of action research (Chapter 1) · The action research process (Chapter 2) · Identifying an area of focus for action research (Chapter 3) · Reviewing related literature (Chapter 3) · Designing an action research study (Chapter 4) · Collecting and analyzing data (Chapters 5 and 6) · Developing an action plan (Chapter 7) · Writing an action research report (Chapter 8) · Sharing the results of an action research study (Chapter 9) · Reflecting on the process of action research (Chapter 9) The book is arranged in this manner because it presents, in sequential order, the process of designing and conducting an action research study—beginning with the development of the topic to be investigated, reviewing related research, designing the study, actually carrying out the procedures, developing an action plan, and ultimately sharing the results and reflecting on the process. It is, however, important to note that action research proceeds through this process in a cyclical manner. Pedagogical Features and Benefits to Students (as Well as Instructors) When compared with other action research books currently on the market, this book provides similar coverage of content. There are, however, several aspects that distinguish it from similar works. These aspects include the following: · Since the book takes an extremely applied approach, it includes numerous examples—not simple discussions or descriptions—of such things as data collection instruments (e.g., checklists, attitude surveys, interview protocols, and journal prompts) and presentation of research results (e.g., tables and graphs resulting from the analysis of quantitative data; summary tables resulting from the analysis of qualitative data; actual reports of action research). · In addition to the various narrative examples that appear throughout the book, two detailed case studies of action research, called “Action Research Portraits,” are developed in Chapter 1 and are extended in each subsequent chapter throughout the book. In each chapter, the case study discussions continue from the previous chapter, highlighting the application of content from the particular chapter as integrated into two practically based action research studies, one of which is conducted by an individual teacher (at the elementary level) and the other by a pair of teachers (at the high school level). In addition, a third “Action Research Portrait” appears on the book’s accompanying website (edge.sagepub.com/mertler5e). · Numerous online resources are available for teachers to use in order to learn more about action research, address questions that they may have about the process, or promote dissemination of their action research results. A section titled “Related Websites” is included near the end of each of the nine chapters. · Special sections, titled “Writing Up Action Research,” are also included in Chapters 3 through 7 and Chapter 9. These sections provide annotated excerpts from published or otherwise disseminated action research reports, highlighting specific concepts presented in each particular chapter. · On the first page of each chapter is a visual organizer for the main contents of that chapter. · Three appendices follow Chapter 9. The first two include complete written reports of teacher-conducted action research projects. We have also included additional complete written reports of teacher-conducted action research projects on the website (edge.sagepub.com/mertler5e). The third appendix contains developmental templates to guide the novice action researcher. These templates are also included on the website. · Each chapter includes a bulleted “Summary” of the main points included in the chapter. · Each chapter also includes a final section titled “Questions and Activities” that can be used to extend student knowledge, understanding, and application. · The text also includes a complete glossary of terms related to action research, a list of references used to compile the book, and a comprehensive subject and author index. Video Clip of Dr. Mertler discussing the new features of the 5th edition. New Features in the Fifth Edition There are several new features in the fifth edition of Action Research: Improving Schools and Empowering Educators: · The discussion of rigor in Chapter 1 has been enhanced. · Social justice advocacy has been added as an important application of action research. · The discussions of ethics in Chapter 2 and Chapter 3 have been enhanced. · The “Writing Up Action Research” sections that appear in Chapter 3 to 7 and Chapter 9 have been supplemented with call-out boxes highlighting the important aspects of each excerpt. · Substantially enhanced presentations of establishing the quality of both qualitative and quantitative data have been added to Chapter 5. · A discussion of the inclusion of abstracts has been incorporated into Chapter 8. · The developmental templates shown in Appendix C—in addition to being provided in an interactive, electronic format on the Student Study Site (edge.sagepub.com/mertler5e)—are now also available at TeachersPayTeachers.com, called the Action Research Mentor Portfolio. · Finally, two new complete action research reports have been added as Appendix A and Appendix B. Both reports are new to this edition. The complete written reports of teacher-conducted action research projects from all previous editions are available on the website that accompanies this book (edge.sagepub.com/mertler5e). Ancillary Material on the Web Open-Access Student Study Site:  edge.sagepub.com/mertler5e This web-based Student Study Site provides a variety of additional resources to enhance students’ understanding of the book’s content and take their learning one step further. The site includes the following: · Interactive PDF Action Research Developmental Templates are provided to assist and guide the novice action researcher through many of the steps and decisions in the process of designing and conducting original action research. · Video vignettes of the author and several practitioner-researchers discussing various aspects of conducting action research. These vignettes are integrated with specific chapter content throughout the book. · Web quizzes allow students to independently assess their progress in learning course material. · eFlashcards are study tools to reinforce student understanding and learning of key terms and concepts that are outlined in the chapters. · Chapter-specific PowerPoint presentations offer assistance by highlighting essential content, features, and artwork from the book. · A  Learning From SAGE Journal Articles  feature provides access to recent, relevant full-text articles from SAGE’s leading research journals. Each article supports and expands on the concepts presented in the chapter. · Carefully selected, web-based video resources feature relevant content for use in independent and classroom-based exploration of key topics. · Links to relevant web resources direct students to additional tools for further research on important chapter topics. · Sample Action Research Reports  are also included. A Note About Action Research Projects One concern that both instructors and students face is how to fit into one semester both the content coverage of the book and the completion of a student-conducted action research project. My advice is first to reinforce with students who are just learning about action research that the important aspect of an action research project as a course assignment is to become familiar with the process of designing and conducting action research and that they should worry less about the final product of their study. If they can become familiar and comfortable with the process as a whole, they will later be able to design and conduct larger-scale research projects that may require more time. For a typical 15-week academic term, I might suggest the following week-by-week activities, for both content coverage of the book and the associated action research project: For a typical 10-week academic term, I might suggest the following week-by-week activities: A Final Note for Students of Action Research I enjoy and value classroom-based/school-based action research because it has the potential to empower educators, to engage them directly in the process of educational improvement, and to provide a mechanism for customizing professional development. I will not mislead you into thinking that this is necessarily an easy road to travel. Learning how to conduct action research studies that will enhance your professional practice does, in fact, take time and practice. However, by gaining familiarity and experience with designing and conducting action research projects, you will, I trust, realize the substantial and positive professional, reflective outcomes of action research discussed repeatedly throughout this book. I sincerely wish you the best of luck in your action research endeavors! In the electronic edition of the book you have purchased, there are several icons that reference links (videos, journal articles) to additional content. Though the electronic edition links are not live, all content referenced may be accessed at edge.sagepub.com/mertler5e . This URL is referenced at several points throughout your electronic edition. Acknowledgments I would like to acknowledge the contributions of several individuals to this project. I would like to recognize and sincerely thank my editorial team at Sage Publications—namely, Terri Accomazzo (acquisitions editor). I’ve worked with Terri for many years on several projects and she amazes me with each successive endeavor. I would also like to thank Jessica Miller (associate editor) and Georgia McLaughlin (editorial assistant) for their support and timely responses to all of my questions. I would like to thank Olivia Weber-Stenis (production editor) and Erin Livingston (copy editor) for their assistance in working with me on the drafts and on the final appearance of the book. Thanks also to Robert Higgins (eLearning editor) for his assistance on the student study site ancillaries. Finally, I would like to thank Ashlee Blunk (marketing manager) and her staff for their continued support of the book. I offer my sincere thanks to those individuals who served as reviewers for this revised edition—their comments and feedback were greatly appreciated and extremely helpful: · Ronald Beebe, University of Houston–Downtown · Tyrone Bynoe, University of the Cumberlands · Stacy Hill, Whitworth University · Kimberly Livengood, Angelo State University · Gene Schwarting, Fontbonne University · Michelle Szpara, Cabrini College · Gay Ward, University of Wisconsin–River Falls Also, I would like to recognize and thank those individuals who provided helpful feedback on previous editions: · Katherine Egan Cunningham, Manhattanville College · Elizabeth Dore, Radford University · Gabrielle Kowalski, Cardinal Stritch University · Yoon-Joo Lee, CUNY System Office–New York · Ida Malian, Arizona State University · Darcy Miller, Washington State University · Barbara Taylor, Western New Mexico University · Robert Wolffe, Bradley University · Maryann Byrnes, University of Massachusetts Boston · John Huss, Northern Kentucky University · Terrence Stange, Marshall University · Tamara Walser, University of North Carolina at Wilmington · Eugene Bartoo, University of Tennessee at Chattanooga · Kevin Carr, George Fox University · Dana Fredebaugh, Nova Southeastern University · Terrance Jakubowski, California State University, Northridge · Maja Miskovic, National Louis University · Phillip Mutisya, North Carolina Central University · Cynthia Williams Resor, Eastern Kentucky University · Gail Ritchie, George Mason University · Margaret Waterman, Southeast Missouri State University · Lois McFadyen Christensen, University of Alabama at Birmingham · Christopher J. Della Pietra, Southeastern Louisiana University · Michael P. Grady, Saint Louis University · K. Fritz Leifeste, Angelo State University · Marilyn Lichtman, Virginia Tech · Jeanne M. McGlinn, University of North Carolina at Asheville · Jill C. Miels, Ball State University · Cathy Mogharreban, Southern Illinois University Carbondale · Ted J. Singletary, Boise State University · Shelley H. Xu, California State University, Long Beach As always, I would like to thank my wife, Kate, for her continued support of my extensive writing projects and for her feedback on numerous aspects of the book, from a classroom teacher’s perspective, and our son, Addison, for providing the invaluable student’s perspective. My books begin to take on a different level of meaning now that Addison is studying to become a professional educator himself. About the Author Craig A. Mertler has been an educator for 30 years, 20 of those in higher education. He is currently an Associate Professor and Director of the EdD Program in Leadership and Innovation at Arizona State University. He teaches doctoral courses focused on the application of action research to promote educator empowerment, school improvement, and job-embedded professional development and also teaches quantitative research methods, introductory statistical analysis, multivariate statistical analysis, and educational assessment methods. He is the author of 20 books, 4 invited book chapters, 18 refereed journal articles, two instructors’ manuals, and numerous nonrefereed articles and manuscripts. He has also presented more than 35 research papers at professional meetings around the country as well as internationally. He conducts workshops for in-service educational professionals (at all levels) on classroom-based action research and on the broad topic of classroom assessment. His primary research and consulting interests include classroom-based action research, data-driven educational decision making, professional learning communities, and classroom teachers’ assessment literacy. Before teaching and researching at the university level, he taught high school biology and earth science and also coached track and volleyball. In his leisure time, he enjoys traveling with his family and playing golf. Dr. Mertler can be reached at [email protected] or [email protected] for consulting, professional development, and speaking engagements. CHAPTER 5 Collecting Data Chapter 5  Organizer · Qualitative Data Collection Techniques · Observations · Interviews · Journals · Existing Documents and Records · Reflective Teaching · Characteristics of Qualitative Data: Accuracy, Credibility, and Dependability · Quantitative Data Collection Techniques · Surveys, Questionnaires, and Rating Scales · Checklists · Formative and Summative Classroom Assessments · Standardized Test Scores · Characteristics of Quantitative Data: Validity and Reliability · Notes About Ethics and Data Collection · Related Websites: Advice and Guidelines About Data Collection · Summary · ➤  Questions and Activities · ➤  Key Terms · ➤  Student Study Site In this chapter, we enter into the second stage—the acting stage—of conducting a classroom-based action research project. Recall that the acting stage is composed of data collection, which will be discussed in the present chapter, and data analysis, the topic for Chapter 6. As you will soon learn, there are numerous techniques that can be used to collect both qualitative and quantitative data for your teacher-led action research studies. 5.1 Qualitative Data Collection Techniques Recall that qualitative data are narrative; that is, the data themselves are words. These “words” may appear in the form of interview transcripts, observational notes, journal entries, or transcriptions of audio- or videotapes or as existing documents, records, or reports. They may be collected using a variety of techniques, but it is important to remember that the resulting qualitative data will always consist of descriptive, narrative accounts. Observations As human beings, we are constantly observing and taking note of the world around us. Furthermore, as teachers, we are constantly observing our students. However, on a daily basis, we typically observe our surroundings in somewhat of a haphazard manner—something more akin to “watching” than observing. Observations, as a means of collecting qualitative data, involve carefully watching and systematically recording what you see and hear going on in a particular setting (Schmuck, 1997). Observations can be extremely useful in certain situations where other forms of data collection simply will not work, such as when teachers want to check for students’ nonverbal reactions to something that is occurring in the classroom or when students are working in small groups in order to better understand how they interact and communicate with one another. Classroom observations can range from highly structured to semistructured to unstructured (Parsons & Brown, 2002; Schmuck, 1997). Structured observations typically require the observer to do nothing else but observe, looking usually for specific behaviors, reactions, or interactions. Because so many other things are going on in a given classroom when observations are being made, it is often difficult to conduct structured observations. Classroom-based action research should never be done at the expense of your teaching (Hubbard & Power, 2003; Johnson, 2008); it should be done only in order to enhance and inform your teaching. Unstructured or semistructured observations allow the practitioner-researcher the flexibility to attend to other events or activities occurring simultaneously in the classroom or to engage in brief but intense periods of observation and note taking (Hubbard & Power, 2003). In addition, unstructured observations are more typical of qualitative data collection, since they are “free flowing,” allowing the practitioner-researcher to shift focus from one event to another as new and perhaps more interesting events arise (Leedy & Ormrod, 2005). Schmuck (1997) discusses several advantages of conducting classroom observations. First, teachers can gather data about actual student behaviors, as opposed to asking students to report their perceptions or feelings. Second, this enables the teacher to see some things that students might not be able to report on themselves. Finally, as you will see shortly, such devices as videotape recorders allow teachers to “observe” even more than they would normally be able to with their own eyes. However, conducting observations also has its limitations (Schmuck, 1997). First, the simple presence of the teacher as a “data collector”—with notebook and pencil or perhaps a video recorder in hand—can change student behavior. There is a great potential for them to behave differently or to say different things if they know that they are being watched carefully. Second, in cases where specific behaviors are sought, and since behavior can be adversely affected because of the presence of an observer, the practitioner-researcher may have to wait for extended periods of time in order to observe the desired behavior. Still, the desired behavior may never occur, even if it is a normal, everyday occurrence. Finally, if teachers are working together on an action research project, different observers may see different things, even while observing the same event. Classroom observations are usually recorded in the form of field notes. Field notes are written observations of what you see taking place in your classroom (Johnson, 2008). It can sometimes be overwhelming to try to record everything that you see, especially when trying to determine what is important (and, therefore, worth recording) and what is not. Johnson (2008) advises practitioner-researchers to simply “stop thinking and just write what you see” (p. 83). As you observe and record what you see, you will undoubtedly begin to focus on things that are interesting or important. As you make observations over time, patterns will begin to emerge from the data you have collected. When recording field notes, you may want to consider dividing each page of your notebook into two columns. You should use the left column for recording your actual observations and the right column for noting preliminary interpretations of what has been observed (Leedy & Ormrod, 2005). Bogdan and Biklen (2007) refer to these interpretations as observer’s comments or OCs. Observer’s comments often shed light on the emerging patterns from your observational data. Including observer’s comments in your observation notes is also one way to integrate reflection into the process of action research. The separation of these two types of commentaries is critical so that actual observations are not confused with what you think the observed event means. Teachers conducting action research studies need to remain as objective as possible in the records kept and data collected. As an aside, this need for objectivity also dictates that you not censor what you record in your notes with your “teacher’s eyes”—do not hesitate to record something even if it reflects negatively on your teaching (Hubbard & Power, 2003); after all, you are trying to learn about and improve your professional practice. In addition, interpretations of observations may change over time as you collect more data; having a record of these changing interpretations can be invaluable over the course of your study. An example of a page from a book of field notes that I recorded several years ago during a study of positive reinforcement in a preschool setting, depicting this two-column format of actual observations and associated observer’s comments, is shown in Figure 5.1. Figure 5.1 A Sample Fieldnote Page, the Left Column Showing Actual Observations and the Right Column Showing Preliminary Interpretations Written field notes can become problematic, however. They are often insufficient to depict the richness and the details of what one is observing (Leedy & Ormrod, 2005). Videotapes can provide assistance as a tool for recording observations, although they are not without their respective limitations, as well. Background noises may prevent you from hearing that on which you were hoping to focus your videotaped observation. Furthermore, video cameras can only capture what is happening in a given direction (i.e., the direction the camera is facing). Leedy and Ormrod (2005) suggest that, prior to beginning any formal observations, researchers should experiment and become familiar with various methods of recording observations in order to find what works best for the particular setting and situation. It is, however, important to remember that whatever mechanism you use to record your observations, you simply cannot physically record everything that you see or that is happening (Mills, 2011); it is best not to put pressure on yourself to try to do so. On a practical note, several tips may facilitate your observations and the development of your observation skills. If you decide to observe and to record those observations using field notes, you may want to consider carrying a clipboard or legal pad with you for several days prior to beginning your observations and recording any field notes. It is important that the act of recording field notes becomes a part of your daily routine, as opposed to something that “feels” unfamiliar, extraneous, or irrelevant. Similarly, if you decide that you will record your observations through the use of a video camera, you may want to set up the camera several days in advance of your recording. This is important because both you and your students or other participants will be more comfortable being videotaped if you and they are accustomed to seeing the camera in the classroom. Again, it becomes part of the daily routine or setting. Interviews An alternative to observing people is to directly ask them questions. This can be accomplished in several ways. Interviews are conversations between the practitioner-researcher and participants in the study in which the teacher poses questions to the participant (Schmuck, 1997). Interviews can be conducted with individuals or with groups. It is best to prepare an interview guide, containing either specific or general questions to be asked prior to conducting any interviews. Similar to observations, interviews are typically classified as being structured, semistructured, or open-ended. In a structured interview, the researcher begins with an interview guide consisting of a specific set of predetermined questions. Those questions—and only those questions—are asked of each person being interviewed. This is typically done for the sake of consistency. Interestingly, consistency is usually not a concern when collecting qualitative data; it is typically more desirable for the researcher to have some flexibility and to be able to ask clarifying questions (not initially included on the interview guide), to pursue information not initially planned for, and to seek different information from different people (Leedy & Ormrod, 2005). When gathering truly qualitative data, interviews are probably best conducted following semistructured or open-ended formats. In semistructured interviews, the researcher asks several “base” questions but also has the option of following up a given response with alternative, optional questions that may or may not be used by the researcher, depending on the situation. When developing interview guides, it is best to keep your questions brief, clear, and stated in simple language (Johnson, 2008; Schwalbach, 2003). For example, if we were interviewing students regarding their opinions of our school, we might ask the following questions, where the italicized questions represent the optional, follow-up, probing questions: · What do you enjoy most about this school? · Why do you enjoy that aspect so much? · Do you think other schools have this particular benefit? · What is your favorite academic subject? · Why is that your favorite subject? · Do you have any others? · What about extracurricular activities? Are there any that you participate in? · Which are your favorites? Why? · What do you like least about this school? · Why do you like that so little? · Is there anything that the principal or teachers could do to improve that aspect? The semistructured interview guide that I used in my positive reinforcement study is shown in Figure 5.2, and a portion of the transcript from one interview that I conducted is shown in Figure 5.3. Open-ended interviews provide the respondent with only a few questions, very broad in their nature. The intent is to gather very different kinds of information from different individuals, depending largely on how each interprets the questions. For example, an open-ended series of interview questions about school climate might include the following: · What does “school” mean to you? · What do you like about school? · What do you dislike? Figure 5.2 A Sample Semistructured Interview Guide Figure 5.3 Portion of a Transcript From a Semistructured Interview, Using the Guide Shown in Figure 5.2 As mentioned earlier, interviews are conducted not only with individuals but also with groups. A focus group is the name given to simultaneous interviews of people making up a relatively small group, usually no more than 10 to 12 people (Leedy & Ormrod, 2005). This type of interview typically lasts between 1 and 2 hours. Focus groups are especially useful when time is limited and because people often are more comfortable talking in a small group, as opposed to individually. Furthermore, interactions among the focus group participants may be extremely informative because of the tendency for people to feed off others’ comments. However, when conducting a focus group interview, it is important to ensure that each participant is provided with the opportunity to speak and share her or his perspective (Mills, 2011). There can be a tendency for one or two individuals to dominate the discussion; it is the responsibility of the practitioner-researcher to closely monitor the discussion in order to prevent this from happening. The set of guiding questions I used for a study incorporating data collected via a focus group is provided in Figure 5.4. Upon completion of the above questions, explain to the participants that the meeting is about to end. Ask them to take a moment and think about what has been discussed. Then, one by one, ask them if they have any additional comments. If necessary, explore relevant or new comments in greater depth. Figure 5.4 Sample of Guiding Questions Used for a Focus Group Interview Qualitative data may also be collected via the use of e-mail interviews (Mills, 2011). With schools becoming increasingly networked, practitioner-researchers can easily collect data from colleagues, parents, and students by sending out a series of questions in an e-mail message. One benefit of doing so is that when the respondent replies to your e-mail questions, the transcription of the interview has already been done for you. However, you must be cautious of possible ethical complications and realize that e-mail responses are not necessarily anonymous or confidential (Mills, 2011). Other individuals who may have access to a server may be able to intercept e-mail responses from targeted respondents. Hubbard and Power (2003) also remind practitioner-researchers not to forget about the value of informal interviews—that is, those that are spontaneous, that take place throughout the data collection process, and that are typically part of the daily interactions with students in a classroom setting. Teachers are constantly asking students questions, trying to gather various types of information from them. Schmuck (1997) provides a discussion of the relative advantages and limitations of conducting interviews as part of action research studies. Advantages include the fact that interviews permit the practitioner-researcher to probe further and ask for clarification in a participant’s response to a given question. In addition, data can be collected—and, therefore, preserved—through the use of audio- and videotapes, although you want to be sure that individuals being interviewed are not made to feel uncomfortable by the presence of an audio or video recorder. Finally, for respondents who cannot or who are unwilling to share their thoughts, feelings, or perceptions in writing, sitting down and carrying on a conversation with them is often a reasonable alternative. On the other hand, interviews can be extremely time-consuming. Not only does it take time to collect data from individuals during a verbal conversation; before the data can be analyzed, the interviews must be transcribed so that the responses can be read and processed. The general rule of thumb that I learned in my graduate school days is that for every hour of audiotaped interview, you can expect approximately 8 to 9 hours of transcription work, depending on the quality of the recording. Other limitations of interviews include the fact that respondents are not able to retain their anonymity. Many people are simply uncomfortable with a tape recorder lying on the table between them and the interviewer. Finally, respondents often fear that something they have said may be used against them at some point in the future. An additional responsibility of the practitioner-researcher is to put the mind of the interviewee at ease about such possibilities. Journals Data journals may be kept by both teachers and students and provide valuable information on the workings of a classroom (Mills, 2011). In a way, student journals provide information similar to homework to the teacher, in that teachers can gain a sense of students’ daily thoughts, perceptions, and experiences in the classroom. Teacher journals can similarly provide practitioner-researchers with the opportunity to maintain narrative accounts of their professional reflections on practice. They truly become an ongoing attempt by teachers “to systematically reflect on their practice by constructing a narrative that honors the unique and powerful voice of the teachers’ language” (Mills, 2011, p. 86) by reflecting not only observations but also the feelings and interpretations associated with those observations. Class journals are another means of incorporating journaling into your action research data collection. A class journal is a less formal version of a student journal. Johnson (2008) suggests that a blank notebook be passed around the class on a periodic basis or put in a learning center for an extended amount of time. Students are encouraged to enter their thoughts, ideas, perceptions, feedback, or other forms of response, such as pictures or diagrams, as they wish. Teachers may want to provide some sort of guidelines for making entries into the class journal so that it does not become a “quasi-teacher-approved” form of graffiti that may be offensive to other students (Johnson, 2008). Existing Documents and Records Often, action research necessitates the gathering of data that already exist. Schools and school districts are veritable treasure troves of existing data. These data are essentially anything collected for a reason other than the action research study but are now being used as data for the study. These existing documents and records might take several forms, including (at the individual student level) curriculum materials, textbooks, instructional manipulatives, attendance records, test scores, previous grades, discipline records, cumulative folders, and (at the school or district level) attendance rates, retention rates, graduation rates, newspaper stories about school events, minutes from faculty or school board meetings, and standardized test scores perhaps disaggregated by grade level, gender, or ethnicity (Johnson, 2008; Mills, 2011; Schmuck, 1997). These various sources and types of data tend to be very under-utilized, especially for purposes of conducting action research. However, a word of caution is in order: Whenever using existing data, it is critical to make sure to follow your school district’s approved procedures for securing access to these various types of data and that you use and report the results of any analyses in an ethical manner (Johnson, 2008). Collecting existing data on students can sometimes become overwhelming—there may be so much information you want to collect. The dilemma often faced by practitioner-researchers is how to organize that information. A nice organizational tool that can aid in your organizational efforts is to record data on a common data form. In other words, you develop for your specific purposes and use a single form as a means of compiling various types of information, as opposed to having a conglomeration of loose papers stuffed in a file folder, for example. Several years ago, I was involved in a study that examined student attendance records, reasons for school absences, discipline referrals, and referrals for special programs and social services. That was a good deal of information to collect on each student. However, I designed a data collection form (see Figure 5.5) that allowed us to collect this wide variety of information on a single form, thus organizing and encapsulating it at the same time. Video Clip 5.1 View a clip of Dr. Mertler discussing important considerations for data collection. One possible type of existing “data” that should not be overlooked is that of classroom artifacts. Classroom artifacts include any written or visual sources of data, contained within the classroom, that contribute to our understanding of what is occurring in classrooms and schools (Mills, 2011). This source of existing data primarily incorporates work done by students as part of their schoolwork but is now used as data for action research purposes (Hubbard & Power, 2003). Such items as student portfolios or products resulting from the administration of a performance-based assessment or such less formal artifacts as students’ responses to a teacher’s request to explain the solution to a constructed-response mathematics problem can serve as prime examples of student-produced classroom artifacts. Reflective Teaching A final source of (usually) qualitative data is a process or instrumentation for examining your own teaching. Reflective practice is a huge theme that undergirds the action research process, so it only makes sense that it can also provide data the for action research cycle. Instrumentation that might help an educator more closely examine his or her actual practice can be immensely helpful in diagnosing areas in need of improvement when it comes to classroom instruction. These types of reflective data can be collected through self-evaluations, peer observations, daily journals or diaries (either individual or collaborative), or videotaping actual lessons. When reflecting on your own teaching, it is important to factor in three aspects: (1) the actual event or lesson, (2) the recollection of the event or lesson, and (3) reviewing and responding to what actually occurred during the event or lesson. While all steps are important, this third step is crucial because it is where any follow-up action takes place. Figure 5.5 Sample of a Data Collection Form for Existing Student Data Characteristics of Qualitative Data: Accuracy, Credibility, and Dependability When collecting data for action research studies, it is important for practitioner-researchers to ensure the quality of their data. If data collected for the study are imprecise or if the researcher has actually measured something other than what was intended to be measured, at a minimum the data will be inaccurate and misleading. The larger concern here is that if the action research study is continued to its logical “end,” the results of the study will follow suit: They too will be inaccurate and misleading. If that occurs, you have essentially wasted your time—not to mention that of your colleagues, students, parents, and anyone else involved in your study. Validity of research data deals with the extent to which the data that have been collected accurately measure what they purport to measure (i.e., that which we intended to measure; Mills, 2011). When dealing with the validity of qualitative data, researchers are essentially concerned with the trustworthiness—for example, the accuracy and believability—of the data. Trustworthiness is established by examining the credibility, transferability, dependability, and confirmability of qualitative data. Gay, Mills, and Airasian (2009) and Trochim (2002c) have described each of these characteristics of qualitative data as follows: · Credibility involves establishing that the results of qualitative research are credible, or believable, from the perspective(s) of various participant(s) in the research. A researcher must consider and incorporate into data collection and analysis all of the complexities of the study and address issues that are not easily explained. · Transferability involves the provision of descriptive and contextualized statements so that someone reading the study can easily identify with the setting. Remember, in qualitative studies—as well as in action research—the goal is not to generalize findings to other settings but instead to have a clear and in-depth understanding of this particular setting. · Dependability emphasizes the need for the researcher to account for the ever-changing context within which qualitative research occurs. The researcher is responsible for describing the changes that occur in the setting and how these changes affected the way the researcher approached the study; this helps ensure the data are stable over time. · Confirmability is a process of establishing the neutrality and objectivity of the data. Maxwell (as cited in Gay et al., 2009) has provided criteria for establishing the validity of qualitative research, using concrete actions that can be taken by the researcher. These five criteria, if incorporated into any research study utilizing qualitative data, can contribute to the overall trustworthiness of the data and research (Mertler, 2016). 1. Descriptive validity—This is the factual accuracy of the account provided in the research. Researchers must be sure not to distort, manipulate, or fabricate events based on inferences. 2. Interpretive validity—This refers to the accuracy of the interpretations of participants’ behaviors and words and concern that their perspectives are accurately represented. 3. Theoretical validity—This concerns the extent to which this study, and its final report, relate the phenomenon being studied to a broader theory. 4. Evaluative validity—This refers to the extent to which the researcher behaved objectively enough to report the data and findings in an unbiased manner, without making evaluations or judgments of the collected data. 5. Generalizability—This has to do with the extent to which the findings can be extended and are applicable both within the community that was studied, as well as to settings that were not studied by the researcher. There are three common practices, typical aspects of any qualitative research study, that can help ensure the trustworthiness of your data. A key aspect of these strategies requires practitioner-researchers to continually check their perceptions to ensure they are not being misinformed and what they think they are seeing and hearing are actually what they are seeing and hearing (Fraenkel, Wallen, & Hyun, 2012). These practices include the following strategies [as adapted from Fraenkel et al. (2012); Gay et al. (2009); and Mertler (2016)]. · Use a variety of instruments, methods, and sources to collect data. Research findings that are supported through the use of multiple methods and sources of data collection will only serve to enhance the validity of those findings. This process of using multiple methods, data collection strategies, data sources—and perhaps even multiple researchers (Glesne, 2006)—is known as triangulation. Triangulation is an inherent component of mixed-methods research designs, which—as you learned in Chapter 4—tend to be more closely aligned with action research studies than qualitative or quantitative data alone (Ivankova, 2015). A given finding is supported by integrating inferences and demonstrating that independent measures of it tend to agree with each other or at least do not directly contradict each other (Hubbard & Power, 2003; Ivankova, 2015). For example, when you observe Susan actually doing something she has told you in an interview she does, and that is also indicated on an open-ended questionnaire (see Figure 5.6), you likely will have more confidence in concluding that it is probably an accurate depiction of Susan’s practice. In other words, your interview data have been supported by your observation data and by the questionnaire responses. Had any of the three sources of data contradicted each other, you likely would have arrived at a different conclusion, perhaps that Susan was telling you what you wanted to hear, although in reality she did not practice it. · Engage in persistent and prolonged participation at the study site. The length of time you spend observing, interviewing, and participating in the site is critical in research using qualitative data. The more time you spend, the more likely you will be able to determine what is typical, atypical, and/or problematic within the setting. The idea here is that you spend a substantial amount of time “in the field,” so to speak; the more time spent productively observing, and interacting and engaging with the participants, the more you are able to develop trust with and get to know your participants, learn the culture of their setting (whether it be a classroom or school building), and observe patterns of behavior to the point of being routine (Glesne, 2006). Observing or interviewing only once or twice will not afford you this luxury. Figure 5.6 Triangulation of Three Sources of Data · Conduct member checks. Member checking is a process of asking participants who were directly involved in the study to review the accuracy of the research report. This procedure involves the sharing of interview transcripts, analytical thoughts (such as observation notes with observer’s comments), and drafts with the participants of the study. The purpose of sharing these data sources is to make sure you have represented your participants and their ideas accurately (Glesne, 2006). · Use peer debriefing and external audits as mechanisms for verifying your processes. Peer debriefing is the act of using other professionals (perhaps a colleague or a critical friend) who can help you reflect on the research by reviewing and critiquing your processes of data collection, analysis, and interpretation. An external audit involves the use of an outside individual (again, a colleague, critical friend, etc.) to review and evaluate the final research report. Both of these strategies provide opportunities for different sets of eyes—as well as additional minds—to review and evaluate the research process, which will likely enhance its overall credibility. · Develop detailed descriptions and notes, and engage in reflexivity. Notes from observations and interviews should be as detailed as possible. In addition, making use of observer’s comments during data collection to document your initial interpretations, assumptions, or biases will be critical in the latter stages of the research study. The process of integrating your own preliminary thoughts and interpretations with your actual observation notes is known as reflexivity. · Analyze negative cases. Sometimes cases (i.e., people or places) do not fit the pattern that you have observed up to a certain point in the study. Instead of ignoring these negative cases, you should attempt to eliminate the fact that they are negative cases by revising the pattern until these cases fit better or more appropriately. Like many of the above techniques, this is sometimes facilitated through prolonged time spent in the setting. 5.2 Quantitative Data Collection Techniques In contrast to qualitative data, quantitative data are numerical. Anything that can be quantified (i.e., counted, calculated, tallied, and rated) can be considered quantitative data. This includes not only items that can be counted but also ratings of one’s feelings, attitudes, interests, or perceptions on some sort of numerical scale. Quantitative data collection techniques include surveys, questionnaires, checklists, and rating scales as well as tests and other more formal types of measurement instruments. Generally speaking, quantitative data collection techniques are more efficient, in that you can collect data from numerous individuals simultaneously. However, the depth of those data does not begin to compare to that resulting from the use of qualitative techniques. Surveys, Questionnaires, and Rating Scales The term survey refers to a collective group of quantitative data collection techniques that involve the administration of a set of questions or statements to a sample of people. Surveys may be administered verbally—which then make them a type of interview—although the resulting data are numerical instead of narrative or in written form. Surveys that are administered in written form, where the researcher asks participants to answer a series of questions or respond to a series of statements and then return their responses to the researcher, are known specifically as questionnaires. Surveys and questionnaires permit the practitioner-researcher to gather a lot of—as well as a variety of—information relatively quickly (Johnson, 2008). There is not much that limits your use of surveys and questionnaires. They may be simple or complex; they can be composed of open-ended questions (where individuals provide their own responses) or of closed-response rating scales (where individuals simply select their response from a set of options provided to them). They may even contain a combination of the two types of questions or statements. Analysis of responses to open-ended items—which are, admittedly, more qualitative in design than are closed-response items—may often reveal unexpected thoughts and feelings from students, the likes of which you may not have been able to anticipate in order to develop closed-response items to address those particular thoughts and feelings (Schmuck, 1997). A closed-response question or statement provides the respondent with a number of choices from which to select. For example, consider the following item: What is your favorite subject in school? · English · Mathematics · Science · Social studies Students would be instructed to select one of the four possible responses. This type of question is easily quantifiable; you simply count the number of students who select each option. Furthermore, it is relatively easy to report the “results” of this item. You might summarize your data and conclude the following: · Science = 35% · Math = 25% · English = 25% · Social studies = 15% It is important to realize that this type of question may be misleading or controlling (Johnson, 2008). If in our example, the favorite subject of a given respondent is a foreign language class, how is that person supposed to respond to the question? Any option that person selects will actually provide inaccurate information. One alternative is to anticipate such an occurrence by revising the item to read as follows: What is your favorite subject in school? · English · Mathematics · Science · Social studies · Other: Open-ended items allow the respondents to provide a seemingly limitless number of responses. For example, we could have reworded our “favorite subject” question as an open-ended question by simply asking: What is your favorite subject in school? Here we might get a wide variety of responses. It is then the responsibility of the researcher to “analyze” the resulting data by grouping similar items together and then tallying the number of responses in each category. The result might look like this: · Science = 26% · English = 25% · Geometry = 15% · Social studies = 15% · Math = 10% · Art = 5% · Physical education or health = 2% · “I don’t know” or “I don’t have a favorite subject” = 2% Obviously, this form of the question provides a more accurate sense of what students really like. The only problem associated with asking open-ended items like this is that you have the sometimes messy task of grouping responses into similar categories before you can count the responses (Johnson, 2008). The main difference between a survey or questionnaire and a rating scale is that surveys are more appropriate for content-based types of questions (similar to our example above), whereas rating scales are appropriate when asking individuals to respond to a set of questions where their response indicates the strength (e.g., the extent of agreement, level of frequency, degree of understanding) of that response (Johnson, 2008). Rating scales can be used very effectively to measure students’ attitudes, perceptions, or behaviors. There are two main types of scales that appear in items on a rating scale: Likert and Likert-type scales. A Likert (pronounced “lick-ert”) scale begins with a statement and then asks individuals to respond on an agree/disagree continuum. The Likert scale typically ranges from strongly agree to strongly disagree. I typically recommend using a 5-point scale, with the 5 points defined as follows: · 1 = strongly disagree · 2 = disagree · 3 = no opinion · 4 = agree · 5 = strongly agree There tends to be quite a bit of disagreement among those with expertise in conducting research through the use of surveys regarding the appropriateness of including a neutral point on a scale. By including it, you allow your respondents to indicate that they truly are neutral or have no opinion, if in fact that is the case for them. However, if provided with a neutral option, there is a tendency for people not to think much about how they truly feel; they simply select the neutral option, which may not represent their true belief (i.e., the data they provide are inaccurate). On the other hand, if individuals truly are indifferent or have no opinion and you do not provide this option—because you are operating under the assumption that no one is truly neutral about anything—you “force” them to choose something that they do not really believe, thus providing inaccurate data once again. There is no right or wrong when it comes to deciding on the inclusion of a neutral point on your rating scale. However, you should consider the implications of both including and excluding such a point and then design your scale accordingly. Figure 5.7 presents a portion of a rating scale that I used in a study that focused on students providing their teachers with feedback on their classroom teaching. Notice the format of the Likert-scaled items. Also notice that a higher number corresponds to a higher level of agreement with a given statement. A similar type of scale is a Likert-type scale. This type of scale also exists on a continuum, but something other than extent of agreement is being measured. For example, a Likert-type item might require participants to respond on a scale that examines quality (“excellent . . . poor”), frequency of occurrence (“always . . . never”), or level of comfort (“very comfortable . . . not at all comfortable”; Mertler & Charles, 2011). An example of a Likert-type scale, used in a study of prekindergarten-to-kindergarten transitions, is shown in Figure 5.8. I want to mention one more thing about using surveys and rating scales with students. Practitioner-researchers need to be sure that the various aspects—not just the reading level—of the instrument are appropriate for the age or grade level of students. Although I recommended earlier that a 5-point scale is typically appropriate, one could see how that might create difficulties for young children—they obviously would not be able to discriminate between adjacent points on the scale. However, do not shy away from using such data collection instruments with younger children. You would likely provide fewer options on the scale and perhaps even use graphics for the children to respond to. Several years ago, I was part of a research team that attempted to “survey” kindergarten students as part of the prekindergarten-to-kindergarten transitions study. We had the teachers read the statements to the children and then asked them to put an X through the face that represented how they felt (see Figure 5.9). Figure 5.7 Portion of a Rating Scale Instrument Depicting a Likert Scale Figure 5.8 Portion of a Rating Scale Instrument Depicting a Likert-Type Scale Unfortunately, the children had no idea—and our explanations did not help at all—what the numbers were for. They were instructed to locate the number “1” on their response sheet, as the teacher read the first statement number, and then place their X on the appropriate face. After the first few statements, we realized that they were simply placing the X over the same faces in the first row. Several of the children had response sheets that looked like the image below left. Obviously, you can see the problems that this created with respect to the accuracy of our data! On the spur of the moment, we decided to revise the nature of the response sheet and came up with what you see in Figure 5.10. Using this format, we could direct the children’s attention to the box with a certain image in it and have them place their response only in that box. Figure 5.9 Rating Scale Instrument Used With Young Children Figure 5.10 Revised Version of the Instrument Presented in Figure 5.9 The Internet has helped facilitate the collection of data through the use of surveys and other types of self-reporting response systems. There is a wide variety of online survey websites, such as SurveyMonkey (http://www.surveymonkey.com), Checkbox (http://www.checkbox.com), Zoomerang (http://www.zoomerang.com), and Poll Everywhere (http://www.polleverywhere.com). Many of these are very affordably priced and allow you easy access to survey design and administration. Many of them also perform varying degrees of data analysis upon completion of data collection. The advantages of surveys and rating scales include the fact that they are very effective at gathering data concerning students’ attitudes, perceptions, or opinions. They are essentially written versions of structured interview guides, where individuals respond to a specific set of questions in writing, as opposed to responding orally. Rating scales and other closed-response items can be answered, and the responses can be tallied or counted quickly. Integrating the use of computer software can make this process of tallying even quicker. There are, of course, also limitations to the use of surveys for action research projects. Analyzing responses to open-ended items can sometimes be time-consuming, because of the fact that responses may be ambiguous (Schmuck, 1997). This limitation can be overcome by replacing open-ended items with rating scales or other closed-response items. Another limitation is that if the practitioner-researcher is not clear about an individual response, there is no opportunity or mechanism for asking respondents to clarify their answers, as with interviews. At this point, I would like to offer several suggestions—adapted from several sources (Johnson, 2008; Mills, 2011; Schmuck, 1997; Schwalbach, 2003)—regarding the development and use of surveys and rating scales as means of collecting action research data. When developing a new instrument, it is important to apply the following: · Each item should focus on a single idea or concept. · Do not use too many questions or questions that are not necessary or are repetitive. · Keep the length of the survey brief and the reading level relatively easy; failing to do so often results in respondents not completing the instrument or providing you with inaccurate information. · If you are designing a rating scale, keep the response scale consistent throughout the survey. Otherwise, respondents can become confused or may provide you with inaccurate data. · Consider using both closed-response and open-ended items, in order to realize the benefits of both. · Do not use leading questions; a good survey or rating scale is one that contains objective items. · Always proofread your survey—and perhaps have someone else proofread it too—before you administer it to your participants. Video Clips 5.2 & 5.3 View clips of practitioner-researchers discussing data collection. Checklists A checklist is a list of behaviors, characteristics, skills, or other entities that a researcher is interested in investigating (Johnson, 2008; Leedy & Ormrod, 2005). The primary difference between a checklist and a survey or rating scale is that checklists present only a dichotomous set of response options, as opposed to some sort of continuum. Instead of indicating the extent, degree, or amount of something, checklists enable the practitioner-researcher to indicate simply if the behavior or characteristic is observed or present or if it is not observed or present. Checklists are quicker for the practitioner-researcher to use than are surveys and rating scales; however, they provide data that are not nearly as detailed as those resulting from the use of rating scales. If you are observing students of any age and are using a checklist to record behaviors, you will want to keep the list of behaviors or characteristics to a manageable number. Otherwise, you may become overwhelmed with the sheer volume of things you must observe and record on the checklist. A sample student checklist is presented in Figure 5.11. Formative and Summative Classroom Assessments When the focus of your action research study is on student learning, multiple sources of data can be utilized to help you answer your research questions. Many of these data are types that are routinely used in the day-to-day process of providing instruction to students and then assessing their mastery of those instructional objectives or units. Formative classroom assessments are those assessments that are administered during instruction, for the purposes of determining what sort of adjustments should be made to that instruction while it is still ongoing (Mertler, 2003). Formative assessments primarily include those that are informal and often administered spontaneously, such as observations, oral questioning, and student reflections. Figure 5.11 Sample Student Checklist Looking at Independent Reading at the Elementary Level Summative classroom assessments are those assessments that are administered after a substantial period of instruction (e.g., following completion of an instructional unit, end of a semester, end of a course), for the purposes of making administrative decisions, such as assigning final grades or simply providing a more broad overview of student achievement (Mertler, 2003). Summative assessments are more formal and their administration is usually scheduled in advance. Included here would be chapter tests, unit tests, end-of-course tests, major performance-based projects (e.g., research reports, research-based presentations)—typically any assessment that would measure student achievement and span numerous instructional objectives and/or skills. Other formal assessment instruments that might be included here are scores on homework assignments and quizzes and final semester or course grades. If they are included as data in your action research study, it is important that they not be the only source of data (Johnson, 2008). The advantage of using formative and summative assessments as sources of data in action research studies is that they are typically another type of “existing” data—not at the school or district level but rather at the teacher or classroom level—in that they are routinely administered during the teaching-learning process. Therefore, incorporating these sources of data can facilitate conducting the study and make it somewhat more feasible, since designing data collection instruments specifically and only for the study is not necessary. However, the disadvantage of using these data is the potential mismatch with the goals of your study. The conscientious and professional practitioner-researcher must ensure that these instruments will provide the data specifically required to answer the research questions stated at the outset of the study. Otherwise, these data—from the perspective of the action research study—are potentially meaningless. In addition, there may be instances—depending on the nature of the questions and design of your action research study—where you might design a pretest and posttest specifically to measure changes that (hopefully) occur as a result of exposing students to some sort of instructional “treatment” or intervention. In this case, these may not be considered as existing data, since you designed and administered them with a specific research-related purpose in mind. They do, however, remain an important source of quantitative data. Standardized Test Scores Standardized test scores can also be used as sources of quantitative data. It is important to realize that these “formal” data collection instruments would also be considered “existing records,” since they are administered to students as a regular part of classroom instruction and district-level accountability. Again, if these types of data are to be included in your action research study, it is advisable that they not be the only source of data; they can be supplemented with more “local” sources of data, such as observations, self-developed surveys, or other forms of existing data. People in general, but especially educators, tend to have strong opinions when it comes to standardized tests. Regardless of how you might feel about them, they are a source of data that can be used to help answer a research question or address an instructional problem with which you might be struggling. I have argued for many years that since we are required to administer these types of assessments to our students and since we receive their scores (usually both as individual scores and as aggregated scores), why not find a productive way to use them to help us make better-informed decisions about our instruction, curriculum, assessments, and so forth. The following quote summarizes my sentiments: I honestly don’t know anyone who loves standardized testing! But the standardized testing movement is not going away anytime soon. An examination of its impact on this country’s educational system over the past 40 years will confirm that. Therefore I approach it from this perspective . . . and I strongly suggest that all professional educators adopt a similar attitude. Anytime we are given the responsibility of making decisions about children, we need as much information as possible in order for those decisions to be as accurate as possible. We ask students questions; we ask them to read to us; we require them to write for us; we test them over units of instruction; we observe them; we encourage them to be creative; we engage them in performance based tasks; etc. The results from standardized tests are just another source of information—about student learning, about our teaching, and about our curriculum. Please use them as such—add them to your long list of various sorts of information about student learning. They can only help improve the accuracy of the decisions that we make about our students, as well as our own instruction. (Mertler, 2007, p. xii) A Planning for Data Collection template appears in Appendix C and can also be found as an interactive PDF on the Student Study Site. Characteristics of Quantitative Data: Validity and Reliability Validity is an essential quality in quantitative research data and has to do with whether the data are, in fact, what they are believed or purported to be—in other words, did we actually measure what we intended to measure, based on the focus of our research? Though any data that you might collect may be entirely accurate, the critical factor is whether or not it is appropriate and accurate for your purposes (i.e., does it enable you to accurately answer your research questions?). For example, imagine that a reading teacher uses the results from the reading portion of a standardized test to group her students into above-average, average, and below-average reading groups. Then imagine that a social studies teacher uses those same reading scores to identify students who he believes would be successful in an advanced placement history course. The first interpretation and use of the scores is valid; the second is not. In terms of the social studies teacher’s use of the data, it was invalid for the purpose for which they were used. The determination of the validity of data ultimately has a substantial effect on the interpretation of those data, once they have been analyzed, and the subsequent conclusions drawn from those results (Mertler & Charles, 2011). Currently, validity is seen as a unitary concept (AERA, APA, & NCME, 1999), combining that which has been previously described as four distinct types of validity: content, concurrent, predictive, and construct. It is defined as the “degree to which all the accumulated evidence supports the intended interpretation of test scores for the proposed purpose” (p. 11). Validity of quantitative data can be determined through the examination of various sources of evidence of validity. Although similar to the four outdated types of validity, the five sources of validity evidence are unique in their own right (Mertler, 2016). These five sources of evidence of validity are as follows: · Evidence of validity based on test (or instrument) content—This is evidence based on the relationship between the content of a test (or other data collection instrument) and the underlying characteristic it is trying to measure. · Evidence of validity based on response processes—This source of validity evidence often results from the analysis of individual patterns of responses on an instrument. · Evidence of validity based on internal structure—This evidence results from an examination of the relationships among individual items appearing on the instrument and the underlying characteristic intended to be measured. · Evidence of validity based on relations to other variables—Analyses of the relationships between test scores and other measures of the same or similar constructs can provide evidence of the validity. · Evidence of validity based on consequences of testing—Any type of data collection is done with the expectation that some benefit will be realized from the intended and appropriate use of the scores. The process of validation should indicate whether these specific benefits are likely to be realized. Many of these sources of validity evidence are more appropriate for large-scale testing programs, especially where it is important for the results to be generalizable to much larger populations than simply those individuals included in a research study. Since this is not a purpose or goal of classroom-based action research, I am suggesting that practitioner-researchers be most concerned with evidence of validity based on instrument content. This source of evidence is based on the relationship between the content addressed on a test or on another instrument used for data collection and the underlying construct (or characteristic) it is trying to measure. For example, assume we wanted to survey students to determine their attitudes toward learning mathematics. We would want to ensure that the questions we asked on the survey dealt directly with various aspects of learning math, not learning in any other subject areas or questions that were completely extraneous to the construct of “learning mathematics.” As another example, consider a test you might administer to students on their understanding of the process of photosynthesis. If you wanted to be able to draw conclusions specifically about their understanding of this scientific process, you would need to be sure to ask only questions related to the process. If unrelated questions were also asked of students on the test—and provided that they contributed to the overall score on the test—interpreting the scores as an indication of their understanding only of photosynthesis would not be a valid, legitimate use of those scores. This type of evidence is typically based on subjective, logical analysis of content coverage on the test and can be established by critical review by teachers as well as by the judgments of experts in the particular content field. In other words, although it is a subjective process, it is important for practitioner-researchers to critically examine the individual items and overall content coverage on a survey, rating scale, checklist, test, or quiz in order to ensure that they are measuring what they are intended to measure. Reliability, a second essential characteristic of quantitative data, refers to the consistency of collected data. If you hear three accounts of a minor car accident from three different individuals but each account differs as to what happened, who was involved, and what the results were, you will likely have little confidence in any of the versions you have heard. In other words, the accounts (the data) are inconsistent and, therefore, unreliable. If, however, each account is essentially similar, the information you have received is consistent and may be considered reliable. Similarly, if you administer a certain test repeatedly under identical circumstances but find that you get different results each time, you will conclude that the test is unreliable. If, however, you get similar results each time you administer the test, you will consider the results reliable and, therefore, potentially useful for your purposes (Mertler & Charles, 2011). As with the determination of the validity of quantitative data, there are several methods of determining the reliability of data (Mertler & Charles, 2011), not all of which are appropriate for practitioners conducting action research. Reliability of quantitative data is usually established by correlating the results with themselves or with other quantitative measures. Three different methods are used—test-retest, equivalent forms, and internal consistency. The test-retest method, as well as the equivalent forms method, require two administrations of an instrument. This is usually not feasible for classroom teachers, or for other practitioners engaged in action research. Internal consistency is a statistical estimate of the reliability of a test that is administered only once. For this reason, this type of reliability estimate is typically most useful for practitioners conducting research. One of the easiest internal consistency formulas to use is the Kuder-Richardson formula 21 (also known as KR-21). The resulting statistic will range from 0.00 to 1.00; the closer the value is to 1.00, the more reliable your data are. This is a relatively straightforward calculation, which can easily be conducted using a hand calculator, employing the following formula: where K = the number of items on the test or other instrument SD = the standard deviation of the total scores x ¯  = the mean of the total scores We often think of validity and reliability as two distinct concepts, but in fact they share an important relationship (Mertler & Charles, 2011). It is possible for scores obtained from an instrument to be reliable (consistent) but not valid (measuring something other than what was intended). In contrast, scores cannot be both valid and unreliable—if scores measure what was intended to be measured, it is implied that they will do so consistently. Therefore, reliability is a necessary, but not sufficient, condition for validity. When establishing the validity and reliability of your research data, always remember the following adage: A valid test is always reliable, but a reliable test is not necessarily valid (Mertler, 2016). Previous section Next section 5.3 Notes About Ethics and Data Collection In Chapter 4, you learned about numerous ethical considerations that need to be incorporated into any action research project. Because we have been discussing data collection in this chapter, it is critical that we revisit several important issues. Recall that prior to collecting any data—especially if you are doing so outside of the normal, routine activities of a classroom teacher—you should obtain permission from both parents and the students themselves. Also recall that you should let your potential audience help dictate whether or not you need to obtain permission—if faculty and staff in your school will serve as the sole audience for the results of your action research study, you may not need to obtain any sort of permission. However, if you intend to share data that you collect with a broader audience (e.g., journals, professional conference, or even a districtwide presentation), you are obligated to secure permission to use student data in that manner. Along those lines, once you have collected data—whether it be from students, teachers, or other individuals—you must ensure that you keep those data secure and confidential. At no time should students’ or teachers’ names appear along with their data. One way to accomplish this is to code your participants so that random numbers—and not their names—represent individuals. A master list of the coding scheme can be kept under lock and key, accessible only to you as the classroom researcher. This way, a person outside of your study would not be able to identify individual participants and their respective data. An action researcher’s ability to ensure anonymity and confidentiality of participants and their data is a vitally important component of the action research process and of any action research project. Writing Up Action Research: Data Collection Methods This class is made up of thirty-one average and above-average science students. I chose this last class of the day for purely logistical reasons. With only one computer in my classroom, I needed to borrow eleven computers daily from neighboring teachers. Seventh period was the most agreeable period to the other teachers. An extra advantage of using the last period of the day was that students could return the computers after the final dismissal bell and not take valuable class time for this task. Additionally, I sent home parent surveys with each student in order to solicit and record the parents’ opinions concerning their child’s enthusiasm for learning science. The survey included two questions: How enthusiastic is your child about learning science? How enthusiastically does your child do science activities at home? I used the same rating scale for the parents that I used with the students. At the beginning of the second six weeks, I introduced a unit on oceanography. Oceanography was used as the unit of study primarily because of the number of resource materials available to the students through the media center. It was during this unit that I began to integrate technology into my curriculum. As the unit was introduced, I asked my students to look through the oceanography chapters in their textbooks and make a prioritized list of the eleven subtopics in physical and biological oceanography they would like to study. Students were grouped according to their interest as much as possible and were assigned to work in groups of two or three to develop a multimedia presentation that would be used as an instructional tool for the other students. During this period, I began to introduce them to the multimedia computer program, HyperStudio (Wagner, 1994). HyperStudio is a program that allows the user to combine sound, graphics, and animation with text to make creative and entertaining presentations. The introduction of HyperStudio and the development of the student presentations took six weeks to complete. Source: Hollis, 1995. During the second week, the class was taped on Monday and Thursday. At the end of the second week, modifications to the student survey were made on questions 1 and 3 due to mixed responses given by students. The modified student survey questions were: I continued to tape my first-period science class twice a week for a total of 5 weeks. Student surveys were given to all students on a random basis throughout the 5-week period. Journal entries were made daily. Source: Graham, 1995. Action Research Portrait 1: Improving Reading Comprehension in a Title I Program Developing a Plan and Collecting Data Recall that the purpose of this action research study is to improve students’ reading comprehension skills within a Title I context. In order to address her initial research question—which proposed to examine differences in students’ reading comprehension skills following the use of revised teacher-developed comprehension items, based on pretest and posttest diagnostic test scores—Kathleen needed to select an appropriate and valid measure of reading comprehension. After reviewing the various diagnostic tests with which she was familiar and had experience administering, Kathleen selected the Woodcock Reading Mastery Test–Revised (Form H) to administer to her reading students in September and in May. From the resulting student score reports, she would extract the Reading Comprehension Cluster score, which appears as a percentile rank. An average score on this subtest is the 50th percentile; Kathleen’s upper-elementary students typically score near the 35th percentile. She obviously hoped to improve that performance over the course of the school year. Kathleen’s second research question dealt with the perceptions held by both her students and herself regarding the students’ reading comprehension skills. She proposed to collect two forms of data to enable her to address the nature of those perceptions. First, she would conduct daily observations of her students and record both what she saw and any analytical thoughts she may have had while conducting the observations. The focus of her observations would be the degree to which the students could answer oral and written questions after having read a passage from a book. Specifically, she would look for how her students used the strategies for reading comprehension that they had been taught. Second, Kathleen also wanted to periodically ask her students direct questions regarding the use of those reading comprehension strategies. She designed a semistructured interview guide for conducting these student interviews. Her interview guide included the following questions: · What does “reading comprehension” mean to you? · Do you have trouble understanding what you read? · Why do you think you have trouble? · What helps someone understand what he or she has read? · Do you ever do any of these things? · What strategies do you use to help you understand what you read? · Do you enjoy reading? Kathleen planned to interview each student at least twice at roughly 2-month intervals during the course of her action research project. She anticipated learning more about their perceptions of reading, in general, and reading for understanding. She was also curious as to whether those perceptions would change over time. Action Research Portrait 2: Conceptual Understanding of Mitosis and Meiosis Developing a Plan and Collecting Data Recall that the purpose of this action research study is to improve students’ understanding of the processes of mitosis and meiosis. Sarah and Tom taught their mitosis and meiosis units in January and February and administered the unit test in mid-February. In order to address their first research question, the two teachers collected students’ test scores and recorded them on a spreadsheet, excerpts of which appear in the box below. Notice that Sarah and Tom recorded not only the test score for each individual student but also the class period (i.e., 1 or 3 in the excerpt of the table) and the type of instruction that class received (i.e., “1” = traditional instruction and “2” = traditional instruction plus supplemental resources). They knew that this information would be necessary for them to be able to conduct any comparative statistical analyses. Because they utilized a two-group comparative design, Sarah knew from her coursework that they would need to analyze the test scores using an independent-samples t test. To help answer the second research question, Sarah and Tom first created a collaborative classroom space on the Internet in the form of a blog (at http://www.blogger.com). In order to initiate the classroom “discussion,” the teachers posed a couple of general questions, in separate threaded discussion boards, about mitotic and meiotic processes. · Threaded Discussion #1—Why are mitosis and meiosis important? · Threaded Discussion #2—Sometimes the processes do not “go smoothly.” Predict what might happen to an organism if the process is flawed. · Threaded Discussion #3—What questions do you have about mitosis and meiosis? What are you still unclear about? All of their students were required to post a minimum of two comments or questions. Sarah and Tom believed that the blogs could provide them with richer qualitative data, since the students could respond to each other’s questions and comments. In addition, use of blogs has a distinct advantage from a data collection perspective—all of the students’ submissions would be recorded for the teachers to analyze at a later time. Sarah and Tom also conducted what they called “group oral exams” and “individual oral exams,” which, as you can imagine, did not excite the students very much. However, both teachers explained to the students that they really just wanted them to discuss what they knew and clearly understood about the processes of mitosis and meiosis, both in a group setting and individually. They informed their students that these oral exams would count in their grades but that the written unit test would serve as the primary basis for grades in this unit. During one day in each class, Sarah and Tom engaged the students in a group discussion about what they had learned. For example, the teacher might ask one person to begin discussing the steps in mitosis, and at some point during the response, the teacher would stop that student and ask another to continue from that point. Sarah and Tom took notes during each respective class, highlighting what the students in each class seemed to clearly understand and what they continued to struggle with. Before the end of the day, both teachers had already noticed some patterns emerging. The next day, each student was called up to his or her teacher’s desk for the individual oral exam. These were very structured (because of time constraints) and consisted of four brief questions. Sarah and Tom asked each of their students the following, carefully recording the responses: · What is the purpose of mitosis? · How do you clearly know when one phase of mitosis ends and the next begins? · What is the purpose of meiosis? · How are offspring created by mitosis and those created by meiosis similar? The final question for each student came from a set of questions about the stages of mitosis. The teachers showed the Java-based animation of the entire process of mitosis, stopping it at random but different places for each student. Students were then asked to identify in which phase the process had been stopped. After recording all of this information and gathering all of the blog entries, Sarah and Tom were ready to begin the analysis of their data. Action Research Checklist 5 Developing a Data Collection Plan for Action Research Revisit your research question(s) and your previous decisions about whether you will use a qualitative, quantitative, or mixed-methods design for your action research. If you intend to collect qualitative data, decide which of the following you will use: · □ Observations (structured, semistructured, or unstructured) · □ Interviews (structured, semistructured, or unstructured) · □ Journals · □ Existing documents · □ Other sources of qualitative data If you intend to collect quantitative data, decide which of the following you will use: · □ Surveys, questionnaires, or rating scales (What types of scales—Likert, Likert-type, or other—or open-ended questions?) · □ Checklists · □ Formative and/or summative classroom assessments · □ Standardized test scores · □ Existing numerical data · □ Other sources of quantitative data If you intend to use anything original (i.e., self-developed), such as interview guides, surveys, or formative assessments, develop drafts of your data collection instrumentation. Engage in appropriate processes (depending on the type of data collection instrument you are using) to ensure that you will collect good quality data. Previous section Next section 5.3 Notes About Ethics and Data Collection In Chapter 4, you learned about numerous ethical considerations that need to be incorporated into any action research project. Because we have been discussing data collection in this chapter, it is critical that we revisit several important issues. Recall that prior to collecting any data—especially if you are doing so outside of the normal, routine activities of a classroom teacher—you should obtain permission from both parents and the students themselves. Also recall that you should let your potential audience help dictate whether or not you need to obtain permission—if faculty and staff in your school will serve as the sole audience for the results of your action research study, you may not need to obtain any sort of permission. However, if you intend to share data that you collect with a broader audience (e.g., journals, professional conference, or even a districtwide presentation), you are obligated to secure permission to use student data in that manner. Along those lines, once you have collected data—whether it be from students, teachers, or other individuals—you must ensure that you keep those data secure and confidential. At no time should students’ or teachers’ names appear along with their data. One way to accomplish this is to code your participants so that random numbers—and not their names—represent individuals. A master list of the coding scheme can be kept under lock and key, accessible only to you as the classroom researcher. This way, a person outside of your study would not be able to identify individual participants and their respective data. An action researcher’s ability to ensure anonymity and confidentiality of participants and their data is a vitally important component of the action research process and of any action research project. Writing Up Action Research: Data Collection Methods This class is made up of thirty-one average and above-average science students. I chose this last class of the day for purely logistical reasons. With only one computer in my classroom, I needed to borrow eleven computers daily from neighboring teachers. Seventh period was the most agreeable period to the other teachers. An extra advantage of using the last period of the day was that students could return the computers after the final dismissal bell and not take valuable class time for this task. Additionally, I sent home parent surveys with each student in order to solicit and record the parents’ opinions concerning their child’s enthusiasm for learning science. The survey included two questions: How enthusiastic is your child about learning science? How enthusiastically does your child do science activities at home? I used the same rating scale for the parents that I used with the students. At the beginning of the second six weeks, I introduced a unit on oceanography. Oceanography was used as the unit of study primarily because of the number of resource materials available to the students through the media center. It was during this unit that I began to integrate technology into my curriculum. As the unit was introduced, I asked my students to look through the oceanography chapters in their textbooks and make a prioritized list of the eleven subtopics in physical and biological oceanography they would like to study. Students were grouped according to their interest as much as possible and were assigned to work in groups of two or three to develop a multimedia presentation that would be used as an instructional tool for the other students. During this period, I began to introduce them to the multimedia computer program, HyperStudio (Wagner, 1994). HyperStudio is a program that allows the user to combine sound, graphics, and animation with text to make creative and entertaining presentations. The introduction of HyperStudio and the development of the student presentations took six weeks to complete. Source: Hollis, 1995. During the second week, the class was taped on Monday and Thursday. At the end of the second week, modifications to the student survey were made on questions 1 and 3 due to mixed responses given by students. The modified student survey questions were: I continued to tape my first-period science class twice a week for a total of 5 weeks. Student surveys were given to all students on a random basis throughout the 5-week period. Journal entries were made daily. Source: Graham, 1995. Action Research Portrait 1: Improving Reading Comprehension in a Title I Program Developing a Plan and Collecting Data Recall that the purpose of this action research study is to improve students’ reading comprehension skills within a Title I context. In order to address her initial research question—which proposed to examine differences in students’ reading comprehension skills following the use of revised teacher-developed comprehension items, based on pretest and posttest diagnostic test scores—Kathleen needed to select an appropriate and valid measure of reading comprehension. After reviewing the various diagnostic tests with which she was familiar and had experience administering, Kathleen selected the Woodcock Reading Mastery Test–Revised (Form H) to administer to her reading students in September and in May. From the resulting student score reports, she would extract the Reading Comprehension Cluster score, which appears as a percentile rank. An average score on this subtest is the 50th percentile; Kathleen’s upper-elementary students typically score near the 35th percentile. She obviously hoped to improve that performance over the course of the school year. Kathleen’s second research question dealt with the perceptions held by both her students and herself regarding the students’ reading comprehension skills. She proposed to collect two forms of data to enable her to address the nature of those perceptions. First, she would conduct daily observations of her students and record both what she saw and any analytical thoughts she may have had while conducting the observations. The focus of her observations would be the degree to which the students could answer oral and written questions after having read a passage from a book. Specifically, she would look for how her students used the strategies for reading comprehension that they had been taught. Second, Kathleen also wanted to periodically ask her students direct questions regarding the use of those reading comprehension strategies. She designed a semistructured interview guide for conducting these student interviews. Her interview guide included the following questions: · What does “reading comprehension” mean to you? · Do you have trouble understanding what you read? · Why do you think you have trouble? · What helps someone understand what he or she has read? · Do you ever do any of these things? · What strategies do you use to help you understand what you read? · Do you enjoy reading? Kathleen planned to interview each student at least twice at roughly 2-month intervals during the course of her action research project. She anticipated learning more about their perceptions of reading, in general, and reading for understanding. She was also curious as to whether those perceptions would change over time. Action Research Portrait 2: Conceptual Understanding of Mitosis and Meiosis Developing a Plan and Collecting Data Recall that the purpose of this action research study is to improve students’ understanding of the processes of mitosis and meiosis. Sarah and Tom taught their mitosis and meiosis units in January and February and administered the unit test in mid-February. In order to address their first research question, the two teachers collected students’ test scores and recorded them on a spreadsheet, excerpts of which appear in the box below. Notice that Sarah and Tom recorded not only the test score for each individual student but also the class period (i.e., 1 or 3 in the excerpt of the table) and the type of instruction that class received (i.e., “1” = traditional instruction and “2” = traditional instruction plus supplemental resources). They knew that this information would be necessary for them to be able to conduct any comparative statistical analyses. Because they utilized a two-group comparative design, Sarah knew from her coursework that they would need to analyze the test scores using an independent-samples t test. To help answer the second research question, Sarah and Tom first created a collaborative classroom space on the Internet in the form of a blog (at http://www.blogger.com). In order to initiate the classroom “discussion,” the teachers posed a couple of general questions, in separate threaded discussion boards, about mitotic and meiotic processes. · Threaded Discussion #1—Why are mitosis and meiosis important? · Threaded Discussion #2—Sometimes the processes do not “go smoothly.” Predict what might happen to an organism if the process is flawed. · Threaded Discussion #3—What questions do you have about mitosis and meiosis? What are you still unclear about? All of their students were required to post a minimum of two comments or questions. Sarah and Tom believed that the blogs could provide them with richer qualitative data, since the students could respond to each other’s questions and comments. In addition, use of blogs has a distinct advantage from a data collection perspective—all of the students’ submissions would be recorded for the teachers to analyze at a later time. Sarah and Tom also conducted what they called “group oral exams” and “individual oral exams,” which, as you can imagine, did not excite the students very much. However, both teachers explained to the students that they really just wanted them to discuss what they knew and clearly understood about the processes of mitosis and meiosis, both in a group setting and individually. They informed their students that these oral exams would count in their grades but that the written unit test would serve as the primary basis for grades in this unit. During one day in each class, Sarah and Tom engaged the students in a group discussion about what they had learned. For example, the teacher might ask one person to begin discussing the steps in mitosis, and at some point during the response, the teacher would stop that student and ask another to continue from that point. Sarah and Tom took notes during each respective class, highlighting what the students in each class seemed to clearly understand and what they continued to struggle with. Before the end of the day, both teachers had already noticed some patterns emerging. The next day, each student was called up to his or her teacher’s desk for the individual oral exam. These were very structured (because of time constraints) and consisted of four brief questions. Sarah and Tom asked each of their students the following, carefully recording the responses: · What is the purpose of mitosis? · How do you clearly know when one phase of mitosis ends and the next begins? · What is the purpose of meiosis? · How are offspring created by mitosis and those created by meiosis similar? The final question for each student came from a set of questions about the stages of mitosis. The teachers showed the Java-based animation of the entire process of mitosis, stopping it at random but different places for each student. Students were then asked to identify in which phase the process had been stopped. After recording all of this information and gathering all of the blog entries, Sarah and Tom were ready to begin the analysis of their data. Action Research Checklist 5 Developing a Data Collection Plan for Action Research Revisit your research question(s) and your previous decisions about whether you will use a qualitative, quantitative, or mixed-methods design for your action research. If you intend to collect qualitative data, decide which of the following you will use: · □ Observations (structured, semistructured, or unstructured) · □ Interviews (structured, semistructured, or unstructured) · □ Journals · □ Existing documents · □ Other sources of qualitative data If you intend to collect quantitative data, decide which of the following you will use: · □ Surveys, questionnaires, or rating scales (What types of scales—Likert, Likert-type, or other—or open-ended questions?) · □ Checklists · □ Formative and/or summative classroom assessments · □ Standardized test scores · □ Existing numerical data · □ Other sources of quantitative data If you intend to use anything original (i.e., self-developed), such as interview guides, surveys, or formative assessments, develop drafts of your data collection instrumentation. Engage in appropriate processes (depending on the type of data collection instrument you are using) to ensure that you will collect good quality data. 5.4 Related Websites: Advice and Guidelines About Data Collection The related websites for this chapter all come from the Action Research Website of the Madison (Wisconsin) Metropolitan School District. They provide several good ideas and suggestions for issues you may very well face when dealing with decisions about collecting data for your action research project. · Guidelines for Data Collection http://oldweb.madison.k12.wi.us/sod/car/cardataguidelines.html Offered here is a bulleted list of suggestions or simply things to consider when planning for your data collection. The page begins with the following statement: “Asking the right questions is the key skill in effective data collection.” Guidelines included on this list are “Be clear as to why you are collecting data,” “Be clear about how you are going to use the data you collect,” “Decide how much data is needed,” and “Use multiple sources of data to increase the believability of the findings.” · Techniques for Gathering Data https://staffdevweb.madison.k12.wi.us/node/242 Fourteen different techniques for collecting data are briefly described. The list includes many that we discussed in this chapter but also includes several additional techniques. These additional techniques include portfolios, still photography, and time-on-task analysis. · Data Collection: The Five Ws and an H https://staffdevweb.madison.k12.wi.us/node/236 The authors suggest that, prior to actually collecting data, teachers should ask themselves several questions, listed on this webpage. These questions fall under the following broader questions: · Why are we collecting this data? · What exactly are we collecting? · Where are we going to collect data and for how long? · When are we going to collect data and for how long? · Who is going to collect the data? · How will the data be collected and displayed? 5.4 Related Websites: Advice and Guidelines About Data Collection The related websites for this chapter all come from the Action Research Website of the Madison (Wisconsin) Metropolitan School District. They provide several good ideas and suggestions for issues you may very well face when dealing with decisions about collecting data for your action research project. · Guidelines for Data Collection http://oldweb.madison.k12.wi.us/sod/car/cardataguidelines.html Offered here is a bulleted list of suggestions or simply things to consider when planning for your data collection. The page begins with the following statement: “Asking the right questions is the key skill in effective data collection.” Guidelines included on this list are “Be clear as to why you are collecting data,” “Be clear about how you are going to use the data you collect,” “Decide how much data is needed,” and “Use multiple sources of data to increase the believability of the findings.” · Techniques for Gathering Data https://staffdevweb.madison.k12.wi.us/node/242 Fourteen different techniques for collecting data are briefly described. The list includes many that we discussed in this chapter but also includes several additional techniques. These additional techniques include portfolios, still photography, and time-on-task analysis. · Data Collection: The Five Ws and an H https://staffdevweb.madison.k12.wi.us/node/236 The authors suggest that, prior to actually collecting data, teachers should ask themselves several questions, listed on this webpage. These questions fall under the following broader questions: · Why are we collecting this data? · What exactly are we collecting? · Where are we going to collect data and for how long? · When are we going to collect data and for how long? · Who is going to collect the data? · How will the data be collected and displayed? Summary · Qualitative data are narrative, appearing primarily as words. · Qualitative data are usually collected through observations, interviews, or journals or by obtaining existing documents or records. · Observations involve carefully and systematically watching and recording what you see and hear in a given setting. · Classroom observations may be structured, semistructured, or unstructured. · Unstructured or semistructured observations allow for the flexibility to attend to other events occurring in the classroom. · Classroom observations are usually recorded in the form of field notes, which may include observer’s comments. · Interviews are typically formal conversations between individuals. · Interviews typically follow an interview guide, which may be structured, semistructured, or open-ended. · Interviews can also be conducted with groups of individuals in an interview known as a focus group. · Interviews may also be conducted informally or via e-mail. · Journals may also be kept by both teachers and students in order to provide valuable insights into the workings of a classroom. · Existing documents and records, originally gathered for reasons other than action research, are abundantly available in schools and may be used as additional sources of information. These include classroom artifacts, such as student work. · It is important for practitioner-researchers to establish the trustworthiness of their data. This includes the credibility, transferability, dependability, and confirmability of one’s qualitative data. · Quantitative data are numerical and include just about anything that can be counted, tallied, or rated. · Surveys are lists of statements or questions to which participants respond. · Questionnaires are one specific type of survey involving the administration of questions or statements in written form. · Items on surveys can consist of open-ended questions or closed-response rating scales. · A closed-response question or statement provides the respondent with a number of choices from which to select. Analysis of the resulting data involves counting the number of responses for each option. · Open-ended items allow for a seemingly limitless number of possible responses. Analysis of these data involves categorizing responses into similar groups and then counting them. · Surveys and rating scales are effective at gathering data simultaneously from numerous individuals, but they can sometimes be time-consuming to analyze. · Checklists are a simple form of rating scale where only a dichotomy of response options (e.g., present or not present) exists. · Tests and other formal instruments can be used as quantitative data, provided they are supplemented with other forms of data. · Validity of quantitative data has to do with the extent to which the data are what they are believed to be. · Reliability refers to the consistency of quantitative data and is determined statistically. · Remember the following: A valid test is always reliable, but a reliable test is not necessarily valid. Questions and Activities 1. Describe what you might see as the benefits of collecting both qualitative and quantitative data as part of an action research study. Do you envision any potential negative aspects or weaknesses associated with collecting both as part of the same study? 2. Why is it important for researchers in general, and specifically for practitioner-researchers, to take measures to ensure the quality of their collected data? 3. Making good, sound observations typically requires some training or at least practice. Find a location with numerous people (e.g., a shopping mall, your student union) and spend 30 minutes observing and making field notes on what you see and hear. Include any observer’s comments as you deem appropriate during your period of observation. After the 30 minutes of observation, reflect on the experience. What did you think, and how did you feel? How could you improve your observation and note-taking skills for your next observation session? 4. Think of a topic of interest to you and appropriate for an action research study. Develop a semistructured interview guide for a 15-minute interview with an individual. Be sure to include in your guide any “optional” probing questions. Next, interview someone using your guide. Afterward, reflect on your experience as an interviewer. What did you think, and how did you feel? How could you improve your skills for your next interview? 5. Using the same topic you identified for Number 4 above, develop a 15-item survey or rating scale that targets a specific audience, paying close attention to the guidelines presented in the chapter. Remember that a rating scale will use either a Likert or a Likert-type scale. Administer your instrument to at least five individuals. Afterward, ask them to provide you with feedback on the instrument. Reflect on the process of instrument development and administration. 6. Can you think of any existing documents or records that would support an investigation of your topic? What are they? How difficult would it be to gain access to them? Key Terms · acting stage 129 · class journal 138 · classroom artifacts 139 · confirmability 141 · construct 155 · credibility 140 · data journals 138 · dependability 141 · external audit 143 · focus group 134 · formative classroom assessments 152 · informal interviews 137 · internal consistency 156 · interview guide 133 · interviews 133 · Kuder-Richardson formula 21 156 · Likert scale 145 · Likert-type scale 146 · member checking 143 · observations 130 · observer’s comments 131 · open-ended interviews 134 · open-ended questions 144 · peer debriefing 143 · reflexivity 143 · reliability 156 · semistructured interviews 134 · semistructured observations 131 · structured interview 133 · structured observations 130 · student journals 138 · summative classroom assessment 153 · teacher journals 138 · transferability 140 · triangulation 141 · trustworthiness 140 · unstructured observations 131 · validity of research data 140 · videotapes 133 Review → Practice → Improve Get the tools you need to sharpen your study skills. Access the videos listed in the margins of this chapter, practice quizzes, eFlashcards and more at  edge.sagepub.com/mertler5e ·