Fluent Essays

Part 2 Journal Search and Peer Review Topic: Plant-Based Meat This past year, a new product has exploded in the food market: plant-based meat. With names such as “Beyond Meat” burgers and “Impossible Whopper™”, these plant-based meat substitutes make a lot of claims about taste, nutritional benefits and ethical issues (Duggan 2019). Many food chains such as Burger King, Red Robin, White Castle, Qdoba and Del Tacos have started marketing the use of these vegetarian and vegan alternatives. Duggan, Tara “Critics question ethics behind Impossible Burger’s rapid fast-food expansion” San Francisco Chronicle, 16 May 2019. Web 18 August 2019 In Part I, websites were investigated as resources for finding information. In Part 2, we will be looking at scientific journal articles, use PlumX metric to evaluate them, and look at what peer-review means and why it is important to science as evidence-based information rather than opinion. 1. You will need to search for a full-text, scholarly article that is closely related to the topic of plant-based meat. 2. You will need to review the two handouts: the informative pamphlet, I don’t know what to believe (https://senseaboutscience.org/wp-content/uploads/2016/11/IDontKnowWhatToBelieve.pdf) by Sense about Science, a science non-profit agency from the United Kingdom and A Rough Guide to Spotting Bad Science (https://www.compoundchem.com/2014/04/02/a-rough-guide-to-spotting-bad-science/ )an infographic by Compound Interest. Your grade will be determined by a. Explanation (using two to three sentences) of how the article was selected and listing the key words used in the search b. Proper citation of the article using the format indicated by instructor: c. Report the PlumX metric for the journal and tell what that number means d. The relevance of article to topic. e. Based on reading the handouts provide a clear and accurate explanation of peer-review including 1) what is peer-review, 2) why peer review is important and 3) a challenge of peer-review. Formats for citations include MLA, APA and American Chemical Society (shown below) Last name, First and middle initial; Last name of second author, First and middle initial of second author. Title of article, Journal title. Year. Volume. Pages NOTE: all authors must be listed in the order in which they appear in the paper. Grading Rubric for Part 2: Journal Search and Peer Review Substantially Exceeds Expectations(5) Exceeds Expectations (4) Meets Expectations (3) Nearly Meets Expectations (2) Does Not Meet Expectations (1) No score (0) Selection of article and keywords used in search The explanation for selecting the article is clear and concise. Three or more relevant keywords used in search. The explanation for selecting article is concise but not clear. Three or more relevant keywords used in search. The explanation for selecting article needs more specific details. One or two keywords used; they are relevant to the topic. The explanation for selecting article is not closely related to article. Two or more keywords used but unrelated to topic. The explanation for selecting article is not related to article. Only one keyword was used. The explanation for selecting article is not given. Keywords not given. Article reference citation Reference is completely correct using format assigned. Reference follows format assigned but has one error. Reference is completely correct for a different format style. Reference follows different format with more than one error. Reference is missing more than two required item. Reference is not given. PlumX metric PlumX metric is correct and explanation is correct. PlumX metric is incorrect but explanation correct PlumX metric is correct but explanation is flawed. PlumX metric is incorrect and explanation is flawed. PlumX metric is correct but explanation is incorrect or missing. PlumX metric is incorrect and explanation is missing. Relevance of article to topic The article is closely related to the topic and is from a peer-reviewed journal. The article is somewhat related to the topic and is from a peer-reviewed journal. The article is closely related to the topic but is not from a peer-reviewed journal. The article is not related to the topic but is from a peer-reviewed journal. The article is neither related to the topic nor is it from a peer-reviewed journal. The article is not given. Explanation of peer review Response is consistent with the main points presented in the peer review handouts; response is clearly and completely articulated (all three questions addressed). Response is generally consistent with the main points given in the peer review handouts; response is not completely accurate or missing a component. Response touches on some of the ideas given in the peer review handouts but is generally inaccurate and/or incoherent. Response touches on one of the ideas given in the peer review handouts but is generally not consistent with the handouts. Response is obviously inconsistent with the main points given in the peer review handouts. Explanation of peer review is not present. Total (25) ??? C BY NC ND A Rough Guide to SPOTTING BAD SCIENCE © COMPOUND INTEREST 2015 - WWW.COMPOUNDCHEM.COM | @COMPOUNDCHEM Shared under a Creative Commons Attribution-NonCommercial-NoDerivatives licence. 1. SENSATIONALISED HEADLINES Aa Article headlines are commonly designed to entice viewers into clicking on and reading the article. At times, they can over-simplify the findings of scientific research. At worst, they sensationalise and misrepresent them. 2. MISINTERPRETED RESULTS News articles can distort or misinterpret the findings of research for the sake of a good story, whether intentionally or otherwise. If possible, try to read the original research, rather than relying on the article based on it for information. 3. CONFLICTS OF INTEREST Many companies will employ scientists to carry out and publish research - whilst this doesn’t necessarily invalidate the research, it should be analysed with this in mind. Research can also be misrepresented for personal or financial gain. 4. CORRELATION & CAUSATION Be wary of any confusion of correlation and causation. A correlation between variables doesn’t always mean one causes the other. Global warming increased since the 1800s, and pirate numbers decreased, but lack of pirates doesn’t cause global warming. 5. UNSUPPORTED CONCLUSIONS Speculation can often help to drive science forward. However, studies should be clear on the facts their study proves, and which conclusions are as yet unsupported ones. A statement framed by speculative language may require further evidence to confirm. 6. PROBLEMS WITH SAMPLE SIZE In trials, the smaller a sample size, the lower the confidence in the results from that sample. Conclusions drawn can still be valid, and in some cases small samples are unavoidable, but larger samples often give more representative results. 7. UNREPRESENTATIVE SAMPLES USED In human trials, subjects are selected that are representative of a larger population. If the sample is different from the population as a whole, then the conclusions from the trial may be biased towards a particular outcome. 8. NO CONTROL GROUP USED In clinical trials, results from test subjects should be compared to a ‘control group’ not given the substance being tested. Groups should also be allocated randomly. In general experiments, a control test should be used where all variables are controlled. 9. NO BLIND TESTING USED To try and prevent bias, subjects should not know if they are in the test or the control group. In ‘double blind’ testing, even researchers don’t know which group subjects are in until after testing. Note, blind testing isn’t always feasible, or ethical. 10. SELECTIVE REPORTING OF DATA Also known as ‘cherry picking’, this involves selecting data from results which supports the conclusion of the research, whilst ignoring those that do not. If a research paper draws conclusions from a selection of its results, not all, it may be guilty of this. 11. UNREPLICABLE RESULTS Results should be replicable by independent research, and tested over a wide range of conditions (where possible) to ensure they are consistent. Extraordinary claims require extraordinary evidence - that is, much more than one independent study! 12. NON-PEER REVIEWED MATERIAL Peer review is an important part of the scientific process. Other scientists appraise and critique studies, before publication in a journal. Research that has not gone through this process is not as reputable, and may be flawed. x x Being able to evaluate the evidence behind a scientific claim is important. Being able to recognise bad science reporting, or faults in scientific studies, is equally important. These 12 points will help you separate the science from the pseudoscience. MAKING SENSE OF SCIENCE STORIES This leaflet is for people who follow debates about science and medicine in the news. It explains how scientists present and judge research and how you can ask questions of the scientific information presented to you. Because evidence matters SO, NO MATTER HOW EXCITING OR COMPELLING NEW SCIENTIFIC OR MEDICAL RESEARCH IS, Is it peer reviewed? If not, why not? YOU MUST ALWAYS ASK… If it is peer reviewed, you can look for more information on what other scientists say about it, the size and approach of the study and whether it is part of a body of evidence pointing towards the same conclusions. • Science has a system for assessing the quality of research before it is published. This system is called peer review. • Peer review means that other scientific experts in the field check research papers for validity, significance and originality — and for clarity. • Editors of scientific journals draw on a large pool of suitable experts to scrutinise papers before deciding whether to publish them. • Many of the research claims you read in newspapers and magazines, find on the internet, or hear on television and the radio are not published in a peer-reviewed journal. Some of this research may turn out to be good but much of it is flawed or incomplete. Many reported findings, such as claims about “wonder cures” and “new dangers”, never come to anything. • Unpublished research is no help to anyone. Scientists can’t repeat or use it and as a society we can’t base decisions about our public safety — or our family’s health for example — on work that has a high chance of being flawed. Summary SCIENTISTS NEVER DRAW FIRM CONCLUSIONS FROM JUST ONE PAPER OR SET OF RESULTS How should you make sense of science stories? What do scientists do when they assess a paper for peer review? Every day we are bombarded with information about science from newspapers, radio and television programmes and the internet. Making sense of it all can be very difficult. What should be taken seriously? Which are ‘scares’? Sometimes scientists are reported as saying conflicting things. How do we know what to believe? There is a system used by scientists to decide which research results should be published in a scientific journal. This system, called peer review, subjects scientific research papers to independent scrutiny by other qualified scientific experts (peers) before they are made public. Peer review can help you make sense of science stories as it tells you that the research has passed the scrutiny of other scientists and is considered valid, significant and original. Peer review means that statements made by scientists in scientific journals are critically different from other kinds of statements or claims, such as those made by politicians, newspaper columnists or campaign groups. Science is therefore more than just another opinion. When a researcher, or team of researchers, finishes a stage of work, they usually write a paper presenting their methods, findings and conclusions. They then send the paper to a scientific journal to be considered for publication. If the journal’s editor thinks it is suitable for their journal they send the paper to other scientists who research and publish in the same field. They consider the contribution it makes in the context of other work and their own experience. It usually takes more than one research paper for results to be seen as good evidence or accepted as a public truth. A short explanation of peer review • Comment on its validity — are the research results credible; are the design and methodology appropriate? • Judge the significance — is it an important finding? • Determine its originality — are the results new? Does the paper refer properly to work done by others? • Give an opinion as to whether the paper should be published, improved or rejected (usually to be submitted elsewhere). This process is called peer review. The scientists (peers) assessing the papers are called referees or reviewers. A NOTE ON JOURNAL FUNDING AND AVAILABILITY Did you know? For scientific knowledge to progress scientists need to share their research findings with other scientists. The main way they do this is by publishing their research in scientific journals — periodical publications intended to further the development of science by reporting new research. Journal editors receive many more papers than they can publish, so they use a two-step selection process. First, they consider whether the paper is a ‘fit’ for their journals. For example, some journals only publish research papers that are groundbreaking; others only publish research in a specific area, such as microbiology. If a journal editor decides that a paper is right for their journal, they send it for peer review to check whether the research findings are valid, significant and original. Most journals receive their income from subscriptions and some from organisational subsidies, conference organising and advertising. Most journals are behind paywalls, though many show abstracts of papers for free, or make the content free after a certain period. Other journal funding models are growing, including open access publishing, where scientists pay the costs of reviewing and publishing their articles so that they can be made freely available. There are around 28,000 scholarly and scientific journals that use the peer-review system. A high proportion of these are scientific, technical or medical journals, publishing over 1 million research papers each year. The science publishing scene How can you tell whether reported results have been peer reviewed? BY THE WAY… PEER REVIEW OF RESEARCH PROPOSALS Peer review is also used to assess scientists’ applications for research funds. Funding bodies, such as medical research charities, seek expert advice on a scientist’s proposal before agreeing to pay for it. Peer review in this instance is used to judge which applications are the best science and have the potential to help the organisation achieve its objectives. It can be difficult! The full reference to peer-reviewed papers is likely to look like this: Fellers J H and Fellers G M (1976) Tool use in a social insect and its implications for competitive interactions. Science, 192, 70-72. or this: Hedenfalk I, Duggan D, Chen Y, et al. Gene- expression profiles in hereditary breast cancer. N Engl J Med, 2001; 344: 539-48. A few unscrupulous people use this style on websites and in articles to cite work that is not peer reviewed. But fortunately this is rare. You are most likely to hear about new research where there is not space or interest in full references, but journalists should indicate if research has been published, and mention the name of the journal. Research papers presented at scientific conferences have often begun a process of peer review but are usually still unpublished and preliminary. The more we ask, ‘is it peer reviewed?’ the more obliged reporters will be to include this information. There is no definitive list of peer- reviewed journals but you can look up the names of selected peer-reviewed journals online at the science news service EurekAlert! Publishing in a journal is an integral part of being a scientist it: • Connects like-minded individuals and tells them about new research. A published paper is read by scientists all around the world. • Shows the quality of the scientist’s work: other experts have rated it as valid, significant and original. • Is a permanent record of what has been discovered, when and by which scientists — like a court register for science. • Helps scientists to promote their work and gain recognition from funders and other institutions. Sounds good, but what happens next? So scientists use peer review, so what? When research findings have been peer reviewed and published in a scientific journal, this indicates that they are sufficiently valid, significant and original to merit the attention of other scientists. Peer review is an essential dividing line for judging what is scientific and what is speculation and opinion. Most scientists make a careful distinction between their peer-reviewed findings and their more general opinions. Publication of a peer-reviewed paper is just the first step: findings, and theories about them, must go on to be re-tested and judged against other work in the same area. Some papers’ conclusions will be disputed or further research will show that they need to be revised as more data are gathered. Why can’t there just be a checklist of scientific validity? Is maverick science rejected through peer review? Does the peer review process slow down advances in scientific and medical knoweldge? Does peer review detect fraud and misconduct? In our world of instant communication and 24-hour news, a deliberative process like peer review can seem frustratingly slow. Electronic communication has improved it, but good assessment of research does take time. Sometimes people justify the promotion of unpublished findings by saying they are ‘too important to wait’. But, although some papers take months to review and improve, if there is a major breakthrough the process can be completed in weeks. Furthermore, if the findings are very important — e.g. they concern public health — then it is all the more necessary to check them through peer review. Assessing scientific papers cannot be done in the same way as giving a car an M.O.T. or marking a maths test. New research usually has its own unique features, which are difficult to predict with a check list and which require expert judgement about their validity, significance and originality. Challenges for peer review Peer review is not a fraud detection system. Referees are likely to detect some wrongdoing, such as copying someone else’s research or misrepresenting data, because they care about their subject. They know what research has been conducted already and the kinds of results that are likely. However, if someone deliberately sets out to falsify data, there is sometimes no way of knowing this until the paper is published and others in the scientific community scrutinise and try to repeat the work. Sometimes people worry that new ideas won’t be understood by other scientists (although this is also an excuse given when researchers don’t want to submit to the scrutiny of their peers). It is true that referees can be cautious about unusual findings; and important insights can initially be overlooked. But if someone has been exceptionally clever, other scientists are most likely to recognise it and to distinguish it from flawed or inflated claims. Journal editors like novel ideas and scientific publishing has brought thousands of important discoveries to light. Acknowledgements THIS LEAFLET WAS ORIGINALLY PRODUCED AND DISTRIBUTED WITH SUPPORT AND HELP FROM: • Ask for Evidence Sense about Science runs the Ask for Evidence campaign to help people request for themselves the evidence behind news stories, marketing claims and policies. www.askforevidence.org • Association of Medical Research Charities The AMRC has a page on peer review for medical charities. www.amrc.org.uk/our-work/peer- review • Committee on Publication Ethics COPE provides a sounding board for journal editors struggling with how to deal with breaches in research and publication ethics. www.publicationethics.org • NHS Evidence This evidence search provides access to selected and authoritative evidence in health, social care and public health. www.evidence.nhs.uk • The Science Media Centre An independent press office helping to ensure that the public have access to the best scientific evidence and expertise through the news media when science hits the headlines. www.sciencemediacentre.org Sense about Science is grateful for the input of the sponsors, the many organisations (in particular Cancer Research UK, Asthma UK, Migraine Trust and Action Medical Research), parliamentarians, government officials, educational organisations, teachers, school students, doctors, pharmacists, science bodies and the many others, who kindly contributed their time and ideas. Responsibility for the content rests fully with Sense about Science. www.senseaboutscience.org Sense about Science 14a Clerkenwell Green London EC1R 0DP Registered Charity No. 1146170 • Company No: 6771027 First edition published in 2005. Updated 2017. Sources for further information Because evidence matters