HLTH 2024 Research Methods in Health

Module 3 Tutorial Autumn 2025

Learning objectives

• Skills in understanding a published report of a clinical


• Skills in critically appraising the reporting quality of a published report of a clinical


• Knowing what to do for the Assessment 1 Quantitative Research

This tutorial is the only opportunity for class discussion about the Quantitative Research Appraisal assessment. You should prepare for and attend this tutorial.

1. Prepare for this tutorial Reporting quality of a randomised controlled trial

Please read the complete journal article by Kloda et al. (2020).¹ The article is also on vUWS. This article is for practice only. Do not use the Kloda article for your actual assignment.

Also read the article for the Assessment 1 critical appraisal Chung et al. (2022), available as full text online or as PDF on vUWS. No other library research is needed. Use Chung et al. and no other article for the assignment.

1.1 Assignment practice in class

Have these documents ready to use. Better still, read them before class.

• Kloda et (2020).


• The Guide to Reporting Quality for Quantitative Research is developed for this Research Methods in Health subject. The Guide for Reporting Quality lists the criteria for critically appraising the reporting quality of a randomised These criteria are mostly based the CONSORT Statement (Schulz et al., 2010) along with concepts from other critical appraisal checklists see the Guide for more.

In class we shall do a pretend assignment using Kloda et al. (2020). Well evaluate the Kloda et al. articles reporting quality against the reporting-quality guide and consider the implications of that reporting quality for evidence-based practice. The aim is to see how well the article meets the criteria for good reporting as listed in the reporting quality guide, and to matching the reporting quality with evidence-based practice implications. The method used

¹ If the links dont work, get the article from vUWS or from the Library search line.

is critical appraisal. You will use the same methods to appraise the article for Assessment 1.

2. How do to the Quantitative Research Appraisal assessment

• Your tutor will answer questions about the marking criteria in the learning

• Your tutor will show the assignment template and how to complete
  
  
  
  • Open the assignment template and save it as a new
  
  
  • Delete instructions, advice and marks percentages to reduce the word
  
  
  
  

• There are six placeholders (single-celled tables) in the template to receive your text answers. The six placeholders match these sections of a typical research report.
  
  
  
  • Title
  
  
  • Abstract
  
  
  • Introduction
  
  
  • Methods
  
  
  • Results
  
  
  • Discussion
  
  
  
  

• All six placeholders have a dropdown, single response, rating-scale checklist to record a forced-choice answer for your evaluation of the reporting quality for that section. The rating scale answers range across: Good, Fair or Poor.


• The drop-down answer must be selected for a complete response. Neglecting to choose a rating-scale answer will limit the score for that section to 60% at most for a text-only answer that correctly evaluates reporting quality for that section.


• After selecting a rating-scale answer, justify that answer in free text by matching the article reporting quality to criteria in the reporting quality guide, including evaluation and evidence-based practice implications of that reporting quality.

Citations and referencing

Assignments should include only one citation and a reference list entry for the appraised article.

• Subsequent citations can be shortened, omitting the year or referring only to the the authors or the article rather than Author (Year) or (Author, Year) citations.

There is no need to cite the article repeatedly throughout your assignment. Markers know which article you are working with; you do not need to keep reminding them. Repeated citations increase your word-count without increasing your mark-count. Youre aiming to increase your mark count, so avoid using words that wont add marks.

Dont cite the article when youre expressing only your own opinion. In any assignment, citations are for what the authors of external sources said, not for your own, original ideas.

• Additional source material, beyond the supplied article and the reporting-quality guide is allowed but is unnecessary and not recommended. Additional source material must be properly cited and referenced using APA 7 style. Additional sources with their citations and references add to the word count, so try to avoid using them.


• Do not cite or reference the reporting-quality guide unless you are directly quoting from it, which there is no need to do. Rely on the supplied article (cited) and the reporting-quality guide (not cited) to write your entire assignment.

Screenshot of assignment template showing drop-down rating scale

An answer must be selected for every section with a dropdown list box.

A seventh placeholder is for the reference list. There is no dropdown answer.

The 1,200-word limit includes all document text as counted by Turnitin. Assignments longer than 1,200 words are liable to receive excess word penalties in addition to late penalties, if any. See the learning guide for more information.

Guide to the dropdown rating scale:

• Good reporting: The reporting for this section of the article is complete or very nearly complete. Good reporting provides mostly explicit rather than implicit information. Reporting is well organised, clear, concise and consistent with other information in the article. All or nearly all essential reporting requirements to inform evidence-based practice are met. Evidence-based practitioners can readily evaluate the studys relevance and importance, its methodological quality, its results and conclusions, internal or external validity or applicability. Ability to replicate the study from the methods description is another indicator of reporting quality. If other researchers could use the methods section to repeat the study using procedures that are identical to those of the original study, that is good quality reporting. Within the word-count for this article, it would be difficult to improve reporting quality.

• Fair reporting: Whilst some essential reporting requirements are met, there are important reporting-quality requirements that have not been met, or the presentation is unclear or could be better organised, or the information is often implicit rather than explicit. Whilst some of the reporting is satisfactory, problems with reporting quality impede an evidence-based practitioners ability to evaluate major aspects of relevance and importance of the research, the studys methodological quality, or its results or conclusions, or the studys internal or external validity (generalisability) or applicability. With fair reporting, enough detail is provided to repeat the original studys sampling, observations, including measurements, interventions, if any, and analysis with reasonable fidelity but without assurance of exact duplication. Within the word-count of this article, reporting quality could be moderately improved.


• Poor reporting: Few, if any, essential reporting requirements are met. Multiple important reporting-quality criteria are omitted or inadequately described. Pervasive problems with reporting quality impede an evidence-based practitioners ability to evaluate major aspects of relevance and importance of the research, the studys methodological quality, or its results or conclusions, or its internal or external validity or its applicability. Omission of at least some of the original studys method and procedures that are critical to replication, and internal and external validity such as sampling, observations, including measurements, interventions, if any, and analysis make genuine repetition nonviable. Keeping to the word length of this article, it is possible to substantially improve reporting quality.

Points to note about high scoring assignments:

• For each section of the article, there is consistency between:
  
  
  
  • Your dropdown list-box
  
  
  • Your critical appraisal in
  
  
  • The section of the journal article you are critically
  
  
  • The Guide to Reporting Quality for Quantitative Research
  
  
  
  

• There is brief, accurate summary of the article relevant to the reporting quality you are appraising.


• There is evaluation of the articles reporting quality in relation to the guide and your dropdown list answer. Your text answer goes beyond just summarising the article.


• Implications for evidence-based practice are
  
  
  
  • Evidence-based practice implications are essential for a Distinction or
  
  
  
  

• Abbreviations are defined, such as evidence-based practice (EBP).

There is no single, correct way of evaluating an article. There are different ways of getting it right. High scoring assignments may emphasise different aspects of reporting quality.

Nevertheless, some answers are better than others.

Dot-point answers are allowed only for a genuine list, not instead of prose paragraphs. Complete sentences in paragraphs are expected for most of your free text. Do not present your assignment as a set of dot-point notes. Presentation and prose style are assessable.

Quick guide to reporting quality

• Good reporting makes the information explicit, meaning plain to Complete and clear information is provided by the authors. Readers can find the information with minimal effort. The reporting quality for this section assists the evidence-based practitioner to evaluate the studys aims and relevance, its methodological quality, the results internal or external validity and the conclusions generalisability and applicability. For example, if it is explicitly stated that no participants dropped out of the study so that all participants who started the study also completed the study, that good reporting saves readers from rummaging through article to compare numbers of participants at the study outset with the sample size at completion, per group. The authors have clearly stated that all participants completed the study, with no changes to the sample size or composition through the project. A high attrition rate threatens internal validity. Attrition is therefore relevant to evidence-based practice and should be reported explicitly.


• Lower quality reporting makes the information implicit or incomplete. Facts are available but the reader may search hard or do their own reasoning to find the facts. Partial information only may be provided. For example, authors reporting the number of participants recruited into the study in the Methods and the number of participants completing the study in the Results enables readers to determine how many participants withdrew from the study and the attrition rate, but the reader must calculate these numbers and the rate rather than easily read them. Numbers withdrawing and the attrition rate are implied rather than explicitly stated, which is not good reporting.


• Poor quality reporting has important information absent altogether. The reader cannot get the information from the article at all. For example, it could be impossible to determine numbers lost to the study or the attrition rate from any information in an

Your own assignment will be marked the same way and according to the marking criteria. In terms of your assignments reporting quality:

• You are doing well if you make clear, explicit statements of important facts and persuasive Its easy for markers to read and understand your work. Your work matches all marking criteria.


• Lower quality answers contain mainly incomplete or implicit rather complete and directly stated information. Markers are forced to work harder so they can understand what you are trying to say. Your answers omit evidence-based practice.


• Poor quality reporting omits important information about the article and relevant to evidence-based practice, or it includes errors of fact or its impossible to understand whats written.

Assignment grading criteria

Marking rubric in subject learning guide has more detail.

• Pass = competent descriptive summary of article materials relevant to the drop-down answer and important for evidence-based practice but may lack deeper evaluation in the text answer.


• Credit = competent descriptive summary of relevant and important article content plus further evaluation in the text answer matching the drop-down answer but may lack evidence-based practice implications.


• Distinction or High Distinction = competent descriptive summary of relevant and important article content plus further evaluation in the text answer matching the drop- down answer plus evidence-based practice implications.

3. Guide to Reporting Quality for Quantitative Research Assessment 1

Based on published reporting quality checklists:

• CONSORT Statement (Schulz et 2010).


• Critical Appraisal Skills Programme (2020).


• global (2020).

You should use only the reporting quality guide on vUWS to write your assignment.

4. Reporting quality appraisal practice Kloda et al. (2020)

Title and Abstract are done for you below. Note what you should do in your assignment:

• Cite the article Do not cite vUWS materials.


• Briefly describe the part of the article you are


• Evaluate the reporting Dont only describe the article (Pass level).


• Say what is well reported as well as what could be reported better. Dont only criticise. All articles report some things properly.


• For a higher score, mention the implications of the articles reporting quality for evidence-based practice.


• Avoid discussing the scientific quality of the Stick to reporting quality.


• Within each section, the ordering of information in the article matched to the order of reporting-quality criteria in the guide is not important and should not be evaluated.

The appraisal below has far more detail than you can put in your assignment. In your assignment, you need to be selective about the criteria you address in detail: Include criteria with the most importance for evidence-based practice for that article. Evidence-based practice implications are omitted from the answers below. Instead, consider the implications in class discussion: internal and external validity, generalisability (similar to external validity) and applicability among others.

4.1 Title

Fair. The title of Kloda et al. (2020) identifies the study as a randomised trial, making it easier for evidence-based practitioners (EBPs) searching titles for randomised trials to find this article. The sample should have been described as physiotherapy and occupational therapy students.

4.2 Abstract

Fair. The Abstract partly conforms to criteria for good reporting. The required structured summary refers to the trial objective, design and interventions for experimental and control conditions. Group sizes, attrition and blinding are omitted from the Abstract Methods. The Abstract Methods mentions some outcomes but only in minimal detail. Reporting of methods in the Abstract is therefore deficient.

Abstract results are presented as text without statistical summaries that would enable numerical comparisons or show the precision of population estimates via confidence intervals. Statistically significant improvements for both groups on one measure are listed in text, along with the lack of statistical significance between the two groups on any outcomes. No indication of actual level of performance is provided.

The brief conclusions match the results. Whilst the interventions are adequately described, the lack of numerical findings or performance measurements prevents EBPs from reaching firm conclusions about the absolute or comparative effectiveness of either search strategy tested in this experiment.

Attempt your own answers for the Introduction, Methods, Results and Discussion sections.

• Introduction


• Methods


• Results


• Discussion


• General comments about the Kloda et (2020) article

Optional reading about the article, not specifically about the assignment or reporting quality.

The study describes in detail an innovative approach to teaching academic search skills and compares students skills after instruction with students receiving ordinary tuition. On a range of outcome measures, descriptive effects between groups vary widely in Tables 2, 3 and 4 but none of these effects translates to statistical significance or persuasive evidence of superiority for the alternative framework. The study reports only partly random allocation, considerable attrition, and possible collaboration between students in treatment and control groups, which could have biased results, either reducing or increasing sample effects. Measurement limitations are admitted as well. The research provides little convincing evidence for or against the treatment over existing methods, even if the unreported additional costs of implementing the advanced frameworks are minimal.

5. Question and answer on the assignment article

Any remaining time.

Question and answer session about the assignment article. Tutors will not match the assignment article to reporting quality criteria, checklist answers or in other ways provide answers to the assignments. Class questions will be answered to assist your understanding of the assignment article without doing the assignment for you.

Questions about how to do the assignment are also welcome. By the end of the class time, be sure that you know exactly what to do.