Appendix 4.1:
Additional Journal Article Reporting Standards for Basic Quantitative Research Designs

Table A4.1. Reporting Standards for Studies With an Experimental Manipulation (in Addition to Material Presented in Table A1.1)

Paper section and topic

Description

Method

Experimental manipulations

Provide details of the experimental manipulations intended for each study condition, including comparison conditions, and how and when experimental manipulations were actually administered, including

  • Content of the specific experimental manipulations (if experimental manipulation is part of a clinical trial, address Table A6.4)
    • Summary or paraphrasing of instructions, unless they are unusual or compose the experimental manipulation, in which case they may be presented verbatim
  • Method of experimental manipulation delivery
    • Description of apparatus and materials used (e.g., specialized equipment by model and supplier) and their function in the experiment
  • Deliverers: who delivered the experimental manipulations
    • Level of professional training
    • Level of training in specific experimental manipulations
  • Number of deliverers and, in the case of experimental manipulations, the M, SD, and range of number of individuals or units treated by each
  • Setting: where the manipulations or experimental manipulations occurred
  • Exposure quantity and duration: how many sessions, episodes, or events were intended to be delivered and how long they were intended to last
  • Time span: how long it took to deliver the experimental manipulation to each unit
  • Activities to increase compliance or adherence (e.g., incentives)
  • Use of a language other than English and translation method
  • Sufficient detail to allow for replication, including reference to or a copy of the manual of procedures (If the manual of procedures is available, how others may obtain it).

Units of delivery and analysis

State the unit of delivery (how participants were grouped during delivery).

Describe the smallest unit that was analyzed (and in the case of experiments, that was randomly assigned to conditions) to assess experimental manipulation effects (e.g., individuals, work groups, classes).

If the unit of analysis differed from the unit of delivery, describe the analytic method used to account for this (e.g., adjusting the standard error estimates by the design effect or using multilevel analysis).

Results

Participant flow

Report the total number of groups (if the experimental manipulation was administered at the group level) and the number of participants assigned to each group, including

  • Number of participants approached for inclusion
  • Number of participants who began the experiment
  • Number of participants who did not complete the experiment or crossed over to other conditions, with reasons
  • Number of participants included in primary analyses.

Include a figure describing the flow of participants through each stage of the study (see Figure 5.1).

Treatment fidelity

Provide evidence on whether the experimental manipulation was implemented as intended.

Baseline data

Describe baseline demographic and clinical characteristics of each group.

Adverse events and side effects

Report all important adverse events or side effects in each experimental condition. If none, state so.

Discussion

Discuss results, taking into account the mechanisms by which the experimental manipulation was intended to work (causal pathways) or alternative mechanisms.

Discuss the success of, and barriers to, implementing the experimental manipulation and the fidelity of implementation if an experimental manipulation was involved.

Discuss generalizability (external validity and construct validity) of the findings, taking into account

  • Characteristics of the experimental manipulation
  • How and what outcomes were measured
  • Length of follow-up
  • Incentives
  • Compliance rates.

Describe the theoretical or practical significance of outcomes and the basis for these interpretations.

Note. Adapted from “Journal Article Reporting Standards for Quantitative Research in Psychology: The APA Publications and Communications Board Task Force Report,” by M. Appelbaum, H. Cooper, R. B. Kline, E. Mayo-Wilson, A. M. Nezu, and S. M. Rao, 2018, American Psychologist, 73, pp. 11–12. Copyright 2018 by the American Psychological Association.

Table A4.2a. Reporting Standards for Studies Using Random Assignment (in Addition to Material Presented in Table A1.1)

Paper section and topic

Description

Method

Random assignment method

Describe the unit of randomization and the procedure used to generate the random assignment sequence, including details of any restriction (e.g., blocking, stratification).

Random assignment implementation and concealment

State whether and how the sequence was concealed until experimental manipulations were assigned, including who

  • Generated the assignment sequence
  • Enrolled participants
  • Assigned participants to groups.

Masking

Report whether participants, those administering the experimental manipulations, and those assessing the outcomes were aware of condition assignments.

Provide a statement regarding how any masking (if it took place) was accomplished and whether and how the success of masking was evaluated.

Statistical methods

Describe statistical methods used to compare groups on primary outcomes.

Describe statistical methods used for additional analyses, such as subgroup comparisons and adjusted analysis.

Describe statistical methods used for mediation or moderation analyses if conducted.

Note. Adapted from “Journal Article Reporting Standards for Quantitative Research in Psychology: The APA Publications and Communications Board Task Force Report,” by M. Appelbaum, H. Cooper, R. B. Kline, E. Mayo-Wilson, A. M. Nezu, and S. M. Rao, 2018, American Psychologist, 73, p. 12. Copyright 2018 by the American Psychological Association.

Table A4.2b. Reporting Standards for Studies Using Nonrandom Assignment (in Addition to Material Presented in Table A1.1)

Paper section and topic

Description

Method

Assignment method

Report the unit of assignment (i.e., the unit being assigned to study conditions; e.g., individual, group, community).

Describe the method used to assign units to study conditions, including details of any restriction (e.g., blocking, stratification, minimization).

State procedures used to help minimize selection bias (e.g., matching, propensity score matching).

Masking

Report whether participants, those administering the experimental manipulation, and those assessing the outcomes were aware of condition assignments.

Report whether masking took place. Provide a statement regarding how it was accomplished and how the success of masking was evaluated, if it was evaluated.

Statistical methods

Describe statistical methods used to compare study groups on primary outcomes, including complex methods for correlated data.

Describe statistical methods used for any additional analyses conducted, such as subgroup analyses and adjusted analysis (e.g., methods for modeling pretest differences and adjusting for them).

Describe statistical methods used for mediation or moderation analyses if used.

Note. Adapted from “Journal Article Reporting Standards for Quantitative Research in Psychology: The APA Publications and Communications Board Task Force Report,” by M. Appelbaum, H. Cooper, R. B. Kline, E. Mayo-Wilson, A. M. Nezu, and S. M. Rao, 2018, American Psychologist, 73, p. 14. Copyright 2018 by the American Psychological Association.

Table A4.3. Reporting Standards for Studies Using No Experimental Manipulation (e.g., Single-Case Designs, Natural-Group Comparisons; in Addition to Material Presented in Table A1.1)

Paper section and topic

Description

Abstract

Study design

Describe the design of the study.

Data use

State the type of data used.

Method

Participant selection

Describe the methods of selecting participants (i.e., the units to be observed or classified, etc.), including

  • Methods of selecting participants for each group (e.g., methods of sampling, place of recruitment) and number of cases in each group
  • Matching criteria (e.g., propensity score) if matching was used.

Identify data sources used (e.g., sources of observations, archival records), and if relevant, include codes or algorithms used to select participants or link records.

Variables

Define all variables clearly, including

  • Exposure
  • Potential predictors, confounders, and effect modifiers.

State how each variable was measured.

Comparability of assessment

Describe comparability of assessment across groups (e.g., the likelihood of observing or recording an outcome in each group for reasons unrelated to the effect of the intervention).

Analysis

Describe how predictors, confounders, and effect modifiers were included in the analysis.

Discussion

Limitations

Describe potential limitations of the study. As relevant, describe the possibility of misclassification, unmeasured confounding, and changing eligibility criteria over time.

Note. Adapted from “Journal Article Reporting Standards for Quantitative Research in Psychology: The APA Publications and Communications Board Task Force Report,” by M. Appelbaum, H. Cooper, R. B. Kline, E. Mayo-Wilson, A. M. Nezu, and S. M. Rao, 2018, American Psychologist, 73, p. 11. Copyright 2018 by the American Psychological Association.