Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that enables research into pragmatic trials. It collects and distributes clean trial data, ratings and evaluations using PRECIS-2. This allows for a variety of meta-epidemiological analyses to evaluate the effects of treatment across trials with different levels of pragmatism.
Background
Pragmatic trials are becoming more widely recognized as providing real-world evidence for clinical decision making. The term "pragmatic", however, is used inconsistently and its definition and assessment need further clarification. Pragmatic trials must be designed to inform policy and clinical practice decisions, rather than to prove an hypothesis that is based on a clinical or physiological basis. www.pragmatickr.com should strive to be as close to real-world clinical practice as is possible, including its participation of participants, setting up and design, the delivery and execution of the intervention, determination and analysis of outcomes as well as primary analysis. This is a major distinction between explanation-based trials, as defined by Schwartz and Lellouch1 that are designed to prove a hypothesis in a more thorough manner.
Truly pragmatic trials should not blind participants or clinicians. This could lead to bias in the estimations of treatment effects. The pragmatic trials also include patients from various healthcare settings to ensure that the results can be generalized to the real world.
Furthermore, pragmatic trials should focus on outcomes that are crucial to patients, such as quality of life or functional recovery. This is particularly important when it comes to trials that involve surgical procedures that are invasive or have potential serious adverse events. The CRASH trial29, for example focused on the functional outcome to compare a 2-page case-report with an electronic system for monitoring of patients admitted to hospitals with chronic heart failure, and the catheter trial28 utilized urinary tract infections that are symptomatic of catheters as the primary outcome.
In addition to these features pragmatic trials should reduce the trial procedures and data collection requirements in order to reduce costs. In the end these trials should strive to make their findings as applicable to current clinical practices as possible. This can be accomplished by ensuring that their primary analysis is based on the intention-to treat method (as described in CONSORT extensions).
Many RCTs which do not meet the criteria for pragmatism, however, they have characteristics that are in opposition to pragmatism, have been published in journals of varying types and incorrectly labeled pragmatic. This can result in misleading claims of pragmatism and the use of the term must be standardized. The development of a PRECIS-2 tool that can provide a standardized objective evaluation of the pragmatic characteristics is a good start.
Methods
In a practical study, the goal is to inform policy or clinical decisions by showing how an intervention can be integrated into routine treatment in real-world situations. Explanatory trials test hypotheses regarding the causal-effect relationship in idealized settings. In this way, pragmatic trials could have lower internal validity than explanation studies and be more susceptible to biases in their design, analysis, and conduct. Despite their limitations, pragmatic studies can provide valuable data for making decisions within the context of healthcare.
The PRECIS-2 tool evaluates an RCT on 9 domains, ranging between 1 and 5 (very pragmatic). In this study the areas of recruitment, organisation as well as flexibility in delivery flexibility in adherence, and follow-up received high scores. However, the main outcome and the method of missing data was scored below the pragmatic limit. This suggests that it is possible to design a trial that has excellent pragmatic features without damaging the quality of its outcomes.
It is hard to determine the amount of pragmatism that is present in a trial because pragmatism does not possess a specific characteristic. Some aspects of a research study can be more pragmatic than others. Furthermore, logistical or protocol modifications made during the trial may alter its score on pragmatism. Additionally, 36% of the 89 pragmatic trials discovered by Koppenaal and co. were placebo-controlled or conducted prior to approval and a majority of them were single-center. They are not in line with the usual practice and are only referred to as pragmatic if their sponsors accept that the trials aren't blinded.
Another common aspect of pragmatic trials is that researchers try to make their results more meaningful by analysing subgroups of the sample. However, this can lead to unbalanced comparisons with a lower statistical power, thereby increasing the risk of either not detecting or incorrectly detecting differences in the primary outcome. This was a problem during the meta-analysis of pragmatic trials due to the fact that secondary outcomes were not corrected for covariates that differed at the baseline.
Furthermore, pragmatic studies can pose difficulties in the gathering and interpretation of safety data. This is due to the fact that adverse events are usually self-reported and are susceptible to delays in reporting, inaccuracies or coding deviations. It is important to improve the quality and accuracy of outcomes in these trials.
Results
While the definition of pragmatism does not require that clinical trials be 100% pragmatic There are advantages of including pragmatic elements in trials. These include:
Enhancing sensitivity to issues in the real world which reduces study size and cost as well as allowing trial results to be faster implemented into clinical practice (by including patients from routine care). However, pragmatic trials may also have disadvantages. The right amount of heterogeneity for instance, can help a study expand its findings to different patients or settings. However the wrong kind of heterogeneity can reduce the sensitivity of an assay, and therefore lessen the power of a trial to detect small treatment effects.
Several studies have attempted to categorize pragmatic trials using various definitions and scoring methods. Schwartz and Lellouch1 developed an approach to distinguish between research studies that prove a physiological or clinical hypothesis as well as pragmatic trials that help in the choice of appropriate therapies in real-world clinical practice. The framework was composed of nine domains that were assessed on a scale of 1-5, with 1 being more informative and 5 being more pragmatic. The domains included recruitment of intervention, setting up, delivery of intervention, flexible adherence and primary analysis.
The original PRECIS tool3 was an adapted version of the PRECIS tool3 that was based on the same scale and domains. Koppenaal et al10 created an adaptation to this assessment, dubbed the Pragmascope that was simpler to use in systematic reviews. They found that pragmatic reviews scored higher in all domains, but scored lower in the primary analysis domain.
This difference in the main analysis domain could be explained by the fact that the majority of pragmatic trials analyse their data in an intention to treat method while some explanation trials do not. The overall score for systematic reviews that were pragmatic was lower when the areas of organisation, flexible delivery and follow-up were merged.
It is crucial to keep in mind that a pragmatic study does not mean that a trial is of poor quality. In fact, there is a growing number of clinical trials that use the word 'pragmatic,' either in their abstract or title (as defined by MEDLINE however it is neither sensitive nor precise). The use of these terms in abstracts and titles could indicate a greater understanding of the importance of pragmatism, however, it is not clear if this is manifested in the content of the articles.
Conclusions
In recent years, pragmatic trials are gaining popularity in research as the importance of real-world evidence is becoming increasingly acknowledged. They are randomized clinical trials that evaluate real-world alternatives to care instead of experimental treatments in development. They include patients that are more similar to the ones who are treated in routine care, they employ comparisons that are commonplace in practice (e.g. existing drugs) and depend on participants' self-reports of outcomes. This method can help overcome the limitations of observational research which include the biases associated with reliance on volunteers and limited availability and the variability of coding in national registry systems.
Other advantages of pragmatic trials are the possibility of using existing data sources, and a higher likelihood of detecting meaningful changes than traditional trials. However, these tests could be prone to limitations that undermine their reliability and generalizability. For instance the rates of participation in some trials may be lower than anticipated due to the healthy-volunteer influence and financial incentives or competition for participants from other research studies (e.g. industry trials). A lot of pragmatic trials are restricted by the need to recruit participants quickly. In addition certain pragmatic trials don't have controls to ensure that the observed differences aren't due to biases in trial conduct.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs that self-described themselves as pragmatic and that were published from 2022. They assessed pragmatism by using the PRECIS-2 tool, which includes the eligibility criteria for domains and recruitment criteria, as well as flexibility in adherence to intervention and follow-up. They discovered that 14 of these trials scored pragmatic or highly pragmatic (i.e., scoring 5 or higher) in one or more of these domains, and that the majority were single-center.
Trials with a high pragmatism rating tend to have broader eligibility criteria than traditional RCTs that have specific criteria that are unlikely to be found in the clinical environment, and they contain patients from a broad range of hospitals. The authors claim that these traits can make the pragmatic trials more relevant and applicable to everyday practice, but they do not guarantee that a trial using a pragmatic approach is free of bias. In addition, the pragmatism that is present in the trial is not a predetermined characteristic A pragmatic trial that doesn't possess all the characteristics of an explanatory trial can yield reliable and relevant results.