Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that facilitates research into pragmatic trials. It shares clean trial data and ratings using PRECIS-2 permitting multiple and varied meta-epidemiological research studies to compare treatment effects estimates across trials that have different levels of pragmatism as well as other design features.
Background
Pragmatic studies provide real-world evidence that can be used to make clinical decisions. However, the use of the term "pragmatic" is not consistent and its definition and assessment requires clarification. The purpose of pragmatic trials is to inform clinical practices and policy choices, rather than verify a physiological hypothesis or clinical hypothesis. A pragmatic trial should also try to be as similar to the real-world clinical environment as is possible, including its selection of participants, setting and design of the intervention, its delivery and implementation of the intervention, and the determination and analysis of outcomes as well as primary analyses. This is a key difference from explanatory trials (as described by Schwartz and Lellouch1), which are intended to provide a more complete confirmation of a hypothesis.
Truely pragmatic trials should not conceal participants or the clinicians. This could lead to a bias in the estimates of treatment effects. Pragmatic trials will also recruit patients from various healthcare settings to ensure that their results can be generalized to the real world.
Furthermore, pragmatic trials should focus on outcomes that are vital to patients, like quality of life or functional recovery. This is particularly important when it comes to trials that involve invasive procedures or those with potential dangerous adverse events. The CRASH trial29, for instance, focused on functional outcomes to evaluate a two-page case report with an electronic system for monitoring of patients in hospitals suffering from chronic heart failure. In addition, the catheter trial28 focused on urinary tract infections that are symptomatic of catheters as its primary outcome.
In addition to these characteristics pragmatic trials should also reduce the procedures for conducting trials and requirements for data collection to cut costs and time commitments. Additionally, pragmatic trials should seek to make their results as applicable to clinical practice as they can by making sure that their primary analysis is based on the intention-to-treat method (as described in CONSORT extensions for pragmatic trials).
Despite these guidelines, a number of RCTs with features that challenge the concept of pragmatism have been mislabeled as pragmatic and published in journals of all types. This can lead to false claims of pragmaticity and the usage of the term needs to be standardized. The development of a PRECIS-2 tool that offers a standardized objective assessment of pragmatic features is a good start.
Methods
In a pragmatic study, the goal is to inform policy or clinical decisions by showing how an intervention can be integrated into routine care in real-world settings. This is distinct from explanation trials, which test hypotheses about the causal-effect relationship in idealized conditions. In this way, pragmatic trials may have lower internal validity than explanatory studies and be more prone to biases in their design, analysis, and conduct. Despite these limitations, pragmatic trials can provide valuable information to decision-making in healthcare.
The PRECIS-2 tool assesses the degree of pragmatism within an RCT by scoring it across 9 domains, ranging from 1 (very explicative) to 5 (very pragmatic). In this study the areas of recruitment, organisation, flexibility in delivery, flexibility in adherence, and follow-up received high scores. However, the main outcome and method of missing data was scored below the pragmatic limit. This indicates that a trial can be designed with effective practical features, yet not damaging the quality.
It is difficult to determine the level of pragmatism in a particular trial since pragmatism doesn't have a single attribute. Some aspects of a study can be more pragmatic than others. Furthermore, logistical or protocol modifications during the course of the trial may alter its pragmatism score. In addition 36% of the 89 pragmatic trials discovered by Koppenaal and colleagues were placebo-controlled or conducted prior to approval and a majority of them were single-center. This means that they are not quite as typical and can only be called pragmatic when their sponsors are accepting of the lack of blinding in these trials.
A common feature of pragmatic studies is that researchers attempt to make their findings more meaningful by analyzing subgroups of the trial sample. This can lead to unbalanced comparisons with a lower statistical power, thereby increasing the likelihood of missing or misinterpreting differences in the primary outcome. In the case of the pragmatic trials that were included in this meta-analysis this was a significant problem because the secondary outcomes were not adjusted for differences in baseline covariates.
Furthermore, pragmatic trials can also have challenges with respect to the gathering and interpretation of safety data. This is due to the fact that adverse events are typically reported by participants themselves and are prone to delays in reporting, inaccuracies, or coding variations. It is therefore crucial to improve the quality of outcome ascertainment in these trials, ideally by using national registries instead of relying on participants to report adverse events in the trial's own database.
Results
While the definition of pragmatism doesn't require that all clinical trials are 100% pragmatic There are advantages when incorporating pragmatic components into trials. These include:
Incorporating routine patients, the trial results are more easily translated into clinical practice. However, pragmatic trials may also have disadvantages. The right type of heterogeneity for instance could help a study extend its findings to different settings or patients. However, the wrong type can decrease the sensitivity of the test and, consequently, lessen the power of a trial to detect minor treatment effects.
Several studies have attempted to categorize pragmatic trials using various definitions and scoring methods. Schwartz and Lellouch1 developed a framework to distinguish between explanatory studies that prove the physiological hypothesis or clinical hypothesis, and pragmatic studies that inform the choice for appropriate therapies in the real-world clinical practice. Their framework comprised nine domains, each scoring on a scale of 1-5, with 1 being more informative and 5 indicating more pragmatic. The domains covered recruitment, setting up, delivery of intervention, flexible adhering to the program and primary analysis.
프라그마틱 불법 tool3 featured similar domains and an assessment scale ranging from 1 to 5. Koppenaal and colleagues10 developed an adaptation to this assessment called the Pragmascope that was easier to use in systematic reviews. They found that pragmatic systematic reviews had a higher average score in most domains but lower scores in the primary analysis domain.
The difference in the primary analysis domains could be due to the way in which most pragmatic trials analyze data. Some explanatory trials, however, do not. The overall score for systematic reviews that were pragmatic was lower when the areas of organization, flexible delivery, and follow-up were merged.
It is important to remember that a pragmatic study does not necessarily mean a low-quality study. In fact, there are an increasing number of clinical trials that employ the term 'pragmatic' either in their abstracts or titles (as defined by MEDLINE however it is neither sensitive nor precise). The use of these terms in abstracts and titles could indicate a greater understanding of the importance of pragmatism, but it is unclear whether this is manifested in the contents of the articles.
Conclusions
In recent years, pragmatic trials are becoming more popular in research as the importance of real-world evidence is increasingly recognized. They are randomized trials that compare real world alternatives to new treatments that are being developed. They involve patient populations that are more similar to those who receive treatment in regular care. This approach has the potential to overcome the limitations of observational research which include the biases associated with reliance on volunteers and the lack of accessibility and coding flexibility in national registries.
Other advantages of pragmatic trials are the possibility of using existing data sources, as well as a higher probability of detecting significant changes than traditional trials. However, these tests could be prone to limitations that undermine their effectiveness and generalizability. The participation rates in certain trials could be lower than anticipated because of the healthy-volunteering effect, financial incentives or competition from other research studies. The need to recruit individuals in a timely manner also reduces the size of the sample and the impact of many practical trials. In addition some pragmatic trials do not have controls to ensure that the observed differences are not due to biases in trial conduct.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs that self-labeled themselves as pragmatic and were published up to 2022. They assessed pragmatism using the PRECIS-2 tool, which includes the eligibility criteria for domains, recruitment, flexibility in adherence to interventions, and follow-up. They found that 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or higher) in at least one of these domains.
Trials with high pragmatism scores are likely to have broader criteria for eligibility than traditional RCTs. They also have populations from various hospitals. According to the authors, may make pragmatic trials more relevant and useful in everyday practice. However they do not ensure that a study is free of bias. In addition, the pragmatism that is present in a trial is not a definite characteristic; a pragmatic trial that doesn't contain all the characteristics of a explanatory trial may yield valid and useful results.