The protocol of our paper about reporting quality of pilot and feasibility studies in physiotherapy interventions is now released in open access. I will get this shared on our social media accounts too.
Best of wishes, Emma. If we used questionaire that have 3 section, should we run test separately or can run test together for the feasibility. Thank you for this interesting article. I have found it very useful in my research of Pilot Studies and has pointed me in the right direction for writing my dissertation.
If i am conducting a RCT then is it necessary to give interventions before conducting pilot study??? Conducting successful research requires choosing the appropriate study design. This article describes the most common types of designs conducted by researchers. What are the key steps in EBM?
Who are S4BE? What is a pilot study? Just a guy trying to be a researcher! Feel free to contact me at luiz. Leave a Reply Cancel reply Your email address will not be published. Good job! Amjad Pilot study is carried before starting work on final study. Khan sameer That was really helpful 25th February at pm Reply to Khan.
Emma Carter Great to hear Mike! Aminata odel Hi, my name is Aminata Odel. Siraj Crystal clear information. SHall be grateful 11th June at pm Reply to Siraj. Thanks 29th June at pm Reply to Petra. Emma Carter Hi Petra. Luiz Scola The protocol of our paper about reporting quality of pilot and feasibility studies in physiotherapy interventions is now released in open access. Shailendra yadav I joined the pilot 3rd August at am Reply to Shailendra.
Metrics details. Pilot studies for phase III trials - which are comparative randomized trials designed to provide preliminary evidence on the clinical efficacy of a drug or intervention - are routinely performed in many clinical areas. Also commonly know as "feasibility" or "vanguard" studies, they are designed to assess the safety of treatment or interventions; to assess recruitment potential; to assess the feasibility of international collaboration or coordination for multicentre trials; to increase clinical experience with the study medication or intervention for the phase III trials.
They are the best way to assess feasibility of a large, expensive full-scale study, and in fact are an almost essential pre-requisite. Conducting a pilot prior to the main study can enhance the likelihood of success of the main study and potentially help to avoid doomed main studies. The objective of this paper is to provide a detailed examination of the key aspects of pilot studies for phase III trials including: 1 the general reasons for conducting a pilot study; 2 the relationships between pilot studies, proof-of-concept studies, and adaptive designs; 3 the challenges of and misconceptions about pilot studies; 4 the criteria for evaluating the success of a pilot study; 5 frequently asked questions about pilot studies; 7 some ethical aspects related to pilot studies; and 8 some suggestions on how to report the results of pilot investigations using the CONSORT format.
Peer Review reports. The Concise Oxford Thesaurus [ 1 ] defines a pilot project or study as an experimental, exploratory, test, preliminary, trial or try out investigation. Epidemiology and statistics dictionaries provide similar definitions of a pilot study as a small scale. Table 1 provides a summary of definitions found on the Internet. A closer look at these definitions reveals that they are similar to the ones above in that a pilot study is synonymous with a feasibility study intended to guide the planning of a large-scale investigation.
Pilot studies are sometimes referred to as "vanguard trials" i. As suggested by an African proverb from the Ashanti people in Ghana " You never test the depth of a river with both feet ", the main goal of pilot studies is to assess feasibility so as to avoid potentially disastrous consequences of embarking on a large study - which could potentially "drown" the whole research effort.
Feasibility studies are routinely performed in many clinical areas. It is fair to say that every major clinical trial had to start with some piloting or a small scale investigation to assess the feasibility of conducting a larger scale study: critical care [ 5 ], diabetes management intervention trials [ 6 ], cardiovascular trials [ 7 ], primary healthcare [ 8 ], to mention a few.
Despite their noted importance, the reality is that pilot studies receive little or no attention in scientific research training.
Few epidemiology or research textbooks cover the topic with the necessary detail. In fact, we are not aware of any textbook that dedicates a chapter on this issue - many just mention it in passing or provide a cursory coverage of the topic.
The objective of this paper is to provide a detailed examination of the key aspects of pilot studies. In the next section, we narrow the focus of our definition of a pilot to phase III trials. Section 3 covers the general reasons for conducting a pilot study.
Section 4 deals with the relationships between pilot studies, proof-of-concept studies, and adaptive designs, while section 5 addresses the challenges of pilot studies. Evaluation of a pilot study i. We deal with several frequently asked questions about pilot studies in Section 7 using a "question-and-answer" approach.
Section 8 covers some ethical aspects related to pilot studies; and in Section 9, we follow the CONSORT format [ 9 ] to offer some suggestions on how to report the results of pilot investigations. Pilot studies can be conducted in both quantitative and qualitative studies. Adopting a similar approach to Lancaster et al. Phase I trials are non-randomized studies designed to investigate the pharmacokinetics of a drug i.
Phase II trials provide preliminary evidence on the clinical efficacy of a drug or intervention. They may or may not be randomized. Phase III trials are randomized studies comparing two or more drugs or intervention strategies to assess efficacy and safety. Phase IV trials, usually done after registration or marketing of a drug, are non-randomized surveillance studies to document experiences e. For the purposes of this paper, our approach to utilizing pilot studies relies on the model for complex interventions advocated by the British Medical Research Council - which explicitly recommends the use of feasibility studies prior to Phase III clinical trials, but stresses the iterative nature of the processes of development, feasibility and piloting, evaluation and implementation [ 11 ].
Van Teijlingen et al. Process: This assesses the feasibility of the steps that need to take place as part of the main study. Examples include determining recruitment rates, retention rates, etc. Resources: This deals with assessing time and budget problems that can occur during the main study. The idea is to collect some pilot data on such things as the length of time to mail or fill out all the survey forms.
Management: This covers potential human and data optimization problems such as personnel and data management issues at participating centres.
Scientific: This deals with the assessment of treatment safety, determination of dose levels and response, and estimation of treatment effect and its variance. Table 2 summarizes this classification with specific examples. A proof-of-concept PoC study is defined as a clinical trial carried out to determine if a treatment drug is biologically active or inactive [ 14 ]. PoC studies usually use surrogate markers as endpoints. Thus, although designed to inform the planning of phase III trials for registration or licensing of new drugs, PoC studies may not necessarily fit our restricted definition of pilot studies aimed at assessing feasibility of phase III trials as outlined in Section 2.
The adaptations can be prospective e. Piloting is normally built into adaptive trial designs by determining a priori decision rules to guide the adaptations based on cumulative data. For example, data from interim analyses could be used to refine sample size calculations [ 17 , 18 ]. This approach is routinely used in internal pilot studies - which are primarily designed to inform sample size calculation for the main study, with recalculation of the sample size as the key adaptation.
Unlike other phase III pilots, an internal pilot investigation does not usually address any other feasibility aspects - because it is essentially part of the main study [ 10 , 19 , 20 ].. Nonetheless, we need to emphasize that whether or not a study is a pilot, depends on its objectives.
An adaptive method is used as a strategy to reach that objective. Both a pilot and a non-pilot could be adaptive. Pilot studies can be very informative, not only to the researchers conducting them but also to others doing similar work. However, many of them never get published, often because of the way the results are presented [ 13 ].
Quite often the emphasis is wrongly placed on statistical significance, not on feasibility - which is the main focus of the pilot study.
Our experience in reviewing submissions to a research ethics board also shows that most of the pilot projects are not well designed: i. In many cases, pilot studies are conducted to generate data for sample size calculations.
This seems especially sensible in situations where there are no data from previous studies to inform this process. Therefore if not used cautiously, results of pilot studies can potentially mislead sample size or power calculations [ 21 ] -- particularly if the pilot study was done to see if there is likely to be a treatment effect in the main study.
In section 6, we provide guidance on how to proceed with caution in this regard. There are also several misconceptions about pilot studies.
Below are some of the common reasons that researchers have put forth for calling their study a pilot. The first common reason is that a pilot study is a small single-centre study. For example, researchers often state lack of resources for a large multi-centre study as a reason for doing a pilot.
The second common reason is that a pilot investigation is a small study that is similar in size to someone else's published study. In reviewing submissions to a research ethics board, we have come across sentiments such as. So-and-so did a similar study with 6 patients and got statistical significance - ours uses 12 patients double the size! The third most common reason is that a pilot is a small study done by a student or an intern - which can be completed quickly and does not require funding.
Specific arguments include. None of the above arguments qualifies as sound reasons for calling a study a pilot. A study should only be conducted if the results will be informative; studies conducted for the reasons above may result in findings of limited utility, which would be a waste of the researchers' and participants' efforts. The focus of a pilot study should be on assessment of feasibility, unless it was powered appropriately to assess statistical significance.
Further, there is a vast number of poorly designed and reported studies. Assessment of the quality of a published report may be helpful to guide decisions of whether the report should be used to guide planning or designing of new studies. Finally, if a trainee or researcher is assigned a project as a pilot it is important to discuss how the results will inform the planning of the main study.
In addition, clearly defined feasibility objectives and rationale to justify piloting should be provided. In general, sample size calculations may not be required for some pilot studies. It is important that the sample for a pilot be representative of the target study population. As a rule of thumb, a pilot study should be large enough to provide useful information about the aspects that are being assessed for feasibility.
Note that PoC studies require sample size estimation based on surrogate markers [ 22 ], but they are usually not powered to detect meaningful differences in clinically important endpoints.
The sample used in the pilot may be included in the main study, but caution is needed to ensure the key features of the main study are preserved in the pilot e. We recommend if any pooling of pilot and main study data is considered, this should be planned beforehand, described clearly in the protocol with clear discussion of the statistical consequences and methods. A good research strategy requires careful planning and a pilot study will often be a part of this strategy.
A pilot study is normally small in comparison with the main experiment and therefore can provide only limited information on the sources and magnitude of variation of response measures. It is unlikely, for example, that a pilot study alone can provide adequate data on variability for a power analysis to estimate the number of animals to include in a well designed experiment.
A systematic review of the literature or even a single publication is a more appropriate source of information on variability. The pilot study may, however, provide vital information on the severity of proposed procedures or treatments. A pilot study can address a number of logistical issues.
As part of the research strategy the following factors can be resolved prior to the main study:. The information obtained on logistical issues should be incorporated into the main study design.
As the purpose of a pilot study is to assess the feasibility of an experiment it is very rarely sensible to present more than summary statistics of the data. In fact, the data might be irrelevant if problems with the methods are discovered. If a pilot study does not lead to modification of materials or procedures then the data might be suitable for incorporation into the main study.
The sampling strategy used to select subjects, and the possibility of changes over time should be carefully considered before incorporating pilot data.
0コメント