The key metrics as seen by five industry experts
Partnerships at every stage of clinical trials are essential to the success of a study. However, measuring the performance of a particular partnership is not always easy. Which metrics and KPIs should be tracked to judge not just the performance of the CRO, but the performance of the combined effort? Here five industry experts offer their opinions.
“This is an aspect we are currently still defining rules for. In the end, there are two main goals – ensuring quality requirements are met and that agreed timelines are adhered to. When outsourcing, it is also important to ensure that the cooperation with the CRO improves or at most equals workload at the Sponsor compared to fully inhouse-ran trials. This is typically achieved by working with a preferred CRO that will thus be familiar with the company’s processes.”
“Relationship KPIs include data such as data currency, TMF completeness and quality, protocol deviation/violation but also costs difference between initial agreed budget and final one. Sometimes a performance survey is used both at the CRO and Sponsor and used as a ‘soft skills’ indicator of the relationship.”
“There are a number of approaches that could be used. At the Metrics Champion Consortium (MCC), we have developed a survey tool to be used between Sponsor and CRO/vendor along with associated metrics and recommended analysis. This can be used to provide a measure of the opinion across the partnership of how well it is working and could be monitored over time to establish any trends.
Over time, you would also expect that in a well-functioning partnership, behaviours would change. By defining what behaviours you expect to change, metrics could be developed to monitor these over time. For example: how often are risk assessments carried out jointly, do lessons from issues lead to updates to SOPs? Outcomes could also be measured, for example: is the volume of change orders reducing, is the number of critical quality issues reducing? By determining the questions you want to answer, metrics can be developed to help answer those questions. Organizations such as the MCC have metrics already defined that can assist.”
“Whilst metrics, timelines and KPIs are important to objectively measure performance of a clinical partner, the strength of the relationship and demonstrated flexibility to rapidly triage and adapt to the dynamic execution of an ongoing clinical trial are also critically important attributes for success. What metrics and KPIs should be measured?
KPIs should lend insight into status of project adherence and timeliness (projected vs actual) including: site activations, enrolment, data completion, query rates, protocol deviations and overall achievement of key milestones as examples. Not just performance of the CRO/vendor but the performance of the combined effort.”
“One of the most critical pieces of the combined effort is agreeing upfront on a realistic (and not too ambitious) plan regarding milestones and timelines (and cost). Creating such a plan will require time and resources on its own, but this is well invested.
The performance of the partnership can then be assessed by measuring whether the actual figures meet the plan with regards to Time, Budget and Quality – the eternal Triangle of Constraints. But this makes sense only if the plan was realistic in the first place, with buy-in from both partners.”
