The eCTD summit

Dr. Sarah Connelly, FDA clinical reviewer and Medical Officer, Division of Antiviral Products, presented some interesting and useful survey results at the DIA Annual Meeting on June 16th.  The survey gathered information on FDA reviewers’ perception of submission quality in a number of specific areas.  It was a collaboration between FDA and the DIA Medical Writers SIAC.

The survey is hot off the presses, having been completed in early June.  It was designed to assess:

• Quality of writing
• Organization
• Completeness of presented information
• Adequacy of cross-referencing

Responses were tabulated specifically in the CMC/Quality, Nonclinical and Clinical areas.  About 53 reviewers responded.

CMC

In CMC, overall writing quality was rated fairly highly: at about 12% poor or fair, 42% average, and 46% good or excellent.  However, 17% judged organization or information as poor or fair, and 31% considered completeness of information poor or fair.  The worst ratings were given to adequacy of cross-references (with 38% rating fair or poor) and adequacy of bookmarks and bookmark names (32% rating fair or poor).  .

In the area of M2 summaries, 19% rated quality as fair or poor, 52% as average, and 29% as good or excellent.

In comments, reviewers highlighted the following issues:

• Documents that lack cohesion and that are sometimes just collections of uninterrupted data
• Too many small files with little content (e.g., excipient documents)
• Poor leaf titles

Nonclinical

In the nonclinical area, overall writing quality was rated at about 28% poor or fair, 34% average, and 37% good or excellent.  Somewhat higher ratings were given to organization and completeness of information.  The worst ratings again were given to adequacy of cross-references (with 46% rating fair or poor) and adequacy of bookmarks and bookmark names (52% rating fair or poor).  This is not surprising considering the quantity of scanned documents presented by sponsors in nonclinical, and the reluctance of sponsors to spend time enhancing the navigation aids in these documents.

In the area of M2 summaries, 29% rated quality as fair or poor, 40% as average, and 31% as good or excellent.

In comments, reviewers highlighted the following issues:

• Repetition: often paragraphs just re-state what is shown in tables with no useful interpretation.  Often the same data will or information will be reproduced in a number of places
• Uninterrupted raw data: summary data and interpretation is often inadequate or not provided at all
• Lack of context and flow: regulatory background for nonclinical program not discussed, missing reports not explained

Clinical

In the clinical area, ratings were generally higher – not surprising again as most sponsors have a larger budget and an in-house or contracted staff of professional medical writers producing clinical documents.  Quality of clinical studies in general was reasonably good, with 22% considering reports poor or fair, 36% average, and 42% poor.

Overall writing quality was rated at about 15% poor or fair, 35% average, and 43% good or excellent.  Similar ratings were given to completeness of information, and  few reviewers had complaints about organization (possibly due to the adoption of E3 as a standard for clinical study reports).  The worst ratings were given to adequacy of cross-references (with 27% rating fair or poor) and adequacy of bookmarks and bookmark names (35% rating fair or poor).

In the area of M3 summaries, 17% rated quality as fair or poor, 39% as average, and 42% as good or excellent.  ISS and ISE were a little less satisfactory.

In comments, clinical reviewers mentioned similar issues repetition (including cutting and pasting into summaries), lack of interpretation, non-searchable documents, and inadequate narrative summaries.  They also mentioned lack of hyperlinks and difficulty in finding information.  On a somewhat harsher note, some reviewers believe that the people writing clinical summaries don’t understand the science.

Summary

There are certainly some significant trends in the feedback from reviewers – poor quality bookmarking and hyperlinking, repetition, and lack of summary and interpretation being the most consistent.  Something that struck me was the lack of “excellent” ratings – only one area reached as high as 10% and most were well below that.  I interpret that as meaning almost all sponsors have some room for improvement.   This is a great opportunity to review your practices and see where you might not be meeting the reviewers’ expectations.  After all, impeding the reviewers in making their assessments is a major contributor to complete response.

I’m going to deviate from my usual approach of not mentioning my company’s product s and services to mention that our soon-to-be-released GlobalSubmit 2010 release has significant enhancements in support of the quality of PDF documents, bookmarks and hyperlinks.  Not only will we identify over 40 PDF-related errors at the validation stage, but our new CrossCheck^TM product provides a revolutionary way to QC bookmarks and hyperlinks without having to open target documents.  Our clients who have previewed CrossCheck view it as a real productivity boost.  For further information or a demo, contact sales@globalsubmit.com.

Finally, Dr. Connolly’s presentation, CTD/eCTD Quality: FDA Survey Results is available on GlobalSubmit’s presentation page.