
Today’s post is a round-up of recent eSubmission-related news from the agencies, based on posts from my twitter account, www.twitter.com/kathie_clark. Next time, a roundup of articles and conferences.
The regulatory authorities haven’t put out much in the way of major updates recently, but there have been some updates worth noting:
The individual agencies in Europe have also issued some updates:
In my document management career, my company provided specialized EDMS solutions for Pharma (for submissions, SOPs, trial master files, etc.). It was a given that these systems needed to be Part 11 compliant, and I authored our position paper on how Part 11 compliance was achieved and demonstrated in our product. If our product had ever been viewed as being noncompliant, no one would buy it – end of story.
For whatever reason, few sponsors seemed to apply that same requirement to their publishing vendors. Back in the days of paper submissions, this didn’t seem too unreasonable. After all, the FDA says in Guidance for Industry: Part 11, Electronic Records; Electronic Signatures — Scope and Application”:
“Under the narrow interpretation of the scope of part 11, with respect to records required to be maintained under predicate rules or submitted to FDA, when persons choose to use records in electronic format in place of paper format, part 11 would apply. On the other hand, when persons use computers to generate paper printouts of electronic records, and those paper records meet all the requirements of the applicable predicate rules and persons rely on the paper records to perform their regulated activities, FDA would generally not consider persons to be “using electronic records in lieu of paper records…”
But eCTDs don’t meet these criteria. By definition, when you submit an eCTD you are asking the government to accept “in electronic format in place of paper”- and as part of the conditions of acceptance, they state that “Electronic documents that bypass the controls for electronic files described in 21 CFR 11 are not considered official documents for review.” (see Guidance for Industry: Providing Regulatory Submissions in Electronic Format — General Considerations.”)
Despite this, I don’t hear a lot of talk about Part 11 and publishing tools. Vendors don’t post much on their websites about Part 11 compliance (and what they have often applies more to document management). Googling Part 11 and electronic publishing doesn’t return anything useful.
So where does that leave sponsors in terms of the need for compliance? Granted that in these days of McNeil plant closures, tainted heparin, accusations of suppressed safety data, and etc., it seems unlikely that the FDA’s risk-based approach to inspections and audits will result in scrutiny of sponsors’ publishing systems. But sponsors certainly don’t intend to entirely flout requirements or fail to follow good practices. It would be interesting to define what the essential elements of compliance and good practices are.
Some areas are straightforward. Most people would agree that their publishing tools should have limited access and operational controls to prevent errors. But other areas aren’t so clear. To add to the ambiguity, in the Scope and Application document, the FDA announced its decision to exercise “enforcement discretion”, specifically lack of intention to take enforcement action to enforce compliance with the validation, audit trail, record retention, and record copying requirements of Part 11.
Audit trail almost always remains one of the most complicated issues for electronic systems – so given the “enforcement discretion” what needs to be audit trailed in a publishing system?
As a reminder, Part 11 applies to “Records that are required to be maintained under predicate rule requirements and that are maintained in electronic format in place of paper format.” In the document management world, almost everyone interprets this as meaning that draft documents do not require an audit trail, as you aren’t required to create or retain them at all in most cases. When you are publishing, everything is a draft, right up until you lock things down and send them through the gateway. When you are working on a sequence, if you add a specification, then remove it, is an audit trail needed? If so, is it because Part 11 requires it, or because the sponsor wants an “electronic watchdog” reporting on who made changes to the submission prior to finalization?
Many other Part 11 requirements should be important to the sponsor regardless of whether they are subject to inspection. I would want my electronic publishing system to be installed in accordance with an IQ protocol and validated with an OQ and PQ protocol regardless of whether they FDA requires this or not, because incorrect installation or operation could have a profound impact on my submission quality or timeliness.
I would be interested in hearing from my readers on this subject. Do you disagree with the assessment that Part 11 applies to publishing software? What do you consider the necessary elements for Part 11 compliance for your publishing tool? Is Part 11 the “elephant in the room” that no one will acknowledge related to publishing software, or addressed by following basic good practices?
As I discussed in my previous post, FDA issued version 1.5.1 of Study Data Specifications in January of 2010. Although the changes in the new version were not great, they were significant for eCTD as a new specification for organizing datasets was defined. The change has to do with how the files are organized in the file structure in which an eCTD is delivered to indicate whether they were STDM or legacy format. However, the change had no effect on the Table of Contents that is visible to a reviewer (created from the eCTD XML backbone). In a recent development, ICH M2 working group has defined an update to the valid values list for study tagging files to introduce file tags that would allow a review tool to identify dataset types when displaying the eCTD Table of Contents.
The new specifications for organizing study datasets and their associated files in folders are summarized in the following figure. In the document, an accompanying table provides further definition of the folders.
New Dataset Folder Organization
Compare this with the folder structure required in the previous specification:
Previous Dataset Folder Organization
You will see an important difference: tabulations must be assigned to a folder based on whether they are STDM datasets or legacy format. A second difference is that there is now a separate folder for datasets under analysis, where previously analysis datasets were placed directly in the analysis folder.
At the recent ICH meeting in Estonia, the valid-values.xml that defines file tags used to identify the documents in studies was amended to better support the different types of data:
Look for the new valid-values.xml on the ICH websites in a zip file. You should talk with your publishing vendor to find out how and when they will support the updated specification.
One final note – the FDA recently retired their own version of the valid-values.xml and reverted to the ICH version. That means that they restored the file tag of nonclinical-data. However, the preferred file tag for a nonclinical study is pre-clinical-study-report.
Dr. Sarah Connelly, FDA clinical reviewer and Medical Officer, Division of Antiviral Products, presented some interesting and useful survey results at the DIA Annual Meeting on June 16th. The survey gathered information on FDA reviewers’ perception of submission quality in a number of specific areas. It was a collaboration between FDA and the DIA Medical Writers SIAC.
The survey is hot off the presses, having been completed in early June. It was designed to assess:
Responses were tabulated specifically in the CMC/Quality, Nonclinical and Clinical areas. About 53 reviewers responded.
CMC
In CMC, overall writing quality was rated fairly highly: at about 12% poor or fair, 42% average, and 46% good or excellent. However, 17% judged organization or information as poor or fair, and 31% considered completeness of information poor or fair. The worst ratings were given to adequacy of cross-references (with 38% rating fair or poor) and adequacy of bookmarks and bookmark names (32% rating fair or poor). .
In the area of M2 summaries, 19% rated quality as fair or poor, 52% as average, and 29% as good or excellent.
In comments, reviewers highlighted the following issues:
Nonclinical
In the nonclinical area, overall writing quality was rated at about 28% poor or fair, 34% average, and 37% good or excellent. Somewhat higher ratings were given to organization and completeness of information. The worst ratings again were given to adequacy of cross-references (with 46% rating fair or poor) and adequacy of bookmarks and bookmark names (52% rating fair or poor). This is not surprising considering the quantity of scanned documents presented by sponsors in nonclinical, and the reluctance of sponsors to spend time enhancing the navigation aids in these documents.
In the area of M2 summaries, 29% rated quality as fair or poor, 40% as average, and 31% as good or excellent.
In comments, reviewers highlighted the following issues:
Clinical
In the clinical area, ratings were generally higher – not surprising again as most sponsors have a larger budget and an in-house or contracted staff of professional medical writers producing clinical documents. Quality of clinical studies in general was reasonably good, with 22% considering reports poor or fair, 36% average, and 42% poor.
Overall writing quality was rated at about 15% poor or fair, 35% average, and 43% good or excellent. Similar ratings were given to completeness of information, and few reviewers had complaints about organization (possibly due to the adoption of E3 as a standard for clinical study reports). The worst ratings were given to adequacy of cross-references (with 27% rating fair or poor) and adequacy of bookmarks and bookmark names (35% rating fair or poor).
In the area of M3 summaries, 17% rated quality as fair or poor, 39% as average, and 42% as good or excellent. ISS and ISE were a little less satisfactory.
In comments, clinical reviewers mentioned similar issues repetition (including cutting and pasting into summaries), lack of interpretation, non-searchable documents, and inadequate narrative summaries. They also mentioned lack of hyperlinks and difficulty in finding information. On a somewhat harsher note, some reviewers believe that the people writing clinical summaries don’t understand the science.
Summary
There are certainly some significant trends in the feedback from reviewers – poor quality bookmarking and hyperlinking, repetition, and lack of summary and interpretation being the most consistent. Something that struck me was the lack of “excellent” ratings – only one area reached as high as 10% and most were well below that. I interpret that as meaning almost all sponsors have some room for improvement. This is a great opportunity to review your practices and see where you might not be meeting the reviewers’ expectations. After all, impeding the reviewers in making their assessments is a major contributor to complete response.
I’m going to deviate from my usual approach of not mentioning my company’s product s and services to mention that our soon-to-be-released GlobalSubmit 2010 release has significant enhancements in support of the quality of PDF documents, bookmarks and hyperlinks. Not only will we identify over 40 PDF-related errors at the validation stage, but our new CrossCheckTM product provides a revolutionary way to QC bookmarks and hyperlinks without having to open target documents. Our clients who have previewed CrossCheck view it as a real productivity boost. For further information or a demo, contact sales@globalsubmit.com.
Finally, Dr. Connolly’s presentation, CTD/eCTD Quality: FDA Survey Results is available on GlobalSubmit’s presentation page.
At a GPhA/FDA Labeling Workshop on April 14, 2010, Virginia Ventura of CDER’s Office of Business Informatics presented CDER Update eCTD & Gateway Submissions. Along with lots of other useful information, Ms. Ventura provided an assessment of the current Top Processing Issues with Esubs:
|
Description |
Percent of total |
|
Unable to extract info from PDF form (sponsor did not use fillable form) |
52% |
|
Missing form (356h, 1571 or 2252) |
13% |
|
Bad Characters (per eCTD Spec) in file or folder names |
10% |
|
Duplicate sequence (sponsor sent twice) |
10% |
|
“High” validation errors |
7% |
|
Media empty or corrupted |
4% |
|
Wrong application number in us-regional.xml |
2% |
|
Invalid submission type identified (eCTD submitted as non-eCTD or vice versa) |
1% |
|
Valid application number with incorrect type (combination is valid but not Sponsor’s application) |
1% |
Certainly, the high percentage of submissions not using fillable forms is startling – Ms. Ventura mentioned that most of these are ANDAs. She spells out the solution for using 356h fillable forms (including their correct use with annual reports) – consult the presentation for details. She also stresses that the correct use of fillable forms results in your submission getting to the reviewer much quicker (best case an amazing 10 minutes!).
The amount of High Severity validation errors is also concerning, as the presentation reiterates what the FDA has said before - High Errors = Submission cannot be accepted. Therefore, up to 7% of these submissions may have been rejected, resulting in time off the PDUFA clock in many cases.
All of this serves as a wake-up call that validation and submission QC are more important than ever. Be sure to read the presentation for more tips on STFs, QC of us-regional.xml, use of the gateway, and many other quality issues and considerations.
A few months ago, FDA issued version 1.5.1 of Study Data Specifications. Although the changes in the new version were not great, they were significant for eCTD as a new specification for organizing datasets was defined. The change has to do with how the files are organized in the file structure in which an eCTD is delivered and has no effect on the Table of Contents that is visible to a reviewer (created from the eCTD XML backbone).
The specifications for organizing study datasets and their associated files in folders are summarized in the following figure. No additional subfolders are needed; unused folders do not need to be supplied. In the document, an accompanying table provides further definition of the folders.
The problem that this is causing in the short term is that current publishing tools are not set up to produce this structure. Looking at the folder structure required in the previous specification:
You will see an important difference: tabulations must be assigned to a folder based on whether they are STDM datasets or legacy format. The problem is that most publishing tools don’t “know” which of the formats are being used, and aren’t prepared to create and populate the legacy and sdtm folders.
A second difference is that there is now a separate folder for datasets under analysis, where previously analysis datasets were placed directly in the analysis folder.
So where does that leave sponsors? I see three main options:
I would be interested in hearing about what approach people are taking.
I often work with the Regulatory Operations department at Cato Research and I’m always impressed by the depth and breadth of their knowledge. Cato creates submissions for such a wide cross-section of the sponsor community that their associates have truly “seen it all”, and they can cite a precedent and define a best practice in response to almost any question they are asked. Recently, I took advantage of an opportunity to interview Evan Richardson, Cato’s Director of Regulatory Operations, about how he works with sponsors new to the eCTD and what advice he would offer sponsors planning to make the switch anytime soon.
Kathie: When Cato takes on creation of an eCTD submission for a sponsor who is new to eCTD, what are some of the most common questions you are asked?
Evan: The two of the most common questions are related to timelines and authoring best practices.
Especially when we’re talking about larger submissions, like an IND or NDA, sponsors always want to know how fast we can make it happen. Fortunately, the eCTD format can allow for shorter timelines than sponsors may be used to from paper submissions. The time needed to print hundreds of volumes is eliminated, and the modular nature of the eCTD means that we can compile it as a “rolling submission,” module by module as documents become available. Recently, we compiled and submitted a full NDA in just two months starting from the receipt of the first document.
Also, most sponsors are now familiar with the CTD structure; however, they’re often unfamiliar with writing for eCTD submissions. They need guidance on how to create internal hyperlinks in a Word document or how to properly reference other documents for external hyper linking. We recently hosted a webinar on authoring best practices for eCTD submissions, and we routinely include such training as part of our project kickoff activities with new sponsors.
Kathie: What could the sponsor have done before contracting with Cato to improve quality and decrease rework and other work effort?
Evan: The most labor intensive part of any eCTD submission is the time spent on document preparation. This includes checking and setting the page size and initial page view, creating bookmarks, and creating internal hyperlinks. For some small documents, this can take just a couple of minutes, while it can take hours for larger (or scanned) documents. The biggest thing a sponsor can do to decrease the work effort required for a submission is to provide eCTD-ready documents, i.e. PDF documents that are already formatted, bookmarked, and hyperlinked to (at a minimum) the standards set out in the FDA guidance.
Kathie: In your opinion, if a sponsor is considering submitting in eCTD format in a year’s time, what are the most important steps they should take now?
Evan: There are many important steps that a sponsor should be taking. However, from the Regulatory Operations viewpoint, there are two things they should do that work together towards the common goal of providing eCTD-ready documents.
First, providing training early on for all document authors on the best practices for electronic submissions authoring is vital. There are so many little things, like using “Tables 1, 2, and 3” instead of “Tables 1-3,” that require little or no extra effort for an author, but can result in significant time savings in the compilation phase. Also, if all of the authors are working from the same playbook, the sponsor will have consistency from one document to the next. This will really help Regulatory Operations in the creation of external hyperlinks. It’s also a really important step in making the entire application easier to read for FDA reviewers.
Second, sponsors should require vendors, such as contract medical writers, laboratories, and CROs, to provide eCTD-ready documents. I recommend including this as a requirement in your contracts with vendors. By requiring this up front, you ensure that you will not have to spend the time and money to bring these documents up to standard during the submission compilation process.
Kathie: What eCTD-related issues require the most dialog between you and the sponsor?
Evan: We’ve been doing eCTD submissions since 2005, and we’ve seen plenty of examples of how a lack of granularity in the original application has complicated lifecycle management. With sponsors who are new to eCTD, it can be difficult to get them to see beyond the current sequence and anticipate future updates and changes to their applications. So, we spend a lot of time educating our sponsors on the benefits of increased granularity.
We also spend a lot of time educating our sponsors on the differences between the Append, Replace, and New operators and how best to use each of them. In the paper submissions world, most amendments or supplements are essentially an Append operation and many authors are often most comfortable writing in this manner. We strive to make every eCTD application reviewer-friendly, and so in the submission planning phase, we often work to help our sponsors see the benefits of revising and Replacing documents rather than just Appending new information to a previously submitted document.
Kathie: What percentage of the documents you receive from sponsors are “eCTD-ready”? What are the most common changes you have to make to prepare the documents for submission?
Even: I’d estimate that only about 10% of the documents we receive are truly eCTD-ready, meaning that they require no additional work by our specialists before they can be dropped into a submission. About 60-70% of the documents we receive are partially eCTD-ready. These documents are often partially bookmarked and/or internally hyperlinked and require moderate effort to get them fully eCTD-ready. The remaining 20-30% of documents require us to start from scratch to get them eCTD-ready.
The most common change we have to make is to set the initial view properties to the correct layout and magnification settings – about 90% of all documents we receive require this.
Kathie: I know that Cato provides 100% QC of bookmarks and hyperlinks. Do your sponsors spend a lot of time addressing navigation issues that you find, or are your Cato eSubmission teams able to resolve those issues themselves?
Evan: One of the things we do best is to integrate ourselves with our sponsors, so that we’re not just a vendor, but a part of their team. We work with our sponsors to head off potential issues early on, so that we don’t find ourselves in a position where we are uncovering major navigation issues during the QC process. As a result, the overwhelming majority of issues identified during the QC process are minor things like misspelled bookmarks or bookmarks and hyperlinks that are nonfunctional or point to an incorrect destination. Our sponsors typically rely on our project teams to resolve these issues. We’ve found that educating our sponsors on eCTD best practices from the very beginning is the best way to minimize problems throughout the duration of the project.
Kathie: How do your sponsors review their eCTD sequences? Do you provide training on what they should be looking for in a review process?
Evan: We always recommend that our sponsors use GlobalSubmit REVIEW to review their eCTD sequences. The ability to view your submission exactly as the FDA reviewers will see it makes REVIEW a great tool for the QC process. For large submissions like a NDA, we encourage our sponsors to visit our office to review the submission in REVIEW and meet with our submissions specialists. This allows us to train our sponsors on what and how to review (if necessary), answer questions, and talk through any issues that arise, all face-to-face.
We tailor the training we provide to the needs of each sponsor. For sponsors new to eCTD, we can provide the same training that our in-house QC reviewers receive. Other sponsors have their own established procedures, and don’t require any additional training from our team.
Kathie: Cato often helps small Pharma with their eCTDs with the long-term plan that these sponsors will eventually become self-sufficient and take their publishing process in-house. How do you these sponsors to learn and gain experience so they will be well-positioned to make this move?
Evan: Any eCTD vendor can take your documents, plug them into the eCTD structure, and send you back a compiled submission, even if it sometimes means forcing a square peg into a round hole. Our goal is to be more than just a vendor; we want to be your partner. In addition to providing you with the technical capabilities to compile and submit eCTD sequences, we provide you with the wealth of knowledge and experience we’ve gained since our first eCTD submission in 2005. Our project teams integrate with a sponsor to provide advice and training from day one on the best practices for eCTD submissions. Over the years, we’ve had several sponsors transition to in-house publishing systems, and we take pride in knowing that these sponsors have built their own publishing capabilities upon the foundation of knowledge, skills, and experience that Cato Research provided.
Evan Richardson is the Director of Regulatory Operations for Cato Research, a full service, international CRO. He contributes frequently to Cato’s blog, Ask Cato. You can reach Evan by email at erichard@cato.com or by phone at 919-361-2286.
As most people who work with eCTD are aware, agencies will accept clinical study reports structured in one of two ways:
There are advantages and disadvantages to each, which are outside the scope of this posting (you can request GlobalSubmit’s white paper “What Makes an eCTD Clinical Study Easily Reviewed by the FDA?” if you are interested in more detail). And in fact, thorough our work with the FDA, GlobalSubmit has found that some clients take a “hybrid” approach and submit a legacy report along with other files. Whatever approach is taken, the study, through the mechanism of the study tagging file, must be constructed so that each component is properly tagged and identified.
In a recent gathering with representatives from a number of Pharma companies, the question was raised (mainly regarding US submissions) as to which of the two formats sponsors were using.
The companies who have been submitting eCTD for some time appear to have made the switch to the granular/E3 format for the most part. However, they had different ways of meeting the review and approval challenges that are brought on when you no longer have a single document (or set of paper binders) to approve:
Sponsors need to understand how clinical reviewers will be seeing the studies they submit. You can see some FDA presentations on this subject on GlobalSubmit’s Agency Presentations page. Sponsors can also contact us to learn more about the FDA’s eCTD viewer, which is a GlobalSubmit product.
As promised, here is an expanded list of posts published recently on my twitter account, this time related to agency news:
Some people who read this blog may be aware that I also post eCTD-related updates on my twitter account (www.twitter.com/kathie_clark). I use this medium to post agency news updates, information I hear from the regulators, and interesting articles or presentations that I come across. For more significant news, I usually follow up with a blog posting.
Recently, several people who read the blog have told me that, due to company restrictions, they are not able to follow twitter updates. In addition, there are certainly people who just don’t like twitter (and I can sympathize as there is certainly a lot more trivia and minutia than useful content on most twitter postings). These colleagues have suggested that I periodically summarize the more interesting information I post on twitter on a blog posting as well. Today, I’m taking them up on that suggestion. For part 1, here is a variety of news I posted on Twitter since February 1, 2010, related to interesting white papers, webinars, presentations, blog postings, and online articles. Next post will feature agency news, documents and presentations.
Interesting White Papers, Webinars and Presentations
Online Articles and Blog Postings
So, to wrap up: if you have made it this far and found “news you can use” in this article, you may want to consider following me on Twitter so you get a more timely update. If you have a twitter account, this is easy to do, but even if you don’t, you can just go to the web page www.twitter.com/kathie_clark for the latest updates (and the last three tweets always appear on the upper left column of this blog). You may also want to follow my company, GlobalSubmit, at www.twitter.com/globalsubmit. You don’t actually need a twitter account to do this.