Earlier this week, GlobalSubmit and Cato Research held an educational webinar on the FDA eCTD Review Process. The purpose of the webinar was to provide an overview of the various steps and stages that a drug application goes through in order to be approved by the FDA and marketed in the United States. During the webinar, GlobalSubmit’s CTO Jason Rock discussed the FDA’s workflow, the challenges they face in the electronic submission process and common errors in a submission. In addition, he shared some leading eCTD practices, while underlining the importance of using validation software when submitting eCTDs to ensure submission success.
We had a great turnout, and received several excellent questions from the audience. Though we were unable to address every question due to time restraints, Jason has reviewed all the questions submitted and provided answers below:
Q: How do you suggest we clean up metadata issues? For instance, in your example of including the manufacturer with two different spellings…do you go back and delete leaves?
A: If it is a minor issue such as a misspelled city or site name, then it is probably best just to remain consistent in the spelling in all subsequent submissions. If it is a significant error, then you should delete/withdraw the entire section and resubmit.
Q: How does this presentation apply to ANDA eCTD submissions?
A: The presentation generally applies to ANDA’s as well with the exception of the approval timeline which is different than for NDA’s.
Q: One of your slides referenced that the Viewer can’t open SAS files. What does the FDA want you to submit instead of SAS files for an ANDA?
A: While it is true that GlobalSubmit REVIEW™ does not open SAS files in the preview pane, the reviewers at the FDA simply have to double-click on the .xpt document in GlobalSubmit REVIEW and the SAS file will open in SAS Viewer which all of the reviewers at the FDA have installed.
Q: Can you comment on submitting leaves at a lower level of granularity than those detailed in the FDA specs? For instance, submitting the Annual Report at 1.13 even though the specs state 1.13.1 – 1.13.x.
A: This question is not clear; however, node extensions (adding your own lower level of granularity) are strongly discouraged unless you have permission from the FDA. The specifications around granularity are there to make the review process more efficient, so please use them when provided. That said, it is not uncommon for sponsors to use their own judgment and submit items at a higher level.
Q: At what point do you get the ACK message from the Gateway?
A: The acknowledgment response should come instantly, but typically you will receive it within 48 hours.
Q: Were you planning to demonstrate how the FDA views the submissions in GlobalSubmit?
A: Yes, GlobalSubmit has monthly open web demonstrations that you can find listed on the events page of website. Alternatively, you can contact our Marketing team to be added to our mailing list, so that you are notified of these monthly events. Or you can contact our Sales team directly to schedule a personal web demonstration.
Q: Is there any coordination between FDA and EMA new validation requirements? And how will that affect vendors and their tool upgrades?
A: The coordination that occurs between the FDA and EMA happens at ICH and is around standards; details are not coordinated. GlobalSubmit software captures and maintains all of the errors of the FDA and EMA as well as Health Canada and Swiss Medic. We cannot speak to how other vendors support the validation criteria and their upgrade processes.
Q: Can you please let me know about Study Tagging Files and Node Extensions in eCTD?
A: This question is a little vague. We will be happy to answer this question in more detail, please contact Candice.email@example.com .
Q: How do we submit updated SDTM CRT files (e.g. ae.xpt with updated coding for summary or ISS) and original ones (which may have been used for the original CSR) in one submission?
A: The guidance states to put all SDTM CRT files in a data set folder in M5 organized by study. Each study needs to have an associated STF that links to each SDTM CRT file. ISS and ISE should also use study tagging files.
Q: When is the US M1 item 1.15-Promotional Materials used? Is it for DDMAC submissions?
A: DDMAC is not currently in eCTD yet, and the timeline is still undefined.
Q: Is there any tool which would make the scanned PDF documents ready which would reduce a lot of time in getting the documents ready as required for eCTD xml backbone?
A: There are many tools which will convert a scanned document using optical character recognition (OCR), including Adobe. Please note: not all scanned documents have to be converted.
Thank you again to all of those who participated in the webinar. For your reference, the webinar slides are now available for download here. We look forward to seeing you at future events!
Update: the webinar slides are now available for download here.
Navigating eCTD regulations and guidelines is easier said than done. Same goes for assembling and reviewing applications. That’s why GlobalSubmit and Cato Research are offering a free educational webinar on the FDA Review Process on Tuesday, March 1st at 3pm EST.
The purpose of this webinar is to provide you with the complete FDA review process to make sure that you are able to create an application that is technically sound and also structured correctly. Most importantly, it must be ready for review by the agency upon submission.
Featured topics include:
Dr. Sarah Connelly, FDA clinical reviewer and Medical Officer, Division of Antiviral Products, presented some interesting and useful survey results at the DIA Annual Meeting on June 16th. The survey gathered information on FDA reviewers’ perception of submission quality in a number of specific areas. It was a collaboration between FDA and the DIA Medical Writers SIAC.
The survey is hot off the presses, having been completed in early June. It was designed to assess:
Responses were tabulated specifically in the CMC/Quality, Nonclinical and Clinical areas. About 53 reviewers responded.
In CMC, overall writing quality was rated fairly highly: at about 12% poor or fair, 42% average, and 46% good or excellent. However, 17% judged organization or information as poor or fair, and 31% considered completeness of information poor or fair. The worst ratings were given to adequacy of cross-references (with 38% rating fair or poor) and adequacy of bookmarks and bookmark names (32% rating fair or poor). .
In the area of M2 summaries, 19% rated quality as fair or poor, 52% as average, and 29% as good or excellent.
In comments, reviewers highlighted the following issues:
In the nonclinical area, overall writing quality was rated at about 28% poor or fair, 34% average, and 37% good or excellent. Somewhat higher ratings were given to organization and completeness of information. The worst ratings again were given to adequacy of cross-references (with 46% rating fair or poor) and adequacy of bookmarks and bookmark names (52% rating fair or poor). This is not surprising considering the quantity of scanned documents presented by sponsors in nonclinical, and the reluctance of sponsors to spend time enhancing the navigation aids in these documents.
In the area of M2 summaries, 29% rated quality as fair or poor, 40% as average, and 31% as good or excellent.
In comments, reviewers highlighted the following issues:
In the clinical area, ratings were generally higher – not surprising again as most sponsors have a larger budget and an in-house or contracted staff of professional medical writers producing clinical documents. Quality of clinical studies in general was reasonably good, with 22% considering reports poor or fair, 36% average, and 42% poor.
Overall writing quality was rated at about 15% poor or fair, 35% average, and 43% good or excellent. Similar ratings were given to completeness of information, and few reviewers had complaints about organization (possibly due to the adoption of E3 as a standard for clinical study reports). The worst ratings were given to adequacy of cross-references (with 27% rating fair or poor) and adequacy of bookmarks and bookmark names (35% rating fair or poor).
In the area of M3 summaries, 17% rated quality as fair or poor, 39% as average, and 42% as good or excellent. ISS and ISE were a little less satisfactory.
In comments, clinical reviewers mentioned similar issues repetition (including cutting and pasting into summaries), lack of interpretation, non-searchable documents, and inadequate narrative summaries. They also mentioned lack of hyperlinks and difficulty in finding information. On a somewhat harsher note, some reviewers believe that the people writing clinical summaries don’t understand the science.
There are certainly some significant trends in the feedback from reviewers – poor quality bookmarking and hyperlinking, repetition, and lack of summary and interpretation being the most consistent. Something that struck me was the lack of “excellent” ratings – only one area reached as high as 10% and most were well below that. I interpret that as meaning almost all sponsors have some room for improvement. This is a great opportunity to review your practices and see where you might not be meeting the reviewers’ expectations. After all, impeding the reviewers in making their assessments is a major contributor to complete response.
I’m going to deviate from my usual approach of not mentioning my company’s product s and services to mention that our soon-to-be-released GlobalSubmit 2010 release has significant enhancements in support of the quality of PDF documents, bookmarks and hyperlinks. Not only will we identify over 40 PDF-related errors at the validation stage, but our new CrossCheckTM product provides a revolutionary way to QC bookmarks and hyperlinks without having to open target documents. Our clients who have previewed CrossCheck view it as a real productivity boost. For further information or a demo, contact firstname.lastname@example.org.
At a GPhA/FDA Labeling Workshop on April 14, 2010, Virginia Ventura of CDER’s Office of Business Informatics presented CDER Update eCTD & Gateway Submissions. Along with lots of other useful information, Ms. Ventura provided an assessment of the current Top Processing Issues with Esubs:
Percent of total
Unable to extract info from PDF form (sponsor did not use fillable form)
Missing form (356h, 1571 or 2252)
Bad Characters (per eCTD Spec) in file or folder names
Duplicate sequence (sponsor sent twice)
“High” validation errors
Media empty or corrupted
Wrong application number in us-regional.xml
Invalid submission type identified (eCTD submitted as non-eCTD or vice versa)
Valid application number with incorrect type (combination is valid but not Sponsor’s application)
Certainly, the high percentage of submissions not using fillable forms is startling – Ms. Ventura mentioned that most of these are ANDAs. She spells out the solution for using 356h fillable forms (including their correct use with annual reports) – consult the presentation for details. She also stresses that the correct use of fillable forms results in your submission getting to the reviewer much quicker (best case an amazing 10 minutes!).
The amount of High Severity validation errors is also concerning, as the presentation reiterates what the FDA has said before – High Errors = Submission cannot be accepted. Therefore, up to 7% of these submissions may have been rejected, resulting in time off the PDUFA clock in many cases.
All of this serves as a wake-up call that validation and submission QC are more important than ever. Be sure to read the presentation for more tips on STFs, QC of us-regional.xml, use of the gateway, and many other quality issues and considerations.
I often work with the Regulatory Operations department at Cato Research and I’m always impressed by the depth and breadth of their knowledge. Cato creates submissions for such a wide cross-section of the sponsor community that their associates have truly “seen it all”, and they can cite a precedent and define a best practice in response to almost any question they are asked. Recently, I took advantage of an opportunity to interview Evan Richardson, Cato’s Director of Regulatory Operations, about how he works with sponsors new to the eCTD and what advice he would offer sponsors planning to make the switch anytime soon.
Kathie: When Cato takes on creation of an eCTD submission for a sponsor who is new to eCTD, what are some of the most common questions you are asked?
Evan: The two of the most common questions are related to timelines and authoring best practices.
Especially when we’re talking about larger submissions, like an IND or NDA, sponsors always want to know how fast we can make it happen. Fortunately, the eCTD format can allow for shorter timelines than sponsors may be used to from paper submissions. The time needed to print hundreds of volumes is eliminated, and the modular nature of the eCTD means that we can compile it as a “rolling submission,” module by module as documents become available. Recently, we compiled and submitted a full NDA in just two months starting from the receipt of the first document.
Also, most sponsors are now familiar with the CTD structure; however, they’re often unfamiliar with writing for eCTD submissions. They need guidance on how to create internal hyperlinks in a Word document or how to properly reference other documents for external hyper linking. We recently hosted a webinar on authoring best practices for eCTD submissions, and we routinely include such training as part of our project kickoff activities with new sponsors.
Kathie: What could the sponsor have done before contracting with Cato to improve quality and decrease rework and other work effort?
Evan: The most labor intensive part of any eCTD submission is the time spent on document preparation. This includes checking and setting the page size and initial page view, creating bookmarks, and creating internal hyperlinks. For some small documents, this can take just a couple of minutes, while it can take hours for larger (or scanned) documents. The biggest thing a sponsor can do to decrease the work effort required for a submission is to provide eCTD-ready documents, i.e. PDF documents that are already formatted, bookmarked, and hyperlinked to (at a minimum) the standards set out in the FDA guidance.
Kathie: In your opinion, if a sponsor is considering submitting in eCTD format in a year’s time, what are the most important steps they should take now?
Evan: There are many important steps that a sponsor should be taking. However, from the Regulatory Operations viewpoint, there are two things they should do that work together towards the common goal of providing eCTD-ready documents.
First, providing training early on for all document authors on the best practices for electronic submissions authoring is vital. There are so many little things, like using “Tables 1, 2, and 3” instead of “Tables 1-3,” that require little or no extra effort for an author, but can result in significant time savings in the compilation phase. Also, if all of the authors are working from the same playbook, the sponsor will have consistency from one document to the next. This will really help Regulatory Operations in the creation of external hyperlinks. It’s also a really important step in making the entire application easier to read for FDA reviewers.
Second, sponsors should require vendors, such as contract medical writers, laboratories, and CROs, to provide eCTD-ready documents. I recommend including this as a requirement in your contracts with vendors. By requiring this up front, you ensure that you will not have to spend the time and money to bring these documents up to standard during the submission compilation process.
Kathie: What eCTD-related issues require the most dialog between you and the sponsor?
Evan: We’ve been doing eCTD submissions since 2005, and we’ve seen plenty of examples of how a lack of granularity in the original application has complicated lifecycle management. With sponsors who are new to eCTD, it can be difficult to get them to see beyond the current sequence and anticipate future updates and changes to their applications. So, we spend a lot of time educating our sponsors on the benefits of increased granularity.
We also spend a lot of time educating our sponsors on the differences between the Append, Replace, and New operators and how best to use each of them. In the paper submissions world, most amendments or supplements are essentially an Append operation and many authors are often most comfortable writing in this manner. We strive to make every eCTD application reviewer-friendly, and so in the submission planning phase, we often work to help our sponsors see the benefits of revising and Replacing documents rather than just Appending new information to a previously submitted document.
Kathie: What percentage of the documents you receive from sponsors are “eCTD-ready”? What are the most common changes you have to make to prepare the documents for submission?
Even: I’d estimate that only about 10% of the documents we receive are truly eCTD-ready, meaning that they require no additional work by our specialists before they can be dropped into a submission. About 60-70% of the documents we receive are partially eCTD-ready. These documents are often partially bookmarked and/or internally hyperlinked and require moderate effort to get them fully eCTD-ready. The remaining 20-30% of documents require us to start from scratch to get them eCTD-ready.
The most common change we have to make is to set the initial view properties to the correct layout and magnification settings – about 90% of all documents we receive require this.
Kathie: I know that Cato provides 100% QC of bookmarks and hyperlinks. Do your sponsors spend a lot of time addressing navigation issues that you find, or are your Cato eSubmission teams able to resolve those issues themselves?
Evan: One of the things we do best is to integrate ourselves with our sponsors, so that we’re not just a vendor, but a part of their team. We work with our sponsors to head off potential issues early on, so that we don’t find ourselves in a position where we are uncovering major navigation issues during the QC process. As a result, the overwhelming majority of issues identified during the QC process are minor things like misspelled bookmarks or bookmarks and hyperlinks that are nonfunctional or point to an incorrect destination. Our sponsors typically rely on our project teams to resolve these issues. We’ve found that educating our sponsors on eCTD best practices from the very beginning is the best way to minimize problems throughout the duration of the project.
Kathie: How do your sponsors review their eCTD sequences? Do you provide training on what they should be looking for in a review process?
Evan: We always recommend that our sponsors use GlobalSubmit REVIEW to review their eCTD sequences. The ability to view your submission exactly as the FDA reviewers will see it makes REVIEW a great tool for the QC process. For large submissions like a NDA, we encourage our sponsors to visit our office to review the submission in REVIEW and meet with our submissions specialists. This allows us to train our sponsors on what and how to review (if necessary), answer questions, and talk through any issues that arise, all face-to-face.
We tailor the training we provide to the needs of each sponsor. For sponsors new to eCTD, we can provide the same training that our in-house QC reviewers receive. Other sponsors have their own established procedures, and don’t require any additional training from our team.
Kathie: Cato often helps small Pharma with their eCTDs with the long-term plan that these sponsors will eventually become self-sufficient and take their publishing process in-house. How do you these sponsors to learn and gain experience so they will be well-positioned to make this move?
Evan: Any eCTD vendor can take your documents, plug them into the eCTD structure, and send you back a compiled submission, even if it sometimes means forcing a square peg into a round hole. Our goal is to be more than just a vendor; we want to be your partner. In addition to providing you with the technical capabilities to compile and submit eCTD sequences, we provide you with the wealth of knowledge and experience we’ve gained since our first eCTD submission in 2005. Our project teams integrate with a sponsor to provide advice and training from day one on the best practices for eCTD submissions. Over the years, we’ve had several sponsors transition to in-house publishing systems, and we take pride in knowing that these sponsors have built their own publishing capabilities upon the foundation of knowledge, skills, and experience that Cato Research provided.
Evan Richardson is the Director of Regulatory Operations for Cato Research, a full service, international CRO. He contributes frequently to Cato’s blog, Ask Cato. You can reach Evan by email at email@example.com or by phone at 919-361-2286.
As most people who work with eCTD are aware, agencies will accept clinical study reports structured in one of two ways:
There are advantages and disadvantages to each, which are outside the scope of this posting (you can request GlobalSubmit’s white paper “What Makes an eCTD Clinical Study Easily Reviewed by the FDA?” if you are interested in more detail). And in fact, thorough our work with the FDA, GlobalSubmit has found that some clients take a “hybrid” approach and submit a legacy report along with other files. Whatever approach is taken, the study, through the mechanism of the study tagging file, must be constructed so that each component is properly tagged and identified.
In a recent gathering with representatives from a number of Pharma companies, the question was raised (mainly regarding US submissions) as to which of the two formats sponsors were using.
The companies who have been submitting eCTD for some time appear to have made the switch to the granular/E3 format for the most part. However, they had different ways of meeting the review and approval challenges that are brought on when you no longer have a single document (or set of paper binders) to approve:
Sponsors need to understand how clinical reviewers will be seeing the studies they submit. You can see some FDA presentations on this subject on GlobalSubmit’s Agency Presentations page. Sponsors can also contact us to learn more about the FDA’s eCTD viewer, which is a GlobalSubmit product.
As promised, here is an expanded list of posts published recently on my twitter account, this time related to agency news:
Some people who read this blog may be aware that I also post eCTD-related updates on my twitter account (www.twitter.com/kathie_clark). I use this medium to post agency news updates, information I hear from the regulators, and interesting articles or presentations that I come across. For more significant news, I usually follow up with a blog posting.
Recently, several people who read the blog have told me that, due to company restrictions, they are not able to follow twitter updates. In addition, there are certainly people who just don’t like twitter (and I can sympathize as there is certainly a lot more trivia and minutia than useful content on most twitter postings). These colleagues have suggested that I periodically summarize the more interesting information I post on twitter on a blog posting as well. Today, I’m taking them up on that suggestion. For part 1, here is a variety of news I posted on Twitter since February 1, 2010, related to interesting white papers, webinars, presentations, blog postings, and online articles. Next post will feature agency news, documents and presentations.
Interesting White Papers, Webinars and Presentations
Online Articles and Blog Postings
So, to wrap up: if you have made it this far and found “news you can use” in this article, you may want to consider following me on Twitter so you get a more timely update. If you have a twitter account, this is easy to do, but even if you don’t, you can just go to the web page www.twitter.com/kathie_clark for the latest updates (and the last three tweets always appear on the upper left column of this blog). You may also want to follow my company, GlobalSubmit, at www.twitter.com/globalsubmit. You don’t actually need a twitter account to do this.
Continued from last time… Notes from my discussion with Virginia Ventura.
Virginia mentioned lack of a cover letter, or lack of a useful cover letter, as a significant issue. Although the cover letter is not a regulatory requirement, it serves as the reviewers’ “Road Map” to the submission, and Virginia can’t understand why a sponsor would not take advantage of the opportunity to provide clarity on the structure and contents of the submission. Her recommendations here:
• When you reference documents in your cover letter, provide hyperlinks to those documents.
• Review your cover letter closely for accuracy. Mistakes in the cover letter can cause significant confusion.
We discussed the concept of providing more detailed reviewer guides as well. Again, these are not a regulatory requirement, but some companies consistently provide well-structured guides that make reviewing much easier. These should not just reproduce the TOC, but might included details and discussions such as the following:
• How the eCTD was constructed (for example, using which software package) and how it was submitted (via the gateway, etc.)
• Version of specs used
• Brief description of pivotal studies, with mention of where they are located and hyperlinks to key documents and summary sections
• Discussion of how study reports were constructed (ICH granularity, etc.)
You also may wish to discuss placement issues (for example, when a study could legitimately be placed in more than one section), instances when the same document was submitted in more than one section, etc.
As a reminder, FDA has organized documents related to electronic submissions on http://www.fda.gov/cder/regulatory/ersr/. This is a great collection of resources, and Virginia mentioned that not all sponsors are aware that this page exists before they contact her.
Specifically pertinent to this post, FDA has a posted presentation, eCTD Validation, available on their website. This was presented by Don Duggan in February at DIA EDM in Philadelphia.
I recently had a great chance to talk to Virginia Ventura of FDA’s Electronic Submission Support Team about the common mistakes made by sponsors in preparing eCTDs. Virginia told me that mistakes are not confined to small or inexperienced companies. In many cases, these errors result from a lack of basic QC. Here are some examples:
Not updating documents that were created based on templates intended for paper.
Of course, adding hyperlinks is a big part of this. In some cases, sponsors have not only failed to add hyperlinks, but have retained references to paper volumes, which are of course useless in the eCTD environment.
My colleague Shannon Strom, Director of Regulatory Operations at Cato Research, gave a great example of the need to change practices at a recent presentation: consider the previous practices of using a sentence such as:
Tables 1-3 Indicate The Significant SAEs Reported For Each Clinical Study.
This was fine in the paper world, but in the eCTD world, it gives the publisher no way to hyperlink to Table 2! Thus a better structure is:
All Significant SAEs Reported For Each Clinical Study Are Indicated In Table 1, Table 2, And Table 3.
Bottom line: make sure your documents have been updated for proper presentation in an electronic environment, and as always, keep the reviewer in mind! Virginia says that the FDA has received a number of submissions that were essentially “un-reviewable” due to lack of hyperlinking.
Lack of consistency in the Application Number
The application number appears in four places in an eCTD:
It’s really important that the application number matches in all four places. (By the way, if it does not match, the FDA will consult your cover letter for arbitration – so heaven help you if it’s wrong!)
That also brings up the topic of using the PDF Fillable Form provided by FDA for your application form. They really want you to do this!! Virginia says that lack of a fillable form is the #2 error that the FDA sees for eCTDs. (Interestingly, the #1 error is clicking twice when submitting via the gateway – this causes your submission to be submitted twice, resulting in a number of errors, as you would expect.)
More on my talk with Virgina next time…