
A number of good articles related to eSubmissions have been posted online recently. Below, a recap based on posts from my twitter account, www.twitter.com/kathie_clark.
In Standards news:
Finally, a few upcoming conferences have been announced:
Dr. Sarah Connelly, FDA clinical reviewer and Medical Officer, Division of Antiviral Products, presented some interesting and useful survey results at the DIA Annual Meeting on June 16th. The survey gathered information on FDA reviewers’ perception of submission quality in a number of specific areas. It was a collaboration between FDA and the DIA Medical Writers SIAC.
The survey is hot off the presses, having been completed in early June. It was designed to assess:
Responses were tabulated specifically in the CMC/Quality, Nonclinical and Clinical areas. About 53 reviewers responded.
CMC
In CMC, overall writing quality was rated fairly highly: at about 12% poor or fair, 42% average, and 46% good or excellent. However, 17% judged organization or information as poor or fair, and 31% considered completeness of information poor or fair. The worst ratings were given to adequacy of cross-references (with 38% rating fair or poor) and adequacy of bookmarks and bookmark names (32% rating fair or poor). .
In the area of M2 summaries, 19% rated quality as fair or poor, 52% as average, and 29% as good or excellent.
In comments, reviewers highlighted the following issues:
Nonclinical
In the nonclinical area, overall writing quality was rated at about 28% poor or fair, 34% average, and 37% good or excellent. Somewhat higher ratings were given to organization and completeness of information. The worst ratings again were given to adequacy of cross-references (with 46% rating fair or poor) and adequacy of bookmarks and bookmark names (52% rating fair or poor). This is not surprising considering the quantity of scanned documents presented by sponsors in nonclinical, and the reluctance of sponsors to spend time enhancing the navigation aids in these documents.
In the area of M2 summaries, 29% rated quality as fair or poor, 40% as average, and 31% as good or excellent.
In comments, reviewers highlighted the following issues:
Clinical
In the clinical area, ratings were generally higher – not surprising again as most sponsors have a larger budget and an in-house or contracted staff of professional medical writers producing clinical documents. Quality of clinical studies in general was reasonably good, with 22% considering reports poor or fair, 36% average, and 42% poor.
Overall writing quality was rated at about 15% poor or fair, 35% average, and 43% good or excellent. Similar ratings were given to completeness of information, and few reviewers had complaints about organization (possibly due to the adoption of E3 as a standard for clinical study reports). The worst ratings were given to adequacy of cross-references (with 27% rating fair or poor) and adequacy of bookmarks and bookmark names (35% rating fair or poor).
In the area of M3 summaries, 17% rated quality as fair or poor, 39% as average, and 42% as good or excellent. ISS and ISE were a little less satisfactory.
In comments, clinical reviewers mentioned similar issues repetition (including cutting and pasting into summaries), lack of interpretation, non-searchable documents, and inadequate narrative summaries. They also mentioned lack of hyperlinks and difficulty in finding information. On a somewhat harsher note, some reviewers believe that the people writing clinical summaries don’t understand the science.
Summary
There are certainly some significant trends in the feedback from reviewers – poor quality bookmarking and hyperlinking, repetition, and lack of summary and interpretation being the most consistent. Something that struck me was the lack of “excellent” ratings – only one area reached as high as 10% and most were well below that. I interpret that as meaning almost all sponsors have some room for improvement. This is a great opportunity to review your practices and see where you might not be meeting the reviewers’ expectations. After all, impeding the reviewers in making their assessments is a major contributor to complete response.
I’m going to deviate from my usual approach of not mentioning my company’s product s and services to mention that our soon-to-be-released GlobalSubmit 2010 release has significant enhancements in support of the quality of PDF documents, bookmarks and hyperlinks. Not only will we identify over 40 PDF-related errors at the validation stage, but our new CrossCheckTM product provides a revolutionary way to QC bookmarks and hyperlinks without having to open target documents. Our clients who have previewed CrossCheck view it as a real productivity boost. For further information or a demo, contact sales@globalsubmit.com.
Finally, Dr. Connolly’s presentation, CTD/eCTD Quality: FDA Survey Results is available on GlobalSubmit’s presentation page.
On March 10th, I attended the eCTD 2010 conference in Philadelphia. The conference speakers included a number of experienced directors and senior directors of regulatory affairs/reg operations from medium and large Pharma companies. During a panel discussion, an excellent question was posed: what are your biggest challenges – what keeps you up at night?
Although we received four different answers from the panelists, they all had a common theme: the impact of globalization and resource constraints on their ability to be successful.
The first panelist worried about how to staff and budget for global expansion (beyond US/EU/CA/JP). He identified the myriad of local format requirements and the need to work with local regulatory groups and consultants as concerns. He emphasized the need to think out plans and procedures before acting.
The second panelist also cited globalization, which for his company increased the need to work with partners and vendors, adding complexity and complications to existing processes. He found that globalization is changing his company’s regulatory strategy and increased the need to identify and comply with standards.
The third director mentioned that technical issues such as Regulated Product Submissions did not concern her. Instead, she identified the impact of the poor economy, which results in both resource constraints and requests for greater speed. She finds her group to be very resource constrained, and is concerned that if she adds temporary help, she may not be able to train them quickly enough for them to take on major responsibilities.
Finally, the fourth director noted that lack of alignment between business and IT was creating a major risk. For example, the business is placing a high priority on expansion into additional countries and regions – but IT does not have the necessary software upgrades in their work plan. Due to resource and funding constraints, IT is being driven to minimize upgrades while the business wants new features that will increase their efficiency due to their reduced or frozen headcount. He sees more and more a discrepancy between what the business needs and what it is possible for IT to do.
I’m grateful for the willingness of the panelists to share their thoughts and concerns, which was very useful to me in understanding the current business climate. I would be interested in your thoughts about this topic, which results in challenges for most of us connected with this business in any capacity.
This week’s blog posting is from Laura Wright, who attended the 23rd Annual DIA EDM conference in National Harbor, MD, February 16 through 19. In Laura’s role as Account Manager at GlobalSubmit, she talks to many people in regulatory, clinical and IT roles in the biopharmaceutical industry. Hearing some of their interests and concerns led Laura to attend a session entitled “Regulatory Update: Future Directions”. Here, she offers her impressions.
Data standards – whew!! Nothing quite as exciting as data standards. Have you ever seen a group of people arguing about SDTM (Study Data Tabulation Model)? It can get pretty heated. If you think that is a joke, then you haven’t seen it.
I am not interested in joining that debate today. I just got back from DIA’s Annual US EDM conference and I would rather share some insights from the new Computational Science Center and the Office of Planning and Analysis at CDER. Turns out that some people at the FDA see the lack of truly standardized data streaming into their servers as a real problem and they are proposing real solutions. Chuck Cooper and Marni Hall spoke at a regulatory session on Thursday that highlighted new directions at CDER and CBER. The data problem was summed up for the audience and it sounded something like this:
The reviewers are spending too much of their review time trying to manipulate incoming data so that they are able to analyze it. And they are spending more time manipulating than analyzing. Because of different data standards, the overall quality, consistency and transparency of the review suffers. And your NDA approval takes longer. Certainly not a new issue, but a problem nonetheless.
In many cases the data is consistent within the sponsor organization. SDTM has been around for more than 6 years and most sponsors really do want to make the reviewer’s life as easy as possible. But what does “standardized” mean to a therapeutic area that doesn’t have universal data sets available to them? What about all of the well-meaning statisticians who are submitting “SDTM-like” data? The FDA is accepting 5,000 eCTD submissions a month. If everyone is doing their own thing, that is a LOT of data to manipulate before actually reviewing it.
Not only is your drug approval taking longer (I know that you are still stuck on that) but the agency’s ability to retrieve important information about trials that have been conducted in the past is completely arrested. Picturing those handcuffs? They can’t retrieve legacy study information today to easily compare it to new data. There are no audit trails showing how the reviewer came up with the analysis they used last year or five years ago. That means more time is spent on finding answers that the reviewers should be able to have access to in order to help them make sense of the data in front of them today. This cycle seemed kind of hopeless.
Until Chuck and Marni told us about what their offices are doing to support reviewers. Between converting the legacy trial data from prior submitted Phase II studies to diving into that age-old debate to create new specs surrounding data standardization, the new Computational Science Center will be a busy place. You can help, too – your input is very valuable, whether you are a physician, statistician, a regulatory ops manager or a software vendor. There are forums and specialized conferences for you to weigh in. After all, who doesn’t want transparency, consistency and quality?
I have often noticed that people attending conferences would like a chance to talk to the speakers that they have found to be interesting. Of course, you can ask questions at the end of the session, but often what you would really like to do is to pick someone’s brain – talk to them about your ideas and challenges. That’s why you often see the speakers mobbed during the cocktail hour!
eCTD 2010, being held March 10th in my hometown of beautiful Philadelphia, PA, has implemented a practical strategy for extending this benefit to their conference attendees. The conference will feature three Interactive Break-Out Discussion Groups:
During these breakouts, you’ll be able to have a more extended discussion with the speakers. These folks are great speakers and industry experts – I can personally vouch for that! I’ll be at eCTD 2010 and urge you to consider attending as well. If you’ll be there, think about asking your colleagues what questions they wish they could ask Don, Meredith and Albert. It will make for interesting conversation.
After the conference, I’ll be posting highlights on The eCTD Summit.
Attending the DIA EU EDM Conference in December gave me a great opportunity to catch up on eCTD-related status and activities at various European agencies. We heard from a number of presenters representing EMEA (now just European Medicines Agency), the MEB, SwissMedic, and AGES PharmMed. Since the updates are fairly lengthy, today I’ll cover EMEA, and will address the other agencies in a future posting.
Tim Buxton gave the update from the EMEA. He clarified what eCTD implementation means to this agency:
EMEA now receives over 500 eCTD sequences a month. In November, they also received 149 NeeS sequences. (A further update can be found in the recently published Update on the implementation of the EU Telematics strategy, which states that since July 1st 2008, over 2,500 eCTD submissions have been received by EMEA, and 406 centrally-authorised products are managed in eCTD format, representing more than two thirds of the total number of centrally-authorised products).
electronic Application Form (eAF)
Although this was an important initiative for the EMEA, adoption has not been good in the past because no tool was provided to create this XML document. The upcoming release of the eMF will include a Data Exchange Standard, receiving tool (initially EMEA only), authoring tool, and validation tool. Prototypes of the receiving tool and authoring tool under evaluation. Support for variations is still under development.
PIM
Likewise, for PIM, EMEA is delivering a Data Exchange Standard, PIM Review System, PIM Light Authoring Tool, and PIM Data Validation Engine. A statement of intent and migration details are still in pilot. The timetable for PIM (from the Statement of Intent) is:
There has been a change of approach for migration to PIM - EMEA had planned to migrate sponsor’s data but sponsors want to do it themselves with “hand holding”.
eSubmission Gateway
The eSubmission Gateway is in production for ICSRs and has been tested for MAAs. Tim characterized the go-live of the gateway as “around the corner” (but said he was glad that he declined to commit to a date at the DIA annual back in June).
Digital Signatures
A limited pilot was being conducted for digital signatures, but is on hold right now due to other priorities. It won’t be completed in 2010, but may be implementatd in 2011. SAFE is a not the only valid form of eSig. Rules in some countries specify some types of electronic signatures. The EC has just released a call for ideas on how to harmonize eSignatures requirements across Europe. To quote Tim - “The storm for eSig is just around the corner – ignore at your own peril.”
Other Initiatives
Other current initiatives include identification of medicinal products, and ICSRs (update of E2B standard for better ID of medicinal products causing problems).
Upcoming initiatives include eCTD Next Major Version (Regulated Product Submissions) – by the way EMEA has just added a web page for this topic, including links to last year’s meeting minutes.
New Q&A/Change Request Tracking Table
In other EMEA news, a new version (V1.21) of the EU Telematics EU eCTD Change Request/Q&A Tracking Table has been posted. This is an update following discussion of open CRs by the TIGes subgroup, and general review of status and presentation of all CRs. All closed/withdrawn/rejected/duplicated CRs have been moved to new worksheets; all CRs implemented in EU M1 v1.4 were moved to the appropriate ‘Implemented in EU M1 v1.4′ worksheet. Most importantly, all CRs for a potential EU M1 v1.4.1 (spec update only) have been identified and marked.
Well,it’s back to business after the holiday break… and as promised, here’s my update on what’s happening at Health Canada.
At the recent DIA eSubmissions conference, Health Canada’s Vianney Caron presented an update on eCTD at Health Canada.
The E-Review Program
The purpose of Health Canada’s E-Review program is to support branch business processes throughout the health product life cycle across product lines using an integrated secure electronic environment that facilitates the exchange of information and conforms to international standards.
Health Canada has made a lot of progress in E-Review recently. Important milestones achieved include:
Regulatory Activities in eCTD Format
Metrics were provided on eCTDs received:
Cumulative through September of 2009:
Totals for 2009 (as of September), received from 53 sponsors:
Health Canada is not seeing a significant rise in applications and sequences received, such as has been experienced by the FDA and EMEA – perhaps because they still require at least part of the dossier in paper. As of 2009, 8.7% of regulatory activities are received in eCTD format.
eCTD Validation
Sponsors are improving the quality of their submissions to Health Canada. In 2009, 4.5% of sequences failed validation. This is down considerably from previous years: 8.3% failed in 2008 and 11% failed in 2007.
Top 10 errors and warnings encountered during validation include:
1. Inactive bookmarks -no link destination
2. Inactive hyperlinks -no link destination
3. External links -pointing to the folder or destination that does not exist in the folder structure on the CD/DVD. The folder or destination would only exist in the sponsor’s repository.
4. Incorrect naming of the 3011 form; correct name is hc-sc-3011-en.pdf
5. Unreferenced files in the index.xml or ca.regional.xml
6. Life cycle management of the document: invalid file reference or no previous ID found
7. Subfolders created in ca regional folder
8. Missing top level folder
9. Missing attestation letter; content as per eCTD Guidance, letter needs to be dated and signed
10. Printed content of MD-5 Checksum does not match the one in the index-md5.txt file
Preliminary Experience with eCTD
Health Canada reports that they have buy-in from all stakeholders, but no legal basis to make eCTD mandatory. They continue to experience logistical and process issues. Reviewers perceive that paper & electronic in parallel is difficult and the process is excessively manual. Validation is perceived as an extra step. Other review issues including understanding lifecycle management in the eCTD and dealing with scanned documents and resolution of graphics.
Some upcoming milestones include the retirement of hybrid submissions as of January 2010 (this month). A date is yet to be announced for accepting fully electronic submissions.
Assessment of Hybrid Pilot
Health Canada surveyed reviewers participating in their hybrid pilot with some interesting results:
Next Steps at Health Canada
Health Canada defined their next steps as:
Health Canada closed by saying that they strongly recommend electronic submissions in eCTD format, and support the development of common standards. They will be expanding the existing infrastructure to support as many application types across product line as possible. Questions can be sent to them at ereview@hc-sc.gc.ca.
Announcement: See GlobalSubmit’s web site for a brand new presentations page linking to recent agency presentations!
GlobalSubmit recently attended two informative DIA conferences:
During these conferences, industry received updates from a number of regulators, including the FDA, Health Canada, the European Medicines Agency, the MEB, AGES PharmMed, and SwissMedic. In my next series of blog postings, I’ll be passing on interesting news from the regulators. Since there’s quite a bit of material, I’ll cover it in three postings: US, Canada, and Europe.
The FDA’s presentations focused on three major areas:
Status and metrics for FDA initiatives
Gary Gensinger provided updates in both San Diego and Vienna. Some of the more significant metrics included:
Gary spoke about the importance of the emerging RPS standard. He emphasized that RPS is critical to FDA’s meeting their PDUFA IV commitments – as well as supporting their goal of to conducting all their business electronically. Gary referenced a Draft Standard for Trial Use (DSTU) date of January 2010, and a target acceptance date for RPS Submissions for drugs and biologics of the 4th quarter of 2011.
Gary also provided an update on the DARRTS initiative. DARRTS is “A flexible, integrated, fully electronic workflow tracking and information management system to receive, log, track, assign, process, and manage official submissions with internal and external stakeholders. The system maintains the official submission records and will manage and track all communications and documentation concerning submission.” Release 3 of DARRTS was implemented successfully in July 2009, resulting in the retirement of 17 legacy systems. Phase 4 (CDER and CBER BLAs) is being planned.
Data Standards
Lilliam Rosario described a major FDA challenge: The FDA receives massive amounts of clinical research data in extremely disparate formats, using a variety of proprietary standards. This makes it extremely difficult, if not impossible, to do cross-study and application reviews.
FDA has been working towards a standardized approach to capture, receive, and analyze study data. Standardization of study data is vital to integrate pre-marketing study data and post-marketing safety data to improve public health and patient safety. Central to this vision is the creation of an enterprise data infrastructure within FDA to improve the management of all structured scientific data (Janus).
Data standards to support this vision are needed in three broad categories: Exchange standards, analysis standards, and terminology standards. FDA is moving towards XML exchange standards based on the HL7 Reference Information Model to submit study data to the FDA. FDA is also currently working on a proposed rule that would require the electronic submission of study data to the FDA. Study data content for creation of SDTM views will be sent to FDA as an XML files modeled using the HL7 RIM.
Study Tagging Files
Virginia Ventura of the Office of Business Process Support spoke on “Study Tagging Files: Their Vital Role In Submissions To The FDA.”
Virginia described some of the most common problems she sees with STFs:
She clarified that an STF is needed any time you are including documents in Modules 4 or 5, except 5.2 Tabular Listings, and 4.3 or 5.4 Literature references.
A case study provided by Virginia should give sponsors pause.
She continued by providing descriptions of additional problems and their correction strategies:
And wrapped up with some sound advice if you run into problems:
Next time… Health Canada news.
We’ve had several good eCTD-related conferences lately. I’ll be providing some updates on them in my next blog entry, but first I wanted to mention an upcoming conference that sounds really good. This is the Cambridge Healthtech Institute conference: eCTD 2010: Achieving Efficiency and Compliance in Electronic Submissions. This conference will be held March 10, 2010 at the Crowne Plaza Philadelphia Downtown in Philadelphia (which is almost across the street from our GlobalSubmit office).
I’m looking forward to attending this conference as I know many of the speakers and their presentations are excellent. Some of the presentations on the agenda include:
There are also panel discussions and ask the experts sessions.
For those of you involved in clinical trials, CHI is holding a co-located Electronic Data in Clinical Trials on March 8-9.
When I spoke to Jim Prudhomme of CHI concerning registration, he alerted me to the early discount for those who register by December 11 via the conference web site or by calling 781-972-5400.