- Email: [email protected]
Commentary on the Benefits of US Pharmacopeial Standards: A Generic Pharmaceutical Industry Survey
Journal of Pharmaceutical Sciences xxx (2019) 1-6
Contents lists available at ScienceDirect
Journal of Pharmaceutical Sciences journal homepage: www.jpharmsci.org
Special Topic Commentary
Commentary on the Benefits of US Pharmacopeial Standards: A Generic Pharmaceutical Industry Survey Ian K. Warthin*, Julia Berik, Doug Podolsky, Vimala Raghavendran, Ravi Reddy, Judy Chang, Noah Porter, Nicholas Garito USP, Rockville, Maryland 28052
a r t i c l e i n f o
a b s t r a c t
Article history: Received 15 July 2019 Revised 19 September 2019 Accepted 1 October 2019
Pharmacopeial quality standards for medicines are associated with ensuring quality, but they can also facilitate innovation and competition in pharmaceutical markets. Unpublished data suggest that pharmacopeial quality standards, while not associated with increasing the likelihood of a first generic entrant in the market, are associated with increasing the overall number of generic entrants. This commentary describes an online survey of finished-dose generic drug manufacturers in the United States and India that explores whether using pharmacopeial documentary standards and physical reference standards (RSs) such as those developed by the US Pharmacopeial Convention, accelerate generic drug development and reduce the duration, cost, and risk involved in bringing generic drugs to the US market. The survey provides new evidence that using pharmacopeial documentary standards and physical RSs (1) saves generic drug companies 19% of the time spent on generic product development overall and 31% of time spent on analytical method development and (2) provides clear guidance on quality, thereby reducing the perceived risk of Abbreviated New Drug Application rejection by the US Food and Drug Administration. Ninety percent of respondents agreed that the use of pharmacopeial documentary standards accelerated product development compared with 85% for physical RSs; and 92% of respondents agreed that the use of documentary standards and physical RSs reduced the risk that their Abbreviated New Drug Application would be rejected. Owing to the relatively small sample size of the survey, the results should be viewed as entirely directional for use in trends analysis on the finished-dose generic pharmaceutical industry. The margin of error for all results is 8%-14% at a 95% confidence interval. © 2019 American Pharmacists Association®. Published by Elsevier Inc. All rights reserved.
Keywords: US Pharmacopeia (USP) formulation physical characterization drug standards drug design drug-like property(s) dissolution
Introduction Compendial standards such as those developed by USPdthe acronym for the independent, scientific, nonprofit organization also known as the United States Pharmacopeial Conventiondinclude documentary standards and physical reference standards (RSs) as well as related programs for medicines, food ingredients, and
Conflicts of interest: The authors are USP employees. USP has financial interests in the sale of documentary standards and physical RSs, which are the subjects of this research. Specific vendors and brands are specified in this article to provide clarity regarding the research methodology. Such identification does not imply approval, endorsement, or certification by USP of a particular entity or brand. All proprietary names and brands are property of their respective owners.This article contains supplementary material available from the authors by request or via the Internet at https://doi.org/10.1016/j.xphs.2019.10.008. * Correspondence to: Ian K. Warthin (Telephone: þ1 (301) 692-3477). E-mail address: [email protected] (I.K. Warthin).
dietary supplements manufactured, distributed, and consumed worldwide. Documentary standards provide users with the name of the ingredient or preparation; the definition; packaging, storage, and labeling requirements; and the specification. The specification consists of a series of tests, procedures for the tests, and acceptance criteria. These tests and procedures require the use of compendial RS materials. Medicinal ingredients and products will have the stipulated identity, strength, quality, and purity if they conform to the requirements of the United States PharmacopeiaeNational Formulary monograph and relevant general chapters. During a 5-week period in the fall of 2018, the authors in collaboration with BioInformatics Inc. (Arlington, VA) conducted a blinded online survey of finished-dosage generic drug manufacturing organizations to gain a better understanding of (1) the perceived value of pharmacopeial documentary standards and physical RSs to generic pharmaceutical manufacturers seeking US Food and Drug Administration (FDA) approval for the manufacturing and sale of their products in the US and (2) to
https://doi.org/10.1016/j.xphs.2019.10.008 0022-3549/© 2019 American Pharmacists Association®. Published by Elsevier Inc. All rights reserved.
2
I.K. Warthin et al. / Journal of Pharmaceutical Sciences xxx (2019) 1-6
Table 1 Description of Titles and Companies Interviewed by USP Before the Survey Title
Company Descriptiona
Executive Vice President, Technical operations Principal Scientific Manager, R&D Senior General Manager (GM), Formulation Analytical Research & Development General Manager, Regulatory Regulatory, ANDA Regulatory, ANDA Product Lead Chief Scientific Officer Senior Vice President, Regulatory Affairs Assistant General Manager Chief Operating Officer Senior Research Chemist Senior Research Chemist; Quality Control Senior Director, Ref Standards Evaluation
Mid-sized generic manufacturer, India Mid-sized generic manufacturer, India Mid-sized generic manufacturer, India Mid-sized generic manufacturer, India Mid-sized generic manufacturer, China Small generic manufacturer, China Large generic manufacturer, USA Large generic manufacturer, USA Large generic manufacturer, USA Large generic manufacturer, India Mid-sized generic manufacturer, Switzerland US Pharmacopeia Internalb US Pharmacopeia Internal US Pharmacopeia Internal
a USP secondary research identified approximate annual revenues; organization size was defined as follows: small (<$100M), mid-sized ($100-400M), and large (>$400M annual revenues). b All internal USP interviewees held prior roles at generic manufacturers.
quantitatively measure the perceived benefit(s) of reference and documentary standards during the generic drug development process. The research objectives of this survey were to further explore the findings of (1) a recent literature review on the effect of USP RSs on the promotion of efficient drug development, market competition, drug quality, and patient safety1 and (2) an analytical study on the effect of USP documentary standards on generic entry and prescription drug costs (B. Murimi, unpublished data, 2019). The latter article’s unpublished data found that the presence of a USP drug product monograph is correlated with a greater number of generic entrants overall but not with increasing the likelihood of the first generic entrant in the market. Therefore, this article focuses on the perceived benefits of USP Standards in generic research and development for later generic entrants. From January through March 2018, authors of this paper conducted primary and secondary research to identify the ways in which pharmacopeial RSs may provide value to pharmaceutical companies in their development of generic drugs. We reviewed an extensive list of secondary sources on generic product development (Supporting Information available). This research supported the claim that generic product development is an expensive, timeconsuming, and risky undertaking. For example, a 2012 survey by the Association for Accessible Medicines (formerly the Generic Pharmaceutical Association) estimated the typical range of research and development costs per generic drug product at between $800,000 and $3,250,000, with “typical” defined as the range between the 25th and 75th percentile of the distribution.2 The accounting firm Deloitte LLP (New York, NY) estimates generic drug development costs at between $1 million and $5 million over a 3-5 years timeline.3 Statistics from FDA's Generic Drug Review Dashboard in 2015 showed that FDA was rejecting far more Abbreviated New Drug Applications (ANDAs) than it was approving after an initial review, highlighting the inherent risks in these applications.4 The annual Generic Drug User Fee Amendments reports detail the lengthy process that approval can take for an ANDA.5 The reason for rejection or continued review is not specified for finished dosage form ANDAs; however, FDA did identify specific reasons for rejecting drug ingredient manufacturers' ANDAs in type II active pharmaceutical ingredient (API) drug master file (DMF) submissions. For ANDA API DMFs, the items having to do with RSs were identified as being more likely, on average, to be marked as incomplete relative to the other items evaluated.6,7 Following the secondary research, 15 interviews were conducted with 7 generic pharmaceutical experts from industry and 8 from USP (A complete list of position titles of
interviewees is provided in Table 1). The learnings from the interviews and secondary research provided the basis for our hypotheses on how and why pharmacopeial documentary standards and physical RSs aid in generic drug product development. This survey report was developed to explore and validate the following 3 hypotheses: (1) API and dosage monographs can reduce analytical teams’ method development and verification time, leading to a reduction in generic product development time; (2) USP RSs help reduce or eliminate the risk in the FDA ANDA process by providing clear guidance on quality; and (3) generic manufacturers who plan on being first to file and those who plan to file well after the first ANDA are both likely to benefit from the use of pharmacopeial documentary standards and physical RSs. Our findings provide a deeper understanding of how pharmacopeial quality standards ensure quality and facilitate innovation and competition in pharmaceutical markets. Materials and Methods Our primary research into the ways pharmacopeial documentary standards and physical RSs help in the development of generic drugs included interviews with 15 current or former employees of generic manufacturers whose work, on either their respective science or regulatory affairs teams, focused on product development.
Figure 1. Professional positions of survey respondents.
I.K. Warthin et al. / Journal of Pharmaceutical Sciences xxx (2019) 1-6
3
Figure 2. Size and locations of organizations associated with survey respondents.
These interviews included USP employees, USP clients (i.e., generic manufacturers), and coordinating 7 interviews with the Gerson Lehrman Group (research company based in New York, NY). Our secondary research included a review of resources on generic product development. These resources were found using keywords identified through an iterative process of uncovering new key relevant themes in the generics market using internal databases, industry databases, government reports, news articles, and science/industry journals retrieved via PubMed. A 41-question survey was administered online in a blinded fashion between October 12 and November 16, 2018, by BioInformatics Inc. (Arlington, VA). The sample of potential respondents came from a BioInformatics’ panel, The Science Advisory Board (SAB) (Arlington, VA), and email lists provided by USP, which funded the study. BioInformatics targeted respondents who worked for generic drug manufacturers that file ANDAs in the United States and who have expertise in formulation and analytical testing. Respondents were also required to have obtained RSs, including physical RS materialsdused in testing to help ensure the article’s identity, strength, quality, and puritydfrom an official pharmacopoeia for the purpose of developing generic drugs during the previous 12 months. Each of the qualified respondents received a personalized email message containing a unique Uniform Resource Locator directing them to the online questionnaire. The email message described the objectives of the study, the incentive for participating, and a privacy guarantee. Of the 92 respondents with various professional positions who completed the survey (Fig. 1), 55 were from companies in the United States, and 37 were from companies in India (Fig. 2). The respondents were from at least 44 unique organizations (42 did not disclose their employer’s name and 6 worked at overlapping organizations). The relatively small sample size contributes to the large margin of error (>5% at 95% confidence interval). The results therefore should be considered directional (not statistically significant) and are intended for use in trends analysis on the finisheddose generic pharmaceutical industry.
The results in this report are presented in aggregate. Our statistical calculations and analysis excluded the following: (1) answer choices that were selected by fewer than 5% of the respondents of a given question and (2) free text responses such as “None” and “Not applicable.” Results and Discussion The directional results of the survey provide a deeper understanding of the value of pharmacopeial documentary standards and physical RSs for finished dosage form generic pharmaceutical manufacturers. Despite finding a plethora of industry and academic research on generic product development, the authors identified no prior research on the specific benefits of pharmacopeial standards in generic product development. For that reason, this survey focused on how RSs could help shrink timelines, decrease risk, and reduce capital equipment utilization, as well as how those benefits varied across organization types. The trends analysis indicates that documentary standards and physical RSs are perceived by industry to have clear benefits pertaining to generic product development. This section provides the hypotheses explored in this survey followed by our key findings for hypotheses 1 and 2 and discussion regarding hypothesis 3, for which findings were less conclusive. Hypothesis 1: API and dosage monographs can reduce analytical teams’ method development and verification time leading to a reduction in generic product development time. Using documentary standards and physical RSs saves time in generic product development. The highest percentage difference in time (31%) is from method development, even though the aggregate decrease is larger for formulation (an on average reduction of 24.2 to 17.9 weeks) (Table 2). A large majority of respondents (90%) agreed that the use of compendial documentary standards accelerates the development of a generic product and (85%) agreed that the use of compendial physical RSs accelerates the development of a generic product (Table 3). These results are broadly in line with
Table 2 Impact of Compendial Standard Use on Product Time Steps in Product Development
Average Number of Weeks With Documentary Standards
Formulation Method development Method validation Total product development
17.9 10.9 10.2 38.6
(n (n (n (n
¼ ¼ ¼ ¼
78) 81) 82) 80)
Average Number of Weeks Without Documentary Standards 24.2 15.8 14.3 47.6
(n (n (n (n
¼ ¼ ¼ ¼
63) 68) 69) 64)
Percentage Difference 26% 31% 29% 19%
4
I.K. Warthin et al. / Journal of Pharmaceutical Sciences xxx (2019) 1-6
Table 3 Impact of Compendial Documentary and Reference Standard Material Use on the Risk of FDA Rejection of ANDAs (n ¼ 92) Survey Question
Highly Agree
Agree
Neither Agree nor Disagree
Disagree
Highly Disagree
Use of compendial documentary and reference material standards reduces the risk that your ANDA will be rejected when submitted to the FDA. Use of compendial documentary standards accelerates the development of a generic product. Use of compendial physical reference standard accelerates the development approval of a generic product. Use of compendial reference standards reduces the likelihood that an ANDA will be found incomplete when submitted to the FDA. Use of compendial documentary standards reduces the likelihood that an ANDA will be found incomplete when submitted to the FDA.
54%
38%
3%
4%
0%
43%
47%
9%
0%
1%
37%
48%
11%
4%
0%
27%
42%
25%
4%
1%
24%
49%
21%
7%
0%
findings from interviews that documentary standards were more helpful in reducing time needed during product development. Asked to define where these time savings originated, participants highlighted specific steps in generic product development as well as the time necessary for FDA to review their ANDA (Table 4). Survey participant quotes as well as interviews conducted before to the survey launch support the time-saving trends. These respondents cited pharmacopeial standards as making the development process both “faster” and “easier.” In some cases, respondents assigned the benefit to the review of the ANDA and not solely to the development of the application. Prior primary and secondary research attributed the majority of this time saving to the supposition that analytical method development is almost unnecessary when a pharmacopeial documentary standard is available. During interviews, these questions were asked directly of respondents, and the answers were consolidated at the end of all interviews. Most of these respondents found that the biggest time savings came during analytical method development, as that step in the process can sometimes be skipped if a documentary standard is available. Hypothesis #2: USP standards help reduce or eliminate the risk in the FDA ANDA application process by providing clear guidance on quality. A large majority (92%) of respondents agreed that the use of compendial documentary standards and RSs reduces the risk of ANDA rejection by the FDA (Table 3). Respondents attributed the risk reduction to both the agreed nature of a standard (by the FDA and industry) as well as an increased likelihood that FDA would be more likely to find that the data provided sufficient information. Specifically, survey respondents mentioned identification and purity of standards, quality target product profile, and characterization as sections of the ANDA submission where clarity provided by a pharmacopeial standard was particularly valuable. During primary interviews, a member of a regulatory affairs team referred to a compendial and RS as a valuable tool in reducing
risk by providing clarity in expectations with regulators. Similarly, survey respondents noted that standards enable their company to optimize capital equipment needed for in-house analytical testing (Tables 5 and 6) and mentioned that standards help provide a “streamlined process” and “easy benchmarks” during the ANDA review process (Table 7). Hypothesis #3: Generic manufacturers who plan on being first to file and those who plan to file well after the first ANDA are both likely to benefit from the use of pharmacopeial documentary and physical RSs. Generic drug manufacturers find value in compendial standards, regardless of the organization’s size or timing of ANDA filing (Table 6). In interviews, some relatively larger firms responded that they benefit more from physical RSs to aid their reverse engineering activities; however, the survey responders did not reflect those sentiments. Smaller companies responded that they rely on USP general chapters as well as USP monographs and compendial standards. The lowest level of agreement with the 3 prior hypotheses was 76%, highlighting the general benefit of standard utilization in product development, no matter the size or “go to market” strategy of a generic manufacturer.
Limitations The members of the SAB represent a segment of the scientific community with a demonstrated willingness to participate in market research activities. As an incentive, participants from the SAB were given ViewPoints that may be redeemed for items in the SAB Rewards Catalog. Contacts from the USP list were given Visa gift cards worth $50. These factors may inject a certain level of bias into the findings presented in this report, and any subsequent analysis should be viewed in this light. In addition, some respondents may have had difficulty answering some of the workflow questions if they have a more managerial job position.
Table 4 Selected Quotes on Compendial Standards Accelerating Generic Drug Development How Does the Use of Compendial Documentary Standards and Reference Standards Accelerate the Development of a Generic Drug Product? (n ¼ 84) Selected Respondent Quotes
Size and Location of Organization Associated With Respondents
“Already accepted materials make FDA review easier and more efficient.” “When following compendial standards, we have noticed a defined decrease in total time after submission to acceptance.” “First, it makes both method development and method validation significantly faster and easier. Second, we find it gives a much higher chance of the ANDA being accepted.” “It is a lot easier, in the development process, to take assumptions made by the producer of the reference standards in completing testing.” “It helps to characterize our product, identify impurities, and characterize impurity profiles. Plus, it helps in determining degradation pathway and kinetics of degradation pathways.” “If compendial documentary standards are available, it requires less time for method development and validation as the methods are already provided … and also the agency [FDA] accepts this compliance as sufficient to demonstrate certain quality attributes and accelerates the regulatory approval.”
Large US organization Large US organization Large US organization Medium US organization Medium India organization Small US organization
I.K. Warthin et al. / Journal of Pharmaceutical Sciences xxx (2019) 1-6
5
Table 5 Selected Quotes on Use of Compendial Standards to Optimize Equipment How Does the Use of Compendial Documentary or Reference Standards for Generic Drug Development Enable Your Company to Optimize Capital Equipment Needed for In-House Analytical Testing? (n ¼ 75) Compendial Standard Type
Selected Quotes
Size and Locations of Organizations Associated With Survey Respondents
Documentary standards
“Somewhat less time on the instruments is needed because somewhat less data are needed in addition to significantly less processing.” “Leveraging the preexisting standards to run the experiments the right way the first time.” “Exact equipment or analytical method selection possible.” “Work progresses faster so lab can take on more number of projects at the same time. Hence more capital utilization intensive.” “Less failures due to known quantity.” “Compendial reference standards, generally speaking, call for robust documentation and hence result in lesser queries from the FDA.” “Less risks of failing and repeating tests.” “Helps provide reference controls and data we can compare on our internal equipment.” “Use of compendial reference standards does not require structural characterization, identification, and purity determination.”
Large US organization
Reference standards
To help ensure that the survey was blinded, the wording of the questions referred to “compendial” standards rather than USP RSs. However, because the survey participants were all recipients of FDA ANDAs and worked in generic drug manufacturing organizations that marketed generic products or are in the process of applying for market approval in the United States, the
Large US organization Large India organization Large India organization Large US organization Large US organization Large US organization Large US organization Small US organization
participants may be assumed to have referenced their experience with USP RSs. In addition, panelists from BioInformatics could be different from other generic pharmaceutical industry members because their panel outreach and selection process only reaches a subsection of all generic manufacturers. Finally, although the overall
Table 6 Impact of Compendial Documentary and Reference Standard Material Use on the Risk of FDA Rejection of ANDAs Across Timing of Filing and Organization Size Percent Agree (%)
My Organization Primarily Develops New Generic Drug Products When Documentary Standards are Available; That is, Other Generic Drugs are Already Available (n ¼ 26)
My Organization Develops New Generic Drug Products When Compendial Documentary Standards are Available as Well as When Standards are not Available (e.g., First Generic Drug Approvals) (n ¼ 59)
My Organization Rarely Uses Compendial Documentary Standards When Developing New Generic Drug Products as we Want to be Early to Market for Most Generic Drug Products we Commercialize (n ¼ 7)
Use of compendial documentary standards enables my company to optimize the capital equipment needed in-house for analytical testing Use of compendial reference standards enables my company to optimize the capital equipment needed in-house for analytical testing Use of compendial documentary standards accelerates the ability of my lab/organization to commercialize generic drug products Use of compendial reference standards accelerates the ability of my lab/ organization to commercialize generic drug products
81%
83%
86%
81%
90%
86%
81%
80%
86%
77%
88%
86%
Percent Agree (%)
Large Organization (More Than 1000 Employees) (n ¼ 54)
Medium Organization (100 to 1000 Employees) (n ¼ 25)
Small Organization (Less Than 100 Employees) (n ¼ 13)
Use of compendial documentary standards enables my company to optimize the capital equipment needed in-house for analytical testing Use of compendial reference standards enables my company to optimize the capital equipment needed in-house for analytical testing Use of compendial documentary standards accelerates the ability of my lab/organization to commercialize generic drug products Use of compendial reference standards accelerates the ability of my lab/ organization to commercialize generic drug products
85%
80%
77%
85%
96%
77%
76%
92%
77%
81%
96%
77%
6
I.K. Warthin et al. / Journal of Pharmaceutical Sciences xxx (2019) 1-6
Table 7 Selected Quotes on Additional Benefits of Compendial Standards What are Some of the Additional Benefits of Using Compendial Documentary or Reference Standards for the Generic Drug Development? (n ¼ 80) Compendial Standard Type
Selected Quotes
Size and Locations of Organizations Associated With Survey Respondents
Documentary standards
“Easier to deal with regulatory agencies and higher chance of acceptance and a more streamlined process.” “Being internationally recognized, using the compendial documentary standards helps us get the drug product produced as effectively and efficiently as possible.” “Documentary standards help establish easy benchmarks against which development is carried out. Important role during quality target product profile (QTPP).” “Don't have to evaluate whether the method is stability indicating. Can do a ‘verification’ instead of a ‘validation.’” “These materials often come with a labeled potency that is crucial for data justifications when setting specifications.” “Time save in terms of additional testing.” “No need to provide additional data for characterization, identification and purity of standards used for ANDA submission.” “The impurity standards match the requirements in the pharmacopoeia.” “More accepted by regulatory authorities. Again would say less chances of failure where comparative data required. Better impurity characterization. Checking potency and biological activity.”
Large US organization
Reference standards
sample had a mix of small, medium, and large companies, the number of respondents in medium and small company subgroups was 25 and 13, respectively, making it difficult to draw firm conclusions. Caution should be taken when reviewing results based on fewer than 30 respondents. Conclusion The results of this survey support previous literature findings suggesting that using pharmacopeial documentary standards and physical RSs streamlines the ANDA process because the standards are built into the ANDA process itself.1 Moreover, the survey findings support research that, for generic drug manufacturers, using standards to demonstrate equivalence to innovator products is likely to reduce the need for costly characterization methods and specifications.8 Furthermore, the use of standards also appears to reduce the perceived risk of rejection by FDA; that is, there is anecdotal evidence that the use of a pharmacopeial standard in an ANDA submission improves the likelihood of approval. In short, use of pharmacopeial standards can make method development and validation faster and easier, while simultaneously improving the chance of ANDA acceptance. Whether generic drug manufacturers find value in compendial standards regardless of organization size or timing of ANDA filing should be explored in future research. Acknowledgments This research would not have been possible without the diligent work of BioInformatics staff and thought partnership from our
Large US organization Large India organization Small US organization Large US organization Large India organization Small US organization Small US organization Small India organization
colleagues at Johns Hopkins University and input from our Key Opinion Leaders and customer interviews. In addition, the authors would like to thank the following USP staff members: Eric Bein, Kamalpreet Arora, Morgan Puderbaugh, Osomwonken Igbinosun, Steven L. Walfish, and Tom Fink.
References 1. Heyward J, Padula W, Tierce JC, et al. The value of US pharmacopeial standards: a review of the literature. J Pharm Sci. 2018;107(10):2611-2617. 2. Dickey B, Orszag J. The benefits of patent settlements: new survey evidence on factors affecting generic drug investment. Drug Store News. 2013. Available at: https://www.drugstorenews.com/wp-content/uploads/2013/07/UNITED-STATES -DISTRICT-COURT.pdf. Accessed June 9, 2019. 3. Jacoby R, Smith E, Wilkins D, et al. Winning with Biosimilars: Opportunities in Global Markets. Deloitte Consulting LLP. 2015. Available at https://www2. deloitte.com/content/dam/Deloitte/us/Documents/life-sciences-health-care/uslshc-biosimilars-whitepaper-final.pdf. Accessed June 9, 2019. 4. Brennan Z. Generic drug review dashboard: FDA offers a first look. Regulatory affairs professionals Society (RAPS). Available at: https://www.raps.org/regul atory-focus%E2%84%A2/news-articles/2016/2/generic-drug-review-dashboardfda-offers-a-first-look. Accessed June 12, 2019. 5. US Food and Drug Administration. Performance report to congress for the generic drug user fee amendments. Available at: https://www.fda.gov/downl oads/AboutFDA/ReportsManualsForms/Reports/UserFeeReports/PerformanceRe ports/UCM607654.pdf. Accessed June 9, 2019. 6. Zhang H, Li H, Song W, et al. Completeness assessment of type II active pharmaceutical ingredient drug master files under generic drug user fee amendment: review metrics and common incomplete items. AAPS J. 2014;16(5):11321141. 7. Li H. Common Incomplete Comments Observed in Completeness Assessment of Type II API DMF. 2013 Generic Pharmaceutical Association (GPhA)/Food and Drug Administration (FDA) Fall Technical Conference/API Workshop.Bethesda, MD: 2013 8. Bhattacharyya L, Cecil T, Dabbah R, et al. The value of USP public standards for therapeutic products. Pharm Res. 2004;21(10):1725-1731.
Contents lists available at ScienceDirect
Journal of Pharmaceutical Sciences journal homepage: www.jpharmsci.org
Special Topic Commentary
Commentary on the Benefits of US Pharmacopeial Standards: A Generic Pharmaceutical Industry Survey Ian K. Warthin*, Julia Berik, Doug Podolsky, Vimala Raghavendran, Ravi Reddy, Judy Chang, Noah Porter, Nicholas Garito USP, Rockville, Maryland 28052
a r t i c l e i n f o
a b s t r a c t
Article history: Received 15 July 2019 Revised 19 September 2019 Accepted 1 October 2019
Pharmacopeial quality standards for medicines are associated with ensuring quality, but they can also facilitate innovation and competition in pharmaceutical markets. Unpublished data suggest that pharmacopeial quality standards, while not associated with increasing the likelihood of a first generic entrant in the market, are associated with increasing the overall number of generic entrants. This commentary describes an online survey of finished-dose generic drug manufacturers in the United States and India that explores whether using pharmacopeial documentary standards and physical reference standards (RSs) such as those developed by the US Pharmacopeial Convention, accelerate generic drug development and reduce the duration, cost, and risk involved in bringing generic drugs to the US market. The survey provides new evidence that using pharmacopeial documentary standards and physical RSs (1) saves generic drug companies 19% of the time spent on generic product development overall and 31% of time spent on analytical method development and (2) provides clear guidance on quality, thereby reducing the perceived risk of Abbreviated New Drug Application rejection by the US Food and Drug Administration. Ninety percent of respondents agreed that the use of pharmacopeial documentary standards accelerated product development compared with 85% for physical RSs; and 92% of respondents agreed that the use of documentary standards and physical RSs reduced the risk that their Abbreviated New Drug Application would be rejected. Owing to the relatively small sample size of the survey, the results should be viewed as entirely directional for use in trends analysis on the finished-dose generic pharmaceutical industry. The margin of error for all results is 8%-14% at a 95% confidence interval. © 2019 American Pharmacists Association®. Published by Elsevier Inc. All rights reserved.
Keywords: US Pharmacopeia (USP) formulation physical characterization drug standards drug design drug-like property(s) dissolution
Introduction Compendial standards such as those developed by USPdthe acronym for the independent, scientific, nonprofit organization also known as the United States Pharmacopeial Conventiondinclude documentary standards and physical reference standards (RSs) as well as related programs for medicines, food ingredients, and
Conflicts of interest: The authors are USP employees. USP has financial interests in the sale of documentary standards and physical RSs, which are the subjects of this research. Specific vendors and brands are specified in this article to provide clarity regarding the research methodology. Such identification does not imply approval, endorsement, or certification by USP of a particular entity or brand. All proprietary names and brands are property of their respective owners.This article contains supplementary material available from the authors by request or via the Internet at https://doi.org/10.1016/j.xphs.2019.10.008. * Correspondence to: Ian K. Warthin (Telephone: þ1 (301) 692-3477). E-mail address: [email protected] (I.K. Warthin).
dietary supplements manufactured, distributed, and consumed worldwide. Documentary standards provide users with the name of the ingredient or preparation; the definition; packaging, storage, and labeling requirements; and the specification. The specification consists of a series of tests, procedures for the tests, and acceptance criteria. These tests and procedures require the use of compendial RS materials. Medicinal ingredients and products will have the stipulated identity, strength, quality, and purity if they conform to the requirements of the United States PharmacopeiaeNational Formulary monograph and relevant general chapters. During a 5-week period in the fall of 2018, the authors in collaboration with BioInformatics Inc. (Arlington, VA) conducted a blinded online survey of finished-dosage generic drug manufacturing organizations to gain a better understanding of (1) the perceived value of pharmacopeial documentary standards and physical RSs to generic pharmaceutical manufacturers seeking US Food and Drug Administration (FDA) approval for the manufacturing and sale of their products in the US and (2) to
https://doi.org/10.1016/j.xphs.2019.10.008 0022-3549/© 2019 American Pharmacists Association®. Published by Elsevier Inc. All rights reserved.
2
I.K. Warthin et al. / Journal of Pharmaceutical Sciences xxx (2019) 1-6
Table 1 Description of Titles and Companies Interviewed by USP Before the Survey Title
Company Descriptiona
Executive Vice President, Technical operations Principal Scientific Manager, R&D Senior General Manager (GM), Formulation Analytical Research & Development General Manager, Regulatory Regulatory, ANDA Regulatory, ANDA Product Lead Chief Scientific Officer Senior Vice President, Regulatory Affairs Assistant General Manager Chief Operating Officer Senior Research Chemist Senior Research Chemist; Quality Control Senior Director, Ref Standards Evaluation
Mid-sized generic manufacturer, India Mid-sized generic manufacturer, India Mid-sized generic manufacturer, India Mid-sized generic manufacturer, India Mid-sized generic manufacturer, China Small generic manufacturer, China Large generic manufacturer, USA Large generic manufacturer, USA Large generic manufacturer, USA Large generic manufacturer, India Mid-sized generic manufacturer, Switzerland US Pharmacopeia Internalb US Pharmacopeia Internal US Pharmacopeia Internal
a USP secondary research identified approximate annual revenues; organization size was defined as follows: small (<$100M), mid-sized ($100-400M), and large (>$400M annual revenues). b All internal USP interviewees held prior roles at generic manufacturers.
quantitatively measure the perceived benefit(s) of reference and documentary standards during the generic drug development process. The research objectives of this survey were to further explore the findings of (1) a recent literature review on the effect of USP RSs on the promotion of efficient drug development, market competition, drug quality, and patient safety1 and (2) an analytical study on the effect of USP documentary standards on generic entry and prescription drug costs (B. Murimi, unpublished data, 2019). The latter article’s unpublished data found that the presence of a USP drug product monograph is correlated with a greater number of generic entrants overall but not with increasing the likelihood of the first generic entrant in the market. Therefore, this article focuses on the perceived benefits of USP Standards in generic research and development for later generic entrants. From January through March 2018, authors of this paper conducted primary and secondary research to identify the ways in which pharmacopeial RSs may provide value to pharmaceutical companies in their development of generic drugs. We reviewed an extensive list of secondary sources on generic product development (Supporting Information available). This research supported the claim that generic product development is an expensive, timeconsuming, and risky undertaking. For example, a 2012 survey by the Association for Accessible Medicines (formerly the Generic Pharmaceutical Association) estimated the typical range of research and development costs per generic drug product at between $800,000 and $3,250,000, with “typical” defined as the range between the 25th and 75th percentile of the distribution.2 The accounting firm Deloitte LLP (New York, NY) estimates generic drug development costs at between $1 million and $5 million over a 3-5 years timeline.3 Statistics from FDA's Generic Drug Review Dashboard in 2015 showed that FDA was rejecting far more Abbreviated New Drug Applications (ANDAs) than it was approving after an initial review, highlighting the inherent risks in these applications.4 The annual Generic Drug User Fee Amendments reports detail the lengthy process that approval can take for an ANDA.5 The reason for rejection or continued review is not specified for finished dosage form ANDAs; however, FDA did identify specific reasons for rejecting drug ingredient manufacturers' ANDAs in type II active pharmaceutical ingredient (API) drug master file (DMF) submissions. For ANDA API DMFs, the items having to do with RSs were identified as being more likely, on average, to be marked as incomplete relative to the other items evaluated.6,7 Following the secondary research, 15 interviews were conducted with 7 generic pharmaceutical experts from industry and 8 from USP (A complete list of position titles of
interviewees is provided in Table 1). The learnings from the interviews and secondary research provided the basis for our hypotheses on how and why pharmacopeial documentary standards and physical RSs aid in generic drug product development. This survey report was developed to explore and validate the following 3 hypotheses: (1) API and dosage monographs can reduce analytical teams’ method development and verification time, leading to a reduction in generic product development time; (2) USP RSs help reduce or eliminate the risk in the FDA ANDA process by providing clear guidance on quality; and (3) generic manufacturers who plan on being first to file and those who plan to file well after the first ANDA are both likely to benefit from the use of pharmacopeial documentary standards and physical RSs. Our findings provide a deeper understanding of how pharmacopeial quality standards ensure quality and facilitate innovation and competition in pharmaceutical markets. Materials and Methods Our primary research into the ways pharmacopeial documentary standards and physical RSs help in the development of generic drugs included interviews with 15 current or former employees of generic manufacturers whose work, on either their respective science or regulatory affairs teams, focused on product development.
Figure 1. Professional positions of survey respondents.
I.K. Warthin et al. / Journal of Pharmaceutical Sciences xxx (2019) 1-6
3
Figure 2. Size and locations of organizations associated with survey respondents.
These interviews included USP employees, USP clients (i.e., generic manufacturers), and coordinating 7 interviews with the Gerson Lehrman Group (research company based in New York, NY). Our secondary research included a review of resources on generic product development. These resources were found using keywords identified through an iterative process of uncovering new key relevant themes in the generics market using internal databases, industry databases, government reports, news articles, and science/industry journals retrieved via PubMed. A 41-question survey was administered online in a blinded fashion between October 12 and November 16, 2018, by BioInformatics Inc. (Arlington, VA). The sample of potential respondents came from a BioInformatics’ panel, The Science Advisory Board (SAB) (Arlington, VA), and email lists provided by USP, which funded the study. BioInformatics targeted respondents who worked for generic drug manufacturers that file ANDAs in the United States and who have expertise in formulation and analytical testing. Respondents were also required to have obtained RSs, including physical RS materialsdused in testing to help ensure the article’s identity, strength, quality, and puritydfrom an official pharmacopoeia for the purpose of developing generic drugs during the previous 12 months. Each of the qualified respondents received a personalized email message containing a unique Uniform Resource Locator directing them to the online questionnaire. The email message described the objectives of the study, the incentive for participating, and a privacy guarantee. Of the 92 respondents with various professional positions who completed the survey (Fig. 1), 55 were from companies in the United States, and 37 were from companies in India (Fig. 2). The respondents were from at least 44 unique organizations (42 did not disclose their employer’s name and 6 worked at overlapping organizations). The relatively small sample size contributes to the large margin of error (>5% at 95% confidence interval). The results therefore should be considered directional (not statistically significant) and are intended for use in trends analysis on the finisheddose generic pharmaceutical industry.
The results in this report are presented in aggregate. Our statistical calculations and analysis excluded the following: (1) answer choices that were selected by fewer than 5% of the respondents of a given question and (2) free text responses such as “None” and “Not applicable.” Results and Discussion The directional results of the survey provide a deeper understanding of the value of pharmacopeial documentary standards and physical RSs for finished dosage form generic pharmaceutical manufacturers. Despite finding a plethora of industry and academic research on generic product development, the authors identified no prior research on the specific benefits of pharmacopeial standards in generic product development. For that reason, this survey focused on how RSs could help shrink timelines, decrease risk, and reduce capital equipment utilization, as well as how those benefits varied across organization types. The trends analysis indicates that documentary standards and physical RSs are perceived by industry to have clear benefits pertaining to generic product development. This section provides the hypotheses explored in this survey followed by our key findings for hypotheses 1 and 2 and discussion regarding hypothesis 3, for which findings were less conclusive. Hypothesis 1: API and dosage monographs can reduce analytical teams’ method development and verification time leading to a reduction in generic product development time. Using documentary standards and physical RSs saves time in generic product development. The highest percentage difference in time (31%) is from method development, even though the aggregate decrease is larger for formulation (an on average reduction of 24.2 to 17.9 weeks) (Table 2). A large majority of respondents (90%) agreed that the use of compendial documentary standards accelerates the development of a generic product and (85%) agreed that the use of compendial physical RSs accelerates the development of a generic product (Table 3). These results are broadly in line with
Table 2 Impact of Compendial Standard Use on Product Time Steps in Product Development
Average Number of Weeks With Documentary Standards
Formulation Method development Method validation Total product development
17.9 10.9 10.2 38.6
(n (n (n (n
¼ ¼ ¼ ¼
78) 81) 82) 80)
Average Number of Weeks Without Documentary Standards 24.2 15.8 14.3 47.6
(n (n (n (n
¼ ¼ ¼ ¼
63) 68) 69) 64)
Percentage Difference 26% 31% 29% 19%
4
I.K. Warthin et al. / Journal of Pharmaceutical Sciences xxx (2019) 1-6
Table 3 Impact of Compendial Documentary and Reference Standard Material Use on the Risk of FDA Rejection of ANDAs (n ¼ 92) Survey Question
Highly Agree
Agree
Neither Agree nor Disagree
Disagree
Highly Disagree
Use of compendial documentary and reference material standards reduces the risk that your ANDA will be rejected when submitted to the FDA. Use of compendial documentary standards accelerates the development of a generic product. Use of compendial physical reference standard accelerates the development approval of a generic product. Use of compendial reference standards reduces the likelihood that an ANDA will be found incomplete when submitted to the FDA. Use of compendial documentary standards reduces the likelihood that an ANDA will be found incomplete when submitted to the FDA.
54%
38%
3%
4%
0%
43%
47%
9%
0%
1%
37%
48%
11%
4%
0%
27%
42%
25%
4%
1%
24%
49%
21%
7%
0%
findings from interviews that documentary standards were more helpful in reducing time needed during product development. Asked to define where these time savings originated, participants highlighted specific steps in generic product development as well as the time necessary for FDA to review their ANDA (Table 4). Survey participant quotes as well as interviews conducted before to the survey launch support the time-saving trends. These respondents cited pharmacopeial standards as making the development process both “faster” and “easier.” In some cases, respondents assigned the benefit to the review of the ANDA and not solely to the development of the application. Prior primary and secondary research attributed the majority of this time saving to the supposition that analytical method development is almost unnecessary when a pharmacopeial documentary standard is available. During interviews, these questions were asked directly of respondents, and the answers were consolidated at the end of all interviews. Most of these respondents found that the biggest time savings came during analytical method development, as that step in the process can sometimes be skipped if a documentary standard is available. Hypothesis #2: USP standards help reduce or eliminate the risk in the FDA ANDA application process by providing clear guidance on quality. A large majority (92%) of respondents agreed that the use of compendial documentary standards and RSs reduces the risk of ANDA rejection by the FDA (Table 3). Respondents attributed the risk reduction to both the agreed nature of a standard (by the FDA and industry) as well as an increased likelihood that FDA would be more likely to find that the data provided sufficient information. Specifically, survey respondents mentioned identification and purity of standards, quality target product profile, and characterization as sections of the ANDA submission where clarity provided by a pharmacopeial standard was particularly valuable. During primary interviews, a member of a regulatory affairs team referred to a compendial and RS as a valuable tool in reducing
risk by providing clarity in expectations with regulators. Similarly, survey respondents noted that standards enable their company to optimize capital equipment needed for in-house analytical testing (Tables 5 and 6) and mentioned that standards help provide a “streamlined process” and “easy benchmarks” during the ANDA review process (Table 7). Hypothesis #3: Generic manufacturers who plan on being first to file and those who plan to file well after the first ANDA are both likely to benefit from the use of pharmacopeial documentary and physical RSs. Generic drug manufacturers find value in compendial standards, regardless of the organization’s size or timing of ANDA filing (Table 6). In interviews, some relatively larger firms responded that they benefit more from physical RSs to aid their reverse engineering activities; however, the survey responders did not reflect those sentiments. Smaller companies responded that they rely on USP general chapters as well as USP monographs and compendial standards. The lowest level of agreement with the 3 prior hypotheses was 76%, highlighting the general benefit of standard utilization in product development, no matter the size or “go to market” strategy of a generic manufacturer.
Limitations The members of the SAB represent a segment of the scientific community with a demonstrated willingness to participate in market research activities. As an incentive, participants from the SAB were given ViewPoints that may be redeemed for items in the SAB Rewards Catalog. Contacts from the USP list were given Visa gift cards worth $50. These factors may inject a certain level of bias into the findings presented in this report, and any subsequent analysis should be viewed in this light. In addition, some respondents may have had difficulty answering some of the workflow questions if they have a more managerial job position.
Table 4 Selected Quotes on Compendial Standards Accelerating Generic Drug Development How Does the Use of Compendial Documentary Standards and Reference Standards Accelerate the Development of a Generic Drug Product? (n ¼ 84) Selected Respondent Quotes
Size and Location of Organization Associated With Respondents
“Already accepted materials make FDA review easier and more efficient.” “When following compendial standards, we have noticed a defined decrease in total time after submission to acceptance.” “First, it makes both method development and method validation significantly faster and easier. Second, we find it gives a much higher chance of the ANDA being accepted.” “It is a lot easier, in the development process, to take assumptions made by the producer of the reference standards in completing testing.” “It helps to characterize our product, identify impurities, and characterize impurity profiles. Plus, it helps in determining degradation pathway and kinetics of degradation pathways.” “If compendial documentary standards are available, it requires less time for method development and validation as the methods are already provided … and also the agency [FDA] accepts this compliance as sufficient to demonstrate certain quality attributes and accelerates the regulatory approval.”
Large US organization Large US organization Large US organization Medium US organization Medium India organization Small US organization
I.K. Warthin et al. / Journal of Pharmaceutical Sciences xxx (2019) 1-6
5
Table 5 Selected Quotes on Use of Compendial Standards to Optimize Equipment How Does the Use of Compendial Documentary or Reference Standards for Generic Drug Development Enable Your Company to Optimize Capital Equipment Needed for In-House Analytical Testing? (n ¼ 75) Compendial Standard Type
Selected Quotes
Size and Locations of Organizations Associated With Survey Respondents
Documentary standards
“Somewhat less time on the instruments is needed because somewhat less data are needed in addition to significantly less processing.” “Leveraging the preexisting standards to run the experiments the right way the first time.” “Exact equipment or analytical method selection possible.” “Work progresses faster so lab can take on more number of projects at the same time. Hence more capital utilization intensive.” “Less failures due to known quantity.” “Compendial reference standards, generally speaking, call for robust documentation and hence result in lesser queries from the FDA.” “Less risks of failing and repeating tests.” “Helps provide reference controls and data we can compare on our internal equipment.” “Use of compendial reference standards does not require structural characterization, identification, and purity determination.”
Large US organization
Reference standards
To help ensure that the survey was blinded, the wording of the questions referred to “compendial” standards rather than USP RSs. However, because the survey participants were all recipients of FDA ANDAs and worked in generic drug manufacturing organizations that marketed generic products or are in the process of applying for market approval in the United States, the
Large US organization Large India organization Large India organization Large US organization Large US organization Large US organization Large US organization Small US organization
participants may be assumed to have referenced their experience with USP RSs. In addition, panelists from BioInformatics could be different from other generic pharmaceutical industry members because their panel outreach and selection process only reaches a subsection of all generic manufacturers. Finally, although the overall
Table 6 Impact of Compendial Documentary and Reference Standard Material Use on the Risk of FDA Rejection of ANDAs Across Timing of Filing and Organization Size Percent Agree (%)
My Organization Primarily Develops New Generic Drug Products When Documentary Standards are Available; That is, Other Generic Drugs are Already Available (n ¼ 26)
My Organization Develops New Generic Drug Products When Compendial Documentary Standards are Available as Well as When Standards are not Available (e.g., First Generic Drug Approvals) (n ¼ 59)
My Organization Rarely Uses Compendial Documentary Standards When Developing New Generic Drug Products as we Want to be Early to Market for Most Generic Drug Products we Commercialize (n ¼ 7)
Use of compendial documentary standards enables my company to optimize the capital equipment needed in-house for analytical testing Use of compendial reference standards enables my company to optimize the capital equipment needed in-house for analytical testing Use of compendial documentary standards accelerates the ability of my lab/organization to commercialize generic drug products Use of compendial reference standards accelerates the ability of my lab/ organization to commercialize generic drug products
81%
83%
86%
81%
90%
86%
81%
80%
86%
77%
88%
86%
Percent Agree (%)
Large Organization (More Than 1000 Employees) (n ¼ 54)
Medium Organization (100 to 1000 Employees) (n ¼ 25)
Small Organization (Less Than 100 Employees) (n ¼ 13)
Use of compendial documentary standards enables my company to optimize the capital equipment needed in-house for analytical testing Use of compendial reference standards enables my company to optimize the capital equipment needed in-house for analytical testing Use of compendial documentary standards accelerates the ability of my lab/organization to commercialize generic drug products Use of compendial reference standards accelerates the ability of my lab/ organization to commercialize generic drug products
85%
80%
77%
85%
96%
77%
76%
92%
77%
81%
96%
77%
6
I.K. Warthin et al. / Journal of Pharmaceutical Sciences xxx (2019) 1-6
Table 7 Selected Quotes on Additional Benefits of Compendial Standards What are Some of the Additional Benefits of Using Compendial Documentary or Reference Standards for the Generic Drug Development? (n ¼ 80) Compendial Standard Type
Selected Quotes
Size and Locations of Organizations Associated With Survey Respondents
Documentary standards
“Easier to deal with regulatory agencies and higher chance of acceptance and a more streamlined process.” “Being internationally recognized, using the compendial documentary standards helps us get the drug product produced as effectively and efficiently as possible.” “Documentary standards help establish easy benchmarks against which development is carried out. Important role during quality target product profile (QTPP).” “Don't have to evaluate whether the method is stability indicating. Can do a ‘verification’ instead of a ‘validation.’” “These materials often come with a labeled potency that is crucial for data justifications when setting specifications.” “Time save in terms of additional testing.” “No need to provide additional data for characterization, identification and purity of standards used for ANDA submission.” “The impurity standards match the requirements in the pharmacopoeia.” “More accepted by regulatory authorities. Again would say less chances of failure where comparative data required. Better impurity characterization. Checking potency and biological activity.”
Large US organization
Reference standards
sample had a mix of small, medium, and large companies, the number of respondents in medium and small company subgroups was 25 and 13, respectively, making it difficult to draw firm conclusions. Caution should be taken when reviewing results based on fewer than 30 respondents. Conclusion The results of this survey support previous literature findings suggesting that using pharmacopeial documentary standards and physical RSs streamlines the ANDA process because the standards are built into the ANDA process itself.1 Moreover, the survey findings support research that, for generic drug manufacturers, using standards to demonstrate equivalence to innovator products is likely to reduce the need for costly characterization methods and specifications.8 Furthermore, the use of standards also appears to reduce the perceived risk of rejection by FDA; that is, there is anecdotal evidence that the use of a pharmacopeial standard in an ANDA submission improves the likelihood of approval. In short, use of pharmacopeial standards can make method development and validation faster and easier, while simultaneously improving the chance of ANDA acceptance. Whether generic drug manufacturers find value in compendial standards regardless of organization size or timing of ANDA filing should be explored in future research. Acknowledgments This research would not have been possible without the diligent work of BioInformatics staff and thought partnership from our
Large US organization Large India organization Small US organization Large US organization Large India organization Small US organization Small US organization Small India organization
colleagues at Johns Hopkins University and input from our Key Opinion Leaders and customer interviews. In addition, the authors would like to thank the following USP staff members: Eric Bein, Kamalpreet Arora, Morgan Puderbaugh, Osomwonken Igbinosun, Steven L. Walfish, and Tom Fink.
References 1. Heyward J, Padula W, Tierce JC, et al. The value of US pharmacopeial standards: a review of the literature. J Pharm Sci. 2018;107(10):2611-2617. 2. Dickey B, Orszag J. The benefits of patent settlements: new survey evidence on factors affecting generic drug investment. Drug Store News. 2013. Available at: https://www.drugstorenews.com/wp-content/uploads/2013/07/UNITED-STATES -DISTRICT-COURT.pdf. Accessed June 9, 2019. 3. Jacoby R, Smith E, Wilkins D, et al. Winning with Biosimilars: Opportunities in Global Markets. Deloitte Consulting LLP. 2015. Available at https://www2. deloitte.com/content/dam/Deloitte/us/Documents/life-sciences-health-care/uslshc-biosimilars-whitepaper-final.pdf. Accessed June 9, 2019. 4. Brennan Z. Generic drug review dashboard: FDA offers a first look. Regulatory affairs professionals Society (RAPS). Available at: https://www.raps.org/regul atory-focus%E2%84%A2/news-articles/2016/2/generic-drug-review-dashboardfda-offers-a-first-look. Accessed June 12, 2019. 5. US Food and Drug Administration. Performance report to congress for the generic drug user fee amendments. Available at: https://www.fda.gov/downl oads/AboutFDA/ReportsManualsForms/Reports/UserFeeReports/PerformanceRe ports/UCM607654.pdf. Accessed June 9, 2019. 6. Zhang H, Li H, Song W, et al. Completeness assessment of type II active pharmaceutical ingredient drug master files under generic drug user fee amendment: review metrics and common incomplete items. AAPS J. 2014;16(5):11321141. 7. Li H. Common Incomplete Comments Observed in Completeness Assessment of Type II API DMF. 2013 Generic Pharmaceutical Association (GPhA)/Food and Drug Administration (FDA) Fall Technical Conference/API Workshop.Bethesda, MD: 2013 8. Bhattacharyya L, Cecil T, Dabbah R, et al. The value of USP public standards for therapeutic products. Pharm Res. 2004;21(10):1725-1731.