Fit for purpose qualification of Kit based Assays.

The use of commercial assay kits is pervasive in clinical diagnostics and scientific research, serving vital roles in measurements like cytokine detection. These kits are typically categorized as diagnostic or research-use-only, and their convenience is unquestionable. However, using them without the right qualification poses significant risks. Regulatory agencies offer guidance on qualification, but it can be intricate, particularly in diverse contexts. Qualifying these assays is paramount to ensure they are "fit-for-purpose." Without this, the danger of inaccurate results is high, which can mislead medical decisions and drug development. Relying on unqualified assays can generate inconsistent, unreliable data, hindering experiment reproducibility and scientific findings' validation, hampering scientific progress, and wasting resources. It also risks regulatory non-compliance during transitions from research to clinical applications due to evolving requirements. In essence, proper qualification is a prerequisite, aligning with the intended use and considering potential requirement changes. A proactive approach to assay qualification ensures reliability, instills confidence, informs decision-making, and upholds public health safeguards. While commercial assay kits are widely used, the need for their qualification should be carefully evaluated and considered. To maintain data integrity, reliability, and regulatory compliance, tailored qualification is a fundamental aspect of clinical diagnostics and scientific research, safeguarding the credibility of data-driven decisions.

What types of kits should be considered?

The use of commercial analytical kits in non-clinical and clinical research has quickly grown in the last few years. These kits can be classified based on their intended purposes and design into two primary categories, diagnostic kits and research use only kits.

Diagnostic Kits: These kits are specifically engineered to detect or quantify a particular marker within a patient's sample. They encompass all the necessary reagents and components required for the assay. Diagnostic kits are meticulously developed to ensure consistent and reproducible performance. They find extensive application in clinical settings, where they serve to confirm the presence of a particular medical condition, aiding in the diagnostic process. For instance, they can be employed to affirm the existence of an illness when a patient is suspected of having a specific medical condition or to predict the likelihood of disease development by identifying genetic mutations associated with the condition.

Research Use Only Kits: These kits are marketed as a collection of reagents, accompanied by a comprehensive protocol for sample testing. They encompass various test systems designed to detect or quantify the presence of diverse biological factors, such as glucose, therapeutic drugs, tumor markers, infectious disease markers, autoantibodies, cytokines, and more. These kits are typically perceived as cost-effective solutions, as they obviate the need for laborious and time-consuming assay development steps. It is essential to note that they are often labeled as "research use only" to emphasize their primary purpose, which is to facilitate research endeavors. These kits are tailored to specific sample types and research objectives and are evaluated based on a battery of kit-specific tests. This evaluation approach aligns with the overarching principle of the fit-for-purpose methodology, which ensures that the kit is suitable for the intended research purpose.

The delineation between diagnostic kits and research use only kits is pivotal, as it guides the selection of the appropriate kit based on the specific goals and applications of the user. The utilization of such kits, whether in clinical diagnostics or research studies, contributes to the expeditious and efficient execution of assays, providing valuable data for medical diagnoses, scientific investigations, and advancements in various fields, as exemplified by the study conducted by Fritzler et al. (2003) on the use and efficacy of commercial kits for detecting autoantibodies in the context of autoimmune diseases. This classification assists in making informed decisions regarding the choice of assay kits, depending on the desired outcomes and the context of use.

Heath Authorities view on kit use and the need for kit validation / qualification.

Bioanalytical method development and validation are crucial stages in ensuring the reliability and appropriateness of analytical methods for their intended applications. These processes are guided by regulatory agencies like the FDA and EMA and are designed to confirm the fitness of the developed methods for specific purposes, particularly in the context of drug development.

The FDA provides comprehensive guidance on bioanalytical method validation (FDA BMV 2018). While the BMV guidelines focus on validating methods for traditional drug analysis, they specify that their recommendations may not be directly applicable to commercial diagnostic kits designed for point-of-care patient diagnosis. In such cases, the FDA references a separate guidance document by the Center for Devices and Radiological Health (CDRH), which deals with the principles for the co-development of in vitro companion diagnostic devices with therapeutic products.

For commercial diagnostic kits repurposed for the analysis of drugs and therapeutic biologics, the FDA emphasizes the need for evaluation to ensure their suitability for pharmacokinetic (PK) and pharmacodynamic (PD) purposes. This evaluation involves conducting various tests, such as assessing assay site-specific performance, calibration standards' location, concentrations of quality control (QC) samples, using a study-relevant matrix, ensuring relevance between the analyte detected in the kit and the intended analyte for detection, and assessing other performance parameters.

Kits play a vital role in the detection and quantification of biomarkers, which are essential for pharmacodynamic evaluations during drug development. Biomarkers provide critical information about drug safety and efficacy and can offer insights into observed adverse effects. Depending on their intended use, biomarkers can be classified as "definitive" or "exploratory," with the extent of validation determined by the specific needs of the company or research project.

The European Medicines Agency (EMA 2011) also offers guidelines for bioanalytical method validation, emphasizing the importance of revalidating commercial kits used for pharmacokinetic evaluations. Particular attention is given to understanding the range of quantitation of the assay. This reference to kit-based assays in the EMA guideline is mainly within the context of quantitative pharmacokinetic (PK) analysis.

Furthermore, the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) M10 guidance acknowledges the application of commercial kits in bioanalytical use for drug development. It emphasizes the necessity of assessing assay performance to ensure the acceptability and reliability of analytical results. Interestingly, the ICH guideline suggests that a full validation should be conducted when applying previously reported assays or commercial kit-based assays, particularly in the context of quantitative analyses intended for regulatory submissions, non-clinical, and clinical trials.

The ICH M10 guidance specifically highlights the evaluation of the performance of commercial kits, whether designed for point-of-care patient diagnosis or research use only, when applied to measure drug concentrations. It outlines specific parameters that require re-evaluation, including the specificity of detection and other critical criteria. In cases where kits are modified, full validation becomes necessary, with additional requirements, such as ensuring the calibration curve's completeness, appropriateness of the quality controls, and the use of a matrix relevant to the study samples for preparing calibration standards and quality controls.

Overall, the guidance provided by regulatory agencies, such as the FDA and EMA, underscores the importance of thorough evaluation and validation of bioanalytical methods, especially when using commercial kits in drug development. These guidelines aim to ensure the reliability and acceptability of analytical results, aligning with the specific needs and purposes of research or clinical application.

Validation vs Qualification:

Bioanalytical assay development, validation, and qualification are integral components of ensuring the accuracy and reliability of analytical methods, especially in the context of drug development and regulatory submissions. These processes involve the assessment of various parameters and characteristics, with distinctions between validation and qualification, as well as the role of Quality Assurance (QA) teams.

A validated assay is typically employed for analyzing samples collected in studies intended to support a regulatory submission. These validated assays are often quantitative, aiming to determine the concentration of a drug (pharmacokinetics, or PK) or its metabolites in a biological matrix. The ICH M10 guidance suggests that the level of assay performance characterization applied in studies not submitted for regulatory approval may vary and is determined by the sponsor organization.

In contrast, assay qualification serves as an intermediate step in the process of assessing assay performance. It occurs after assay development but before full validation. Method qualification verifies that the assay functions as expected and can be effectively used for its intended purpose, ensuring that it is "fit-for-purpose." In some instances, assay qualification may reveal discrepancies, prompting the need to revisit and optimize the assay. While not obligatory, method qualification is highly recommended.

One key factor that distinguishes assay qualification from the validation phases is the involvement of the Quality Assurance (QA) team. Company QA departments may choose to review assay validation data and reports, particularly in Contract Research Organizations (CROs). However, this level of QA involvement is less frequent when it comes to assay qualifications.

Below are some of the parameters and characteristics commonly assessed during assay qualification:

1. Critical Reagent Identity and Performance: It is crucial to ascertain the specific reagents used in the analysis and understand their critical characteristics that contribute to the method's success.

2. Performance of Calibration Standards, Quality Controls (QCs), and Assay Controls: In quantitative methods, the performance of assay calibration standards and QCs is rigorously assessed. For qualitative or semi-quantitative protocols, assay controls come under scrutiny. This assessment establishes criteria for assay run acceptance (pass/fail), assay sensitivity, and the range of quantitation.

3. Dilutional Linearity and Parallelism Assessment: This evaluation provides insights into the assay's ability to accurately measure analyte concentration when analyte concentrations exceed the assay's limits of detection.

4. Selectivity and Specificity: Inclusion of these characteristics in assay qualification is often a subject of discussion and consideration.

5. Analyte Stability in a Relevant Biological Matrix: Similar to selectivity and specificity, the criticality of this parameter in assay qualification is a topic that is frequently deliberated.

6. Other Method-Specific Steps: Factors such as carryover or sample processing that may impact the overall assay performance should be carefully evaluated.

Overall, the processes of assay validation and qualification are vital in ensuring that analytical methods are suitable for their intended purposes, with careful consideration of various parameters and characteristics. The role of QA teams in reviewing validation and qualification data adds an additional layer of quality control, with a focus on ensuring the reliability and accuracy of the results obtained from these assays.

Case Study: Cytokine detecting kits and how they are used.

The use of commercial kits for detecting human cytokines is a common practice and a good example of kit application in drug development practice. Such kits are often relying on application of various Ligand Binding Assays (LBAs) platforms. These kits offer a convenient and straightforward approach to cytokine detection, utilizing the reagents provided within the kit and adhering to the specified method protocol. However, the reliability of these methods is subject to rigorous testing of various performance parameters.

One key consideration in cytokine detection is the assessment of performance parameters, which include intra and inter-assay precision, recovery at the Quality Control (QC) levels mentioned in the kit's insert, and the linearity of response. These parameters ensure that the assays produce consistent and accurate results, which are essential for meaningful data analysis.

Cytokines are frequently combined into panels to enable the multiplexed detection of multiple analytes in a single test. While this approach offers significant advantages such as reduced sample volume, time efficiency, and cost-effectiveness, it also introduces challenges due to the diverse expectations regarding the analytical performance for each analyte within the panel.

The growing use of multiplexing technologies is driven by the research community's interest in understanding the intricate patterns of cytokine modulation within various biological processes. However, the integration of multiple analytes into a single test places substantial demands on both analytical instrumentation and the user. These challenges become particularly apparent when various analytical platforms are compared for their ability to detect the same cytokines.

For instance, performance variations between different techniques such as Enzyme-Linked Immunosorbent Assay (ELISA), Luminex, and cytokine bead array have been documented. This underscores the importance of using a control sample with a known concentration to normalize data between different runs, ensuring the reliability and consistency of results across various analytical platforms. (Richens et al 2010).

Utilization of commercial cytokine detection kits is widespread, with LBAs serving as a common approach. To ensure the credibility of the results, rigorous testing of performance parameters is essential. The multiplexing of cytokines in a single test offers numerous advantages but also presents challenges related to analytical performance, necessitating careful consideration of different factors, especially when using diverse analytical platforms.

How are kits used for on the example of cytokine detection in clinical studies.

Assessment of cytokine release syndrome (CRS) may be viewed as a good example of the use of cytokine detection data in the clinic. The CRS is a well-known complication associated with the administration of chimeric antigen receptor (CAR) T-cell (CAR-T) therapies (Xiao, X., Huang, S., Chen, S. et al. 2021). CRS, particularly when it reaches high grades, can lead to serious adverse events and potentially life-threatening toxicity, necessitating vigilant monitoring.

Commercially available multiplexed cytokine detection kits have proven valuable for measuring cytokine release associated with CRS in CAR-T therapy. These kits can effectively detect a range of specific cytokines closely linked to CRS, including IL6, IL10, IFN-g, MCP-1, GM-CSF, TNF, IL1, IL2, IL8, and others (Murthy H, Iqbal M, Chavez JC, Kharfan-Dabaja MA. 2019; Wei, Z., Cheng, Q., Xu, N. et al. 2022; Norelli M et al. 2018; Wang Z, Han W. 2018).

Timely intervention strategies, such as the use of tocilizumab (a humanized anti-IL6 receptor antibody therapeutic) or corticosteroids, have been explored to mitigate severe toxicity (US Food and Drug Administration, 2017).

The product labels for CAR-T therapies, including ABECMA, BREYANZI, KYMRIAH, TECARTUS, YESCARTA, and CARVYKTI, acknowledge the potential for CRS induction and provide recommendations for treatment. These recommendations often include the use of anti-cytokine agents like tocilizumab, with alternative agents considered if tocilizumab fails to provide improvement (Package Insert - ABECMA; Package Insert - BREYANZI; Package Insert-KYMRIAH; Package Insert - TECARTUS; Package Insert - YESCARTA; Package Insert - CARVYKTI). Some labels specify that cytokine levels should be analyzed, with peak elevation observed around day 7-14 after infusion, depending on the specific therapeutic agent (TECARTUS, YESCARTA, CARVYKTI labels).

In the context of CAR-T therapeutics, the detection of cytokines serves as a valuable tool for patient treatment, potentially aiding in the prediction of toxicities. While initial assessments often involve a wide panel of exploratory analysis, the identification of the most critical cytokines allows for more focused testing, which can provide essential supportive data, potentially included in regulatory submissions. At this point, the evaluated analytes may transition into the category of definitive biomarkers.

Cytokine release has also been observed in response to T and B-cell engaging therapeutic monoclonal antibodies, such as anti-CD3 (OKT3), anti-CD28 (TGN1412), CD3/CD19 bispecific blinatumomab, and anti-PD-1 (nivolumab) (Norman DJ et al. 1993; Suntharalingam G et al. 2006; Wing MG et al. 1996; Teachey DT et al. 2013; Rotz SJ et al. 2017).

In the field of adeno-associated virus (AAV)-based gene therapy, measuring cytokine secretion has been explored as a predictive tool to assess the potential human immune response against various AAV serotypes (Gehrke, M.; Diedrichs-Möhring, M.; Bogedein, J.; Büning, H.; Michalakis, S.; Wildner, G. 2022).

Criticality of assay qualification. Risks associated with not understanding assay performance parameters.

Bioanalytical assay qualification involves rigorous evaluation and validation of analytical methods to ensure their suitability for specific purposes. It plays a crucial role in ensuring the reliability and validity of the data generated, which, in turn, has a profound impact on decision-making and the progress of various scientific endeavors.

The primary goal of bioanalytical assay qualification is to assess and establish the fitness-for-purpose of analytical methods. Access to assay performance data generated during qualification ensures understanding of how accurate and consistent measurements are. For example, understanding variability of the results produced in the assay allows us to establish an assay quantification range that can be applied to sample testing with high reliability. Understanding of potential interferences that may occur during analysis of diverse matrix types enables us to be prepared to better interpret assay data and make educated conclusions. Having clear, qualification based, assay run acceptance criteria is critical to avoiding reporting sample analysis results that are generated in failed assays. Some elements of assessment may or may not be relevant. For example, if all samples in a given study are intended to be analyzed in a single batch (run) by a known analyst, qualification of a second analysis may not be required. In this example, sample freeze/thaw stability may also be omitted. If samples collected from healthy volunteers only are expected to be tested, there may not be a need to evaluate assay performance in a disease matrix. These are, of course, only examples and other exceptions may be considered. One needs to appreciate that fact that all assays have a life cycle during which intended use of the method may change. The assay sensitivity expectations, matrix type or details related to sample handling may change as drug candidate moves through the development process. These adjustments should be anticipated, and proactive forward-looking qualification considered to facilitate and simplify future transitions.

The fit for purpose use of each assay depends greatly on its intended use and therefore the extent to which method needs to be qualified may vary. Forward looking understanding of the method intended use is critical to be able to predict what assay parameters will be most critical to knowing whether the sample analysis results are acceptable and can be applied to data analysis and reporting.

Bioanalytical assay qualification therefore serves to demonstrate and verify analytical method performance, ensuring their fitness for a specific purpose of use and delivering data critical for informed decision-making.

The overall risks associated with the lack of bioanalytical assay qualification can be grouped in several categories listed below.

1. Inaccurate results: Without proper assay qualification, the results obtained from analytical methods may lack accuracy. Inaccurate data can misguide decisions in drug development, leading to the advancement of ineffective or unsafe drugs.

2. Unreliable data: Lack of assay qualification can lead to inconsistent or unreliable data, making it impossible to replicate experiments or validate scientific findings. This unreliability of assay data may potentially lead to erroneous conclusions.

3. Wasted resources: In drug development and research, the resources invested in conducting experiments using unqualified assays may go to waste, as the data produced is questionable and may not contribute to scientific knowledge or product development. This can lead to financial losses and hinder the efficient allocation of resources.

4. Regulatory non-compliance: In some instances, absence of assay qualification may result in regulatory non-compliance. This is particularly a concern when an assay initially developed for a different purpose is applied without conducting additional qualification and performance verification. Understanding whether the sample analysis results will be at any point used in a regulatory submission or otherwise shared with a health authority is critical for knowing whether and to what extent assay performance needs to be evaluated. It is often challenging to know for certain ahead of time whether data will be shared broadly and not used for internal decision making. Drug development process is highly complex and often requires change of course, including changes to the doses administered to patients, translating into potentially changing assay sensitivity expectations, or change of the patient's population, translating into different matrix type. Organizations may determine if a forward looking, proactive approach to full assay qualification is value added to ensure possible changes are anticipated. Alternatively, an organization may decide to conduct limited fit for purposes qualification that will potentially necessitate additional assay qualification(s) later in time. One should also be mindful of the need for data comparison between sample results obtained in assays that were qualified to a different degree. This can be a subject for another conversation.

Overall, the nature of data use underscores the need to appreciate and anticipate the need and degree of assay qualification. Reliable and qualified assays not only ensure the accuracy and precision of data but also instill confidence in the results, enabling stakeholders to make informed decisions, safeguard public health, and advance scientific knowledge. The interplay between data use and assay qualification is fundamental to maintaining the integrity and credibility of data-driven decision-making processes in today's complex and interconnected world.

Conclusion

The use of commercial kits for various measurements, such as cytokine detection, is widespread, serving critical roles in both clinical diagnostics and scientific research. These kits are typically categorized into diagnostic kits, primarily for medical purposes, and research-use-only kits, designed for scientific experiments.

It is important to emphasize that while these kits offer convenience and efficiency, they should not be used without appropriate qualification. Regulatory agencies provide guidance on method validation and qualification, but the rules are not always straightforward, especially when applying kits in different contexts.

The qualification of these assays is of paramount importance, ensuring that they are "fit-for-purpose." Without proper qualification, the risks associated with inaccurate and unreliable results become significant. Inaccuracy in the data can mislead decision-making in medical diagnoses and drug development, potentially leading to ineffective or unsafe treatments.

Furthermore, relying on unqualified assays can result in the generation of inconsistent or unreliable data, hindering the reproducibility of experiments and the validation of scientific findings. Such unreliability can have far-reaching consequences for scientific progress and product development, wasting valuable resources.

Moreover, there is a risk of regulatory non-compliance when unqualified assays are used, particularly when transitioning from research to clinical applications. Regulatory requirements may change, and a lack of foresight in qualification may lead to complications when attempting to submit data to regulatory authorities.

The key takeaway is that the degree of qualification should align with the intended use of the assay, keeping in mind the possibility of changes in assay requirements over time. A proactive approach to assay qualification is valuable, ensuring that the kits remain suitable for their evolving applications. This not only enhances the reliability of the data but also instills confidence in the results, enabling informed decision-making and safeguarding public health.

In summary, while the use of commercial kits for various assays is common, the importance of qualifying these kits cannot be overstated. To ensure data accuracy, reliability, and regulatory compliance, proper qualification tailored to the intended purpose is essential. It is a fundamental aspect of maintaining the integrity and credibility of data-driven decision-making in the dynamic fields of clinical diagnostics and scientific research.

References

References are presented in alphabetical order:

1. FDA 2017. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-tisagenlecleucel-b-cell-all-and-tocilizumab-cytokine-release-syndrome)

2. FDA BMV 2018 https://www.fda.gov/regulatory-information/search-fda-guidance-documents/bioanalytical-method-validation-guidance-industry

3. Fritzler, M.J., Wiik, A., Fritzler, M.L. et al. The use and abuse of commercial kits used to detect autoantibodies. Arthritis Res Ther 5, 192 (2003). https://doi.org/10.1186/ar782

4. Gehrke, M.; Diedrichs-Möhring, M.; Bogedein, J.; Büning, H.; Michalakis, S.; Wildner, G. Immunogenicity of Novel AAV Capsids for Retinal Gene Therapy. Cells 2022, 11, 1881. https://doi.org/10.3390/cells11121881

5. ICH M10 https://www.ema.europa.eu/en/ich-m10-bioanalytical-method-validation-scientific-guideline

6. EMA 2011 https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-bioanalytical-method-validation_en.pdf

7. Joanna L. Richens, Richard A. Urbanowicz, Rebecca Metcalf, Jonathan Corne, Paul O'shea, and Lucy Fairclough, Journal of Biomolecular Screening, 15(5), 2010

8. Murthy H, Iqbal M, Chavez JC, Kharfan-Dabaja MA. Cytokine Release Syndrome: Current Perspectives. Immunotargets Ther. 2019 Oct 29;8:43-52. doi: 10.2147/ITT.S202015. PMID: 31754614; PMCID: PMC6825470;

9. Norman DJ, Chatenoud L, Cohen D, Goldman M. Consensus statement regarding OKT3-induced cytokine-release syndrome and human antimouse antibodies. Transplant Proc. 1993;25:89–92

10. Norelli M, Camisa B, Barbiera G, et al. Monocyte-derived IL-1 and IL-6 are differentially required for cytokine-release syndrome and neurotoxicity due to CAR T cells. Nat Med. 2018;24(6):739–748. doi: 10.1038/s41591-018-0036-4;

11. Package Insert - ABECMA (fda.gov)

12. Package Insert - BREYANZI (fda.gov)

13. Package Insert - CARVYKTI (fda.gov)

14. Package Insert - YESCARTA (fda.gov)

15. Package Insert-KYMRIAH (fda.gov)

16. Package Insert - TECARTUS (fda.gov)

17. Suntharalingam G, Perry MR, Ward S, Brett SJ, Castello-Cortes A, Brunner MD, et al. Cytokine storm in a phase 1 trial of the anti-CD28 monoclonal antibody TGN1412. N Engl J Med. 2006;355:1018–1028. doi: 10.1056/NEJMoa063842

18. Teachey DT, Rheingold SR, Maude SL, Zugmaier G, Barrett DM, Seif AE, et al. Cytokine release syndrome after blinatumomab treatment related to abnormal macrophage activation and ameliorated with cytokine-directed therapy. Blood. 2013;121:5154–5157. doi: 10.1182/blood-2013-02-485623

19. Wang Z, Han W. Biomarkers of cytokine release syndrome and neurotoxicity related to CAR-T cell therapy. Biomark Res. 2018;6:4. doi: 10.1186/s40364-018-0116-0).

20. Wei, Z., Cheng, Q., Xu, N. et al. Investigation of CRS-associated cytokines in CAR-T therapy with meta-GNN and pathway crosstalk. BMC Bioinformatics 23, 373 (2022). https://doi.org/10.1186/s12859-022-04917-2;

21. Wing MG, Moreau T, Greenwood J, Smith RM, Hale G, Isaacs J, et al. Mechanism of first-dose cytokine-release syndrome by CAMPATH 1-H: involvement of CD16 (FcgammaRIII) and CD11a/CD18 (LFA-1) on NK cells. J Clin Invest. 1996;98:2819–2826. doi: 10.1172/JCI119110

22. Xiao, X., Huang, S., Chen, S. et al. Mechanisms of cytokine release syndrome and neurotoxicity of CAR T-cell therapy and associated prevention and management strategies. J Exp Clin Cancer Res 40, 367 (2021). https://doi.org/10.1186/s13046-2021-02148-6