Out Of Trend Results – OOT’s

 

An out-of-trend (OOT) result that does not follow the expected trend, either in comparison with previous results collected from past history. This article discusses the possible statistical approaches and implementation challenges to the identification of OOT results.

It is intended to begin a conversation toward achieving clarity about how to address the identification of out-of-trend results. It is noted that the identification of OOT results is a complicated issue and that further research and discussion is needed. This article is not a detailed proposal but is meant to begin the discussion toward achieving more clarity about how to address the identification of out-of-trend results.

Regulatory Basis

A review of recent Establishment Inspection Reports (EIRs),FDA 483s, and FDA
Warning Letters indicates the identification of OOT data is becoming a regulatory issue for marketed products. Several companies have received 483 observations requesting the development of procedures documenting how OOT results will be identified and investigated. It is important to distinguish between OOS and OOT results. FDA issued a OOS guidance in the scientific literature and discussed at many scientific conferences about OOS results.

Although the FDA guidance indicates in a footnote that much of the guidance presented for OOS can be used to examine OOT results, there is no clearly established legal or regulatory basis to require consideration of data within specification but not following expected trends.

Identification of Out-of-Trend Results

Avoiding potential issues with marketed product, as well as avoid potential regulatory issues apply of OOT control in the analysis is a best practice in the industry. In summary, the issue of OOT is an important topic both from a regulatory and business point of view. Despite this, little has been discussed in the scientific literature or in regulatory guidance on this topic. This article will introduce some approaches that might be used to identify OOT data and discuss some issues that companies will likely need to address before implementation and during use of an OOT identification procedure.

Statistical Approach Background

There is a need for efficient and practical statistical approach to identify OOT results to detect when a batch is not behaving as expected. To judge whether a particular result is OOT, one must first decide what is expected and in particular what data comparisons are appropriate.

Methodology [3 sigma approach]

  • A minimum of 25 – 30 batches data shall be compiled for fixing the Trend range.
  • Results that shall be obtained from the 25 batches tabulated, average value,
    minimum and maximum values are noted.
  • Standard deviation will be calculated for these 25 batches. Excel spread sheet
    shall be used for Standard deviation calculation.
  • Standard deviation will be multiplied by 3 to get the 3 sigma (3 )) value.
  • Maximum limit will be arrived by adding the 3 1 value to the Average value of
    25 batches.
  • Minimum limit will be arrived by subtracting the 3 1 value from the Average
    value of 25 batches. Minimum value may come in negative also at times.
  • The above maximum and minimum limits in 4.1.5 and 4.1.6 shall be taken as the
    Trend range for upper and lower limits.
  • Any value that shall be out of this range will be considered as Out of Trend
    (OOT) value or Outlier value.
  • Wherever specification has only Not more than, then only Maximum limit for
    trend can be considered. Minimum limit should be excluded.
  • Wherever specification has range then both the Maximum and Minimum limits
    for trend should be considered.

Limitations

One advantage of this approach is that as long as the assumptions are met, the rate of false positives can be set when one calculates the limits. However, a disadvantage is the products with limited data, the appropriate limits may be difficult to determine.This can lead to wrongly centered, too narrow, or too wide OOT limits.

Implementation challenges

The purpose of developing a criterion for OOT assessments is to identify the quantitative analytical results during a study that are atypical enough to warrant a
follow-up investigation. Numerous challenges exist that a company must overcome to implement an OOT procedure for commercial batches are …

  • What statistical approaches are used to determine OOT criterion? What data are used to determine OOT limits?
  • What are the minimum data requirements? What evaluation is performed if the minimum data requirement is not met?
  • What data should be used to update limits?
  • The investigation requirements (i.e., who is responsible, what is the timeline, how is it documented, who should be notified must be clearly defined.
  • Who is responsible for comparing the result with the OOT criterion?
  • How is an OOT result confirmed? What additional analytical testing or statistical analyses are appropriate?
  • What actions should be taken if an OOT result is confirmed as an unusual result?
  • How are OOT investigations incorporated into the annual product review?

Conclusion

Identifying OOT results is a growing concern for FDA and the pharmaceutical industry. Ideally, the method to determine an OOT alarm should not be too complex.

References

  • A.M.Hoinowski et al., “Investigation of Out-of-Specification Results,”
    Pharm. Technol. 26 (1), 40–50 (2002).
  • FDA Guidance Document, Investigating Out of Specification (OOS)
    Test Results for Pharmaceutical Production.

Related Reading

Author

V.J.V.Prasad
Managar – Quality Assurance
Symed Labs Limited [Hetero Group]
Hyderabad, India.
Email: jyotiprasad_pharma@yahoo.co.in

27 thoughts on “Out Of Trend Results – OOT’s

  1. some transcriptional errors found in the methodology.
    1. Please read 3 sigma instead of 3 1
    2. Avoid para numbers in methodology i.e.4.1.5, 4.1.6

  2. some transcriptional errors found in the methodology.
    1. Please read 3 sigma instead of 3 1
    2. Avoid para numbers in methodology i.e.4.1.5, 4.1.6

  3. Chris Gates • I suspect this discussion will long, so I’ll start off with a brief comment. Why don’t we substitute OOC for OOT? Out of control in the quality engineering world probably covers what we want, i.e., identifying results that are strange enough to merit some attention. There is already a rich literature to support this approach. You will find that the accepted method of calculating the standard deviation (if you were to use a control chart) is different from your proposal. Also, OOC results can occur for reasons other than ‘beyond 3 sigma’.

  4. Chris Gates • I suspect this discussion will long, so I’ll start off with a brief comment. Why don’t we substitute OOC for OOT? Out of control in the quality engineering world probably covers what we want, i.e., identifying results that are strange enough to merit some attention. There is already a rich literature to support this approach. You will find that the accepted method of calculating the standard deviation (if you were to use a control chart) is different from your proposal. Also, OOC results can occur for reasons other than ‘beyond 3 sigma’.

  5. Olivier MICHEL • Hello Chris, I think some OOC-like are (should be) run through Product Quality Review or Annual Product Review in the pharmaceutical world. Do these PQR/APR are relevant enough from the OOC/OOT view point and using relevant enough statistics / capability indexes is another matter ..

  6. Olivier MICHEL • Hello Chris, I think some OOC-like are (should be) run through Product Quality Review or Annual Product Review in the pharmaceutical world. Do these PQR/APR are relevant enough from the OOC/OOT view point and using relevant enough statistics / capability indexes is another matter ..

  7. Chris Gates • I’ve had quite a lot of experience with APRs and the main problem I see with them related to OOT is that they don’t react to the OOT result when it happens, so the root cause trail is cold, and the reaction is less helpful than if it was done real time.

  8. Chris Gates • I’ve had quite a lot of experience with APRs and the main problem I see with them related to OOT is that they don’t react to the OOT result when it happens, so the root cause trail is cold, and the reaction is less helpful than if it was done real time.

  9. Olivier MICHEL • Fully correct Chris! I meant that APR should be a good basis for establishing relevant Statistical Process Controls, then allowing for OOT scrutiny. My experiences with APRs is that it hardly get an eye on SPC or manufacturing quality trends ; deals more with QC results trend analysis.

  10. Olivier MICHEL • Fully correct Chris! I meant that APR should be a good basis for establishing relevant Statistical Process Controls, then allowing for OOT scrutiny. My experiences with APRs is that it hardly get an eye on SPC or manufacturing quality trends ; deals more with QC results trend analysis.

  11. Dear chris gates sir.
    if you have manufactured 6 to 08 batches for a particular product in a year,while making trends for ph,assay, RS wt are the limits we should give, how to establish the limits, could u pls clarify

  12. Dear chris gates sir.
    if you have manufactured 6 to 08 batches for a particular product in a year,while making trends for ph,assay, RS wt are the limits we should give, how to establish the limits, could u pls clarify

  13. Your CommentThe Article is very good explanation over the OOT. Concerns such as Minimum no. of batches should be more than one ?…Statstical Tool utilization…? Methodology for communicationg & Implementing the OOT limits in routine & Investigations (If Any). I would like to suggest that APR’s should have this calculation of OOT for particular tests. Investigation of OOT must be done & infered in APR. Opinions Please.

  14. Marco Antonio Gonzalez L. • Hi, what is the difference between Ppk and Cpk? which one is better?

  15. Bruce Tive • The main difference is the term used for the standard deviation. Cpk uses R-bar/d2 as an estimate. It basically looks at the ranges between points to arrive at the standard deviation. Ppk is considered more long term. It uses the pooled standard deviation and doesn’t care what order the points are in. Neither is better, it is really what metric fits what you want to do.You’ll find numerous sites that have explanations. Here’s a link that might help.
    http://elsmar.com/Forums/showthread.php?t=7065

  16. Regarding OOT. You can calculate LCL (Lower Control Limit=AVERAGE-3sigma) and UCL (Upper Control Limit=AVERAGE+3sigma) based on 3 sigma (sigma is calculate in excel like STDEV function). My question is about interpretation of OOT: you calculate this limits based on batches made last year (this batches are part from APR in that year) and now we have a batch made this days and we compare results with the limits established on batches which are part from APR. This routine batch is part of the investigation and if this batch is OOT you can’t release the batch only after investigation is finished. The investigation is requested by CC Laboratory. Or, you calculate the limits based on batches from APR and at the final of the year you compare the results for all this batches which are part of the APR with this limits, but in this case your batches are already released on the market and your investigation is limitated. Personal I think the first interpretation is more correct. I wait some other opinion! On the other hand exists some physical parameters which can’t be part of OOT in my opinion, like average weight, tickness, hardness, fill wight etc. or dissolution ehere I think is not correct to calculate LCL and UCL and is more correct to investigate an OOT if dissolution failled on stage one. I wait for another opinion.

  17. I want to clarify that we have to calculata the limit for OOT from APQR. So can we use the same limit for the upcoming batches or it is only limited to the batches from which limit is derived.

  18. Very nice information and discussion on OOT. Like it very much. From long time I m in search of such information.

  19. Dear All,
    Is there any alternate formula for calculating OOT other than 3 Sigma

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Privacy | Terms © 2006 - 2014 askaboutvalidation. All Rights Reserved
Single Sign On provided by vBSSO
redredblackwhiteguest
Log in
Register