Journal of Clinical Oncology Reviews on Cell Culture Drug Resistance Testing (CCDRT)

(Referred to in the J Clin Oncol as "Chemotherapy Sensitivity and Restance Assays (CSRAs)"

Larry M. Weisenthal, M.D., Ph.D., Weisenthal Cancer Group, 15140 Transistor Lane Huntington Beach, CA 92649 714-894-0011,  mail@weisenthal.org

The September 1, 2004 edition of the Journal of Clinical Oncology contained two reviews ^1,2 relating to cell culture drug resistance testing (CCDRT). Both reviews were restricted to a consideration of published studies describing the results of clinical trials in which comparisons were made between groups of patients treated with and without consideration of CCDRT test results. Neither paper evaluated the considerable literature which describes the predictive accuracy of CCDRT. Thus, the first paper is not a "systematic review" and the second paper is not a true "technology assessment."

It is beyond comprehension to suppose that the Journal of Clinical Oncology would have accepted for publication a "systematic review" of the estrogen receptor test or of the Her2/neu test which omitted data pertaining to the predictive accuracy of these tests. If the same criteria were used for selecting studies for review (studies evaluating whether or not basing treatment decisions on test results improved patient outcomes), there would be no papers to review in the case of the ER and Her2/neu tests, and the authors would logically have to conclude that the use of these (and all other laboratory and radiographic) tests must "remain investigational."

The first paper ¹ was written by an insurance industry group. Although the authors did not review the data pertaining to predictive accuracy, they quoted their own previous review ³ showing that "a negative assay result can be false in 20% of cases, and a positive assay result can be false in 13% of cases." Firstly, the authors got their data from my reviews ^4,5 and they got the data wrong. Considering nearly 2,000 peer-review published clinical correlations, the published data indicate that patients treated with assay "positive" drugs had an overall 80% response rate, while patients treated with assay "negative" drugs had an overall 13% response rate. The authors failed to note that even the incorrect level of accuracy quoted in their own paper would still mean that there would be better than a 4:1 advantage for choosing "assay positive" drugs over "assay negative" drugs, and most patients would prefer an 87% probability of clinical benefit, over a 20% probability of clinical benefit. The correct numbers indicate better than a 6:1 advantage (80% vs. 13%). Numerous studies have also confirmed significantly better survival for treatment with "positive" drugs ⁵.

The second paper ² (written by a working group from the American Society for Clinical Oncology) was broadly similar to the first paper, and also concluded that the existing literature did not established that assay-directed therapy was superior to empiric therapy. The conclusion of both sets of authors is that prospective clinical trials (first paper) "are needed" and should be (second paper) "a priority." The first paper does not presume to tell clinical oncologists that currently-available tests should or should not be used, but the second paper comes right out and says that use of the tests "is not recommended outside of the clinical trial setting." Instead, the group says, "oncologists should make chemotherapy treatment recommendations on the basis of published reports of clinical trials and a patient's health status and treatment preferences." The paper does not state what should be done in the event that the clinical trials data are inconclusive and the patient prefers to have assay-guided treatment (see discussion below).

It is of interest that the first author of the ASCO paper is the second author of another, very recent review published in the JCO ⁶. A literature-based meta-analysis found no evidence of a survival benefit of adjuvant chemotherapy for stage II colon cancer patients. The authors concluded that "Direct evidence from randomized controlled trials does not support the routine use of adjuvant chemotherapy for patients with stage II colon cancer. Patients and oncologists who accept the relative benefit in stage III disease as adequate indirect evidence of benefit for stage II disease are justified in considering the use of adjuvant chemotherapy, particularly for those patients with high-risk stage II disease. The ultimate clinical decision should be based on discussions with the patient about the nature of the evidence supporting treatment, the anticipated morbidity of treatment, the presence of high-risk prognostic features on individual prognosis, and patient preferences."

Now, it is interesting that the author supports the use of unproven treatment, based on physician and patient preferences in one situation, while not supporting physician and patient preferences in any situation to obtain a laboratory test which has been validated for accuracy, but not for "efficacy."

The problem with both of the above reviews is that they are, in effect, attempting to impose an entirely new standard for the evaluation of medical tests. The previous standard always used to evaluate any type of medical test (e.g. laboratory tests, radiographic tests, functional tests, etc.) has always been the correlative and predictive accuracy of the test. How well does an estrogen receptor assay predict for clinical benefit of tamoxifen? How well does a Her2/neu test predict for clinical benefit of Herceptin? How well does a bacterial culture and sensitivity test predict for clinical success or failure of penicillin therapy? How well does a CT scan or MRI scan or PET scan or CA-125 level or CEA level correlate with the presence and growth of a cancer? How well does a battery of immunohistochemical tests performed on tumor biopsies correlate with diagnosis and prognosis? Not only is test accuracy (not "efficacy") the established standard for evaluating every single test used in medicine, it is also the precise standard used by the FDA in approving a test kit for CCDRT ^7,8. The FDA didn't require proof of "efficacy" (as it has never required proof of "efficacy" for any medical tests).

What has never been shown, with any of the above tests, is whether patients treated with the benefit of test information have higher response rates, longer durations of survival, less toxicity, or improved quality of life than patients managed without the benefit of test results. And these tests are used to select therapy in cancer patients no less than are the cell culture drug resistance tests currently being considered. Should a patient receive tamoxifen or chemotherapy? Should a patient currently on chemotherapy be maintained on the same chemotherapy, or should the patient be switched to a different form of chemotherapy? Are serial CT scans or MRI scans or PET scans better or worse at making this determination than are simply histories, physical examinations, and simple laboratory and "plain film" radiographic tests? We don't really know the answer to any of those questions; yet oncologists routinely order these tests and insurance companies routinely pay for them.

The degree to which "standard therapy" has been well documented in published clinical trials to be "effective" has been often overstated. Let's take ovarian cancer as only one of scores of obvious examples. "Standard" first line therapy for ovarian cancer in the USA is carboplatin plus paclitaxel. Firstly, there is no established difference between platinum combinations and single agent alkylators (such as melphalan) (P=0.23 on meta-analysis) ^9,10. Secondly, there is no established advantage to platinum/paclitaxel over single agent platinum. British experts have concluded that there is a range of acceptable treatment alternatives, including single agent platinum and non-taxane platinum combinations ¹¹. So why do American oncologists use carboplatin/paclitaxel? First, because they just take it on faith that it is better. Second, because it paid very well. (It should be noted that, while the American Society of Clinical Oncology has consistently tried to extinguish the use of CCDRT in cancer chemotherapy, it has, at the same time, worked hard to preserve the system of reimbursement wherein medical oncologists had the impossible conflict of interest of choosing between different forms of chemotherapy with wildly differing profit margins ^12,13. In the absence of CCDRT, medical oncologists are free to choose from among many otherwise therapeutically equivalent regimens, with markedly differing profit margins, while the use of CCDRT severely constrains this choice. The American Society of Clinical Oncology has never supported nor suggested clinical trials to show that patients had equivalent outcomes in the presence and absence of profit motives in choice of therapy.).

The quite impossible challenge of documenting the clinical standard of "efficacy" (as opposed to the heretofore traditional laboratory standard of "accuracy") with these non-proprietary, public domain technologies was, in fact, pointed out by Dr. Maurie Markman (a noted critic of CCDRT) who correctly wrote that "even if it were possible to establish the efficacy of [the assays] in a particular situation, this would do nothing at all to establish the efficacy of [the] assays in any other situation" ¹⁴.

The challenge of "validating" a single test for a single treatment in a single disease is challenging enough (e.g. estrogen receptor in breast cancer, which has still, after 30 years, only been shown to correlate with clinical outcome and has yet to be shown to improve clinical outcome, despite the fact that Dr. Daniel F. Hayes, at the time Director of the Georgetown U Breast Cancer Program and a member of ASCO's "Tumor Expert Guidelines Panel," has referred to Estrogen Receptor as being "the best predictive factor in oncology" ¹⁵). Now consider the challenge of "validating" a test for 40 different drugs which can be used in tens of thousands of combinations in hundreds of diseases. If documented clinical "efficacy" is the standard to be demanded of non-proprietary laboratory tests, then clinicians should abandon all tests currently used in their practices. It will be interesting to see which standard is applied in the future to other laboratory tests associated with the prediction of drug resistance, such as tests based gene expression patterns.

Another of the authors of the ASCO study was Daniel Von Hoff, who performed more than 20,000 cell culture assays on specimens from patients, mostly on a non-investigational basis in work which provided tumor specimens to support the work of the San Antonio Institute for Drug Development. Von Hoff published over one hundred papers on his assay of choice, yet failed to complete a single controlled study of assay-directed chemotherapy. His largest partially-completed study ¹⁶ included only 19 patients with assay-directed therapy. He even wrote an editorial, bemoaning the impossibility of completing controlled clinical trials in this area. Von Hoff wrote that "it is unlikely that these trials and others will ever be completed" and "a tool with the potential for helping oncologists select patient therapy could be lost" ¹⁷

It is very easy to say "prospective clinical trials should remain a priority;" it is quite another to support such trials. In point of fact, the cooperative oncology groups have consistently refused to cooperate with the proponents of cell culture drug resistance testing (CCDRT) to carry out the needed trials. For example, the Gynecologic Oncology Group has refused to consider my proposal of clinical trials of cell culture drug resistance testing with cell death endpoints. All that the proponents of CCDRT can offer the cooperative groups is donated assays and the prospects of improving the clinical results of cancer chemotherapy, but this is not enough to motivate the cooperative groups to do the clinical trials. By raising the bar of acceptance to levels unprecedented for a laboratory test, in essence a tariff has been erected to protect the paradigm of the "best" empiric treatment for the average patient, as identified in appallingly non-productive clinical trials. This tariff also serves to protect the paradigm of drug selection with consideration of the spread between wholesale cost and reimbursement.

It is ironic that the Journal of Clinical Oncology reviews are following so closely on the heels of a seminal New England Journal of Medicine publication ¹⁸. The authors of this latter paper used a 96 hour suspension culture drug resistance assay with a cell death (MTT) endpoint to define cut-offs for "sensitivity" and "resistance." They then used these data to define gene expression patterns associated with sensitivity and resistance to each of 4 drugs commonly used in the treatment of childhood leukemia. They were then able to show that these gene expression definitions of sensitivity and resistance were significantly and independently associated with treatment outcome on multivariate analysis. Note that this work could not have been done without the prior work in more than a thousand CCDRT assays from children with leukemia to define sensitivity and resistance cut-offs for each of the four drugs. The cell culture assays, therefore, are the "Rosetta Stone" which allows for identification of clinically relevant gene expression patterns which correlate with clinical drug resistance for different drugs in specific diseases. This further shows how short-sighted it has been for the academic and clinical oncology community not to support the development and clinical application of CCDRT.

Why is it so necessary to protect the patient from the information provided by a perfectly rational laboratory test, supported by a wealth of entirely consistent data? If used to assist in the selection of a regimen chosen from a series of otherwise reasonable alternatives, then patients will never be harmed and best available evidence strongly indicates that they will often be helped. If one wishes to see an example of an entirely rational technology advance, in a human disease crying out for precisely such a technology advance, supported by an entirely consistent body of data, where the advance continues to be held hostage to a high bar of extraordinarily difficult clinical trials which the critics have been entirely unwilling to support, in an area (laboratory testing) for which such trials would be entirely unprecedented, one need look no further.

Literature Cited

1. Samson, D.J., Seidenfeld, J., Ziegler, K. and Aronson, N.: Chemotherapy sensitivity and resistance assays: a systematic review. J. Clin. Oncol., 22:3618-3630, 2004.

2. Schrag, D., Garewal, H.S., Burstein, H., Samson, D.J., Von Hoff, D.D. and Somerfeld, M.R.: American Society of Clinical Oncology technology assessment: chemotherapy sensitivity and resistance assays. J. Clin. Oncol., 22:3631-3638, 2004.

3. Blue Cross and Blue Shield Association Technology Evaluation Center, Chemotherapy Sensitivity and Resistance Assays. TEC Assessments 17:1-39, 2002. Available at http://www.bcbs.com/tec/vol17/17_12.pdf

4. Weisenthal, L.M.: Cell culture drug resistance testing (CCDRT): misconceptions versus objective data. 1999. Available at http://www.weisenthal.org/contents.htm

5. Weisenthal, L.M. and Nygren, P.: Current status of cell culture drug resistance testing (CCDRT). 2002. Available at http://weisenthal.org/oncol_t.htm

6. Benson, A.B., Schrag, D., Somerfield, M.R., et al: American Society of Clinical Oncology recommendations on adjuvant chemotherapy for stage II colon cancer. J. Clin. Oncol., 22:3408-3419, 2004.

7. Premarket approval for an in vitro diagnostic device (Bartels ChemoResponse Assay) intended for use to determine resistance to 5-fluorouracil of cells isolated from breast tumors to assist physicians in determining if 5-fluorouracil is and ineffective treatment for relapsed breast cancer patients (Federal Register 61(243), pp. 66282-66283, 1996.

8. Alpert, S., Director, Office of Device Evaluation, Center for Devices and Radiological Health, U.S.Food and Drug Administration, Dept.of Health and Human Services. Premarket approval of Bartels Prognostics, Inc. Bartels ChemoResponse Assay. 1996. Available (50 page pdf file) at http://weisenthal.org/fda_bartels_approval.pdf

9. Aabo, K., Adams, M., Adnitt, P., et al: Chemotherapy in advanced ovarian cancer: four systematic meta-analyses of individual patient data from 37 randomized trials. Advanced Ovarian Cancer Trialists' Group. Br J Cancer, 78:1479-1487, 1998.

10. Weisenthal, L.M.: Lack of proof from randomized trials for the superiority of platinum combinations over single agent alkylators. 2003. Available at http://www.weisenthal.org/Ova_met1.htm

11. National Institute for Clinical Excellence: Guidance on the use of paclitaxel in the treatment of ovarian cancer. 2003. Available at http://www.nice.org.uk/pdf/55_Paclitaxel_ovarianreviewfullguidance.pdf

12. Editorial Board. Cancer Scare Tactics. In: New York Times, March 22, 2004 ("Free" subscription only. Mirrored copy of this New York Times editorial available at http://www.aaoc.org.ar/noticias/2004/marzo/marzo22.htm)

13. Public Interest Watch, : Public Interest Watch calls for government investigation of Ketchum Communications and American Society of Clinical Oncologists. 2004. Available at http://publicinterestwatch.org/press_9_8_04.htm

14. Markman, M.: Chemosensitivity and chemoresistance assays: are they clinically relevant? J. Cancer Res. Clin. Oncol., 121:441-442, 1995.

15. Hayes, D.F.: Medicare Coverage - MCAC. Laboratory and Diagnostic Services Panel. Human Tumor Assay Systems. Transcript of November 15, 1999 Morning Session. Available at http://www.cms.hhs.gov/mcac/8b1-h1.asp? (pages 175-177)

16. Von Hoff, D.D., Sandbach, J.F., Clark, G.M., et al: Selection of cancer chemotherapy for a patient by an in vitro assay versus a clinician. J. Natl. Cancer Inst. 82:110-116, 1990

17. Von Hoff, D.D.: He's not going to talk about in vitro predictive assays again, is he? J Natl. Cancer Inst., 82:96-101, 1990.

18. Holleman, A., Cheok, M.H., den Boer, M.L., et al: Gene-expression patterns in drug-resistant acute lymphoblastic leukemia and response to treatment. New. Engl. J. Med., 351:601-603, 2004.