Cell Culture Drug Resistance Testing (CCDRT) Cell Death Assays:
Misconceptions Versus Objective Data
What the critics say: A. The Maurie Markman Editorials
Earlier, I noted the two relatively negative reviews (1,2) by Dr. Maurie Markman (who is not nor never has been a recognized expert or published author in this field). As noted and cited earlier, balancing the negative editorials by Markman are more extensive and objective contemporaneous reviews (one a chapter in the most respected textbook in the field of cancer medicine).
Markman's two papers are not original research, nor are they even comprehensive reviews. Rather, they are editorials, supported by a highly selective consideration of a handful of studies, which are not considered in the context of the entire literature.
But the whole of Markman's arguments really come down to a single point: Markman does not feel that it is justifiable to use these assays outside of an investigational setting until it has been established in controlled trials that the use of these assays improves clinical outcomes, the most relevant of which he considers to be patient survival. Markman's view of this is very extreme, and his own writings, I believe, establish his bias, as he states that "even if it were ultimately possible to find a particular tumor type where the cost and technical difficulties associated with chemosensitivity testing were justified, it would not be appropriate to generalize the findings of such a study to other tumor types with unique natural histories and different sensitivities to different drugs."
The implications of this point of view are daunting, to say the least. There are more than forty drugs approved for cancer treatment, and this number is growing at the rate of several per year and is expected to accelerate. The number of potential drug combinations is an exponential of the number of individual drugs. Additionally, the number of disease states, histologies, and stages number in the 100s.
Yet, on an intellectual basis, if not on a clinical or humanitarian basis, I agree entirely with Markman. Let us say that I get a grant to do a prospective clinical trial in breast cancer. A thousand patients are enrolled over a period of 5 years to compare standard "FAC" (FU/Adriamycin/Cytoxan) adjuvant chemotherapy with assay directed chemotherapy. Patients are then followed for ten years or more to document the superior efficacy of assay-directed therapy over empiric FAC. Does this prove anything at all about ovarian cancer or melanoma or ovarian cancer or lymphoma? What if, during the interim, empiric treatment with Taxol plus cisplatin is shown to be superior to empiric treatment with FAC? Has my million dollar, 15 year study shown that assay-directed therapy is superior to empiric Taxol + cisplatin? Of course it has not. So Markman is correct in saying that a study in one particular situation only applies to that precise situation and that there are innumerable other situations and that the situations are continually changing. But what this means is a constantly moving target.
It is very important to note that Markman is attempting to impose a standard which has never before been met by any laboratory or radiographic test. It is, in fact, a standard which has not been met by the majority of treatments used in the majority of cancer patients today (we could all come up with myriad examples of such treatments, beginning with second line chemotherapy of virtually any cancer, including melanoma, and proceeding to the first line surgical or radiologic treatment of prostate cancer and to the use of MRI scans and CT scans to "follow" the results of chemotherapy, rather than relying on history, physical, and simple blood and radiographic tests, surgical lymph node samplings, and every single other laboratory test). Markman demands proof that use of the test improves clinical outcome. This has never been demonstrated for any laboratory or radiographic test, in any specialty of medicine, to my knowledge, and certainly not in cancer medicine. No one has ever shown that managing patients with the benefit of any laboratory or radiographic test has been shown to improve treatment outcome, compared to treatment of patients without benefit of the knowledge provided by test results. So Markman is presenting a standard which is entirely without precedent. It is also a standard which has not been applied by the FDA, as discussed below.
The standard previously applied to support the use of medical tests was the acceptable accuracy of the test and clinical utility, in the judgement of the physician ordering the test, supported by clinical logic and common sense. This was also the standard applied by the FDA in its consideration of a kit for cell culture drug resistance testing (for which FDA approval was received).
As discussed in the attached data and cited reviews and papers, the evidence is overwhelming that the assays can identify treatments associated with relatively good and bad prognoses with regard to tumor response and patient survival. Before discussing these data, I'd like to address the strong implication by Markman that the use of these tests has been shown in scientific studies not to improve clinical outcome.
The Markman reviews state and/or imply that the assays have not been shown to improve outcome. This is very misleading. In the first place, several of the trials certainly do show a trend suggesting improved outcome. That the trends for earlier studies were of borderline significance for response and not significant for survival would be obvious from the very small numbers of patients evaluated in each of these studies. Markman's editorials do not make note of the statistical power of these studies to show differences, presuming that true differences exist. Even a casual estimate indicates a power for each of the studies on the order of only about 0.1 or less, meaning that there was less than a 10% chance that a study of that size would show a difference, assuming that a real difference does exist.
It took studies in more than a thousand patients to "prove" the efficacy of FU/levamisole in colon cancer. It took a meta-analysis of studies with tens of thousands of patients to "prove" that adjuvant chemotherapy of post-menopausal breast cancer was worthwhile. And yet it is implied that studies with handfuls of patients somehow suggest that the use of these assays does not improve results of therapy (I am eliminating from consideration as irrelevant the Maenpaa study of the subrenal capsule assay in ovarian cancer quoted by Markman. This is an in vivo (not in vitro) assay, in which human tumors are implanted into the kidney capsules of mice. The assay was widely criticized years ago because of evidence presented by several different groups that the apparent "growth" of the tumor was largely owing to an infiltration of mouse inflammatory cells and, thus, drug effects were mainly being measured on mouse inflammatory cells and not the human tumor cells).
The most useful way to consider Markman's point of view is to quote from a recent study published in Clinical Cancer Research by the lung cancer group at the National Cancer Institute (Cortazar, et al. Clin Cancer Res. 3:741-47,'97).
In this latter study, the investigators used an assay technique which I invented (the DISC assay, Weisenthal, et al, Cancer Res 43:749-57,'83). Previously-untreated patients with small cell lung cancer underwent surgical biopsies (under general anesthesia, if necessary) solely for the purpose of obtaining tissue for cell culture drug resistance testing using the DISC assay. All patients initially received 4 cycles of cisplatin/etoposide. This was followed either by 4 cycles of individualized treatment based on DISC assay results (if available) or with 4 cycles of Cytoxan/Adriamycin/vincristine (if assay results not available). Following 8 cycles of chemotherapy, 75% of the standard therapy group achieved complete response (to either initial or initial plus secondary therapy), while 87.5% of the assay-directed group achieved complete response (all following assay-directed therapy, as there were no complete responders in this group after empiric initial cisplatin/etoposide chemotherapy). This difference was not significant, but the power of the study to show significance was less than 0.1, because of the small size of the study. The median actuarial survival of the handful of patients on whom DISC assay results were available was 38.5 months, compared to 19 months for patients treated with empiric chemotherapy, despite possibly superior prognostic characteristics of the latter group (included patients with complete responses to initial cisplatin/etoposide chemotherapy). Despite the small numbers of patients, these survival differences were statistically significant, in contradistinction to the other small studies quoted by Markman, which were also hampered by a low power (low study population size).
Most provocatively, there was a survival plateau of 37.5% (with the longest survivor followed beyond 8 years) in the assay-directed arm, while the survival curve for empiric therapy went down to zero within this time period. While the numbers of patients studied was very small, it is just as legitimate to quote from a positive small study as it is to quote from ostensibly "negative" studies which showed response but not survival benefits or trend response benefits which fell short of significance because of small numbers of patients.
The true state of the literature which addresses the issue of benefit to assay-assisted therapy can be summarized as follows: the handful of published studies have been very small studies, which, in aggregate, point in the direction of improved response and which also point in the direction of improved survival, given the severe limitations imposed by study size. These findings are entirely in line with expectations based on the voluminous literature (cited in the attached reviews and in data presented below) showing the ability of cell culture assays to distinguish between treatments associated with favorable treatment outcome (response and survival) on one hand and unfavorable treatment outcome (response and survival) on the other hand.
To the extent that truly definitive data are lacking regarding the impact of assay results on response and survival, this is not owing to the fact that studies were done and assays were found to be lacking, but rather that the studies performed were hopelessly inadequate in scope to seriously address the issue.
So why haven't the studies been done? To quote the National Cancer Institute Group which reported the above study in small cell lung cancer: "If the trend in prolonged survival observed in our patients treated with an IVBR [in vitro best regimen] continued, a study would require 91 patients per arm to demonstrate a 20% difference in two year survival at the P = 0.05 level with 80% power. However, because only 15% of the patients entering our study were able to be treated with an IVBR, we would need to enter 1229 patients to find a 20% difference in survival. We do not believe that such a trial is justified at the present time....We suggest that investigators plan future trials to be able to obtain tumor tissues from a large number of patients entering a study."
This is the fact which plagues these studies - that of a shrinking denominator. Thus, Von Hoff showed a significant response advantage for a handful of endstage patients treated with assay-directed therapy. But Markman doesn't accept this, noting that "considering all 133 patients [most of which could not be successfully assayed], there was no significant difference." Traditionally, controlled clinical trials demand data analysis based on "intention to treat." But this presents insurmountable variables in situations in which many patients are ineligible to receive assay-assisted therapy because of a myriad of clinical or logistical reasons. This requires a potentially huge starting number of patients, even if the analysis of results is based only on patients actually treated with assay-directed therapy. But Markman would not accept the results of such a study, unless all patients were included in the analysis, correctly worrying about selection bias. But if an intention to treat standard is used, the pool of required patients can easily reach into the several thousands to more than 10,000, as can be seen from an armchair analysis of the NCI data. So there is no wonder that the NCI investigators "do not believe that such a trial is justified a the present time," despite their provocative results which are entirely consistent with reasonable expectations based on the more persuasive clinical correlation data described below and in the attached reviews. And even if such a study were to be successfully completed, the study would become instantly irrelevant were the standard empiric therapy to be changed owing to the introduction of new drugs or the testing of new empiric combinations, as explained by Markman (J Cancer Res Clin Oncol 121:441,'95).
And who is motivated to carry out such trials? All of the major technologies are either public domain or offer absolutely no advantage over universally-available public domain technologies which essentially do the same thing. What for profit entity is going to invest millions of dollars and years of time into doing a study which is likely to be instantly obsolete and only to pave the way for others to leap into the "market" without the R & D investment? What third party payer is going to invest in a study which will result in improved treatments which may cure more patients but which would also undoubtedly provide superior palliation and keep more cancer patients alive for longer periods of time, at greater ultimate cost to the third party payers, quite apart from the expense of the assays, themselves? What university clinical investigators want to get involved in studies which divert patients away from much easier, straight-forward protocols which will generate more publications and more grant and contract support for much less effort? As noted recently by the President of Johns Hopkins University, William Brody, MD, PhD: "It is the very integrity of the patent system that fosters the large financial investments needed to bring medical advances through the development stage to reach the bedside." These cell culture assays are truly orphan technologies. No one will do such a complex study, which will, in the end, apply only to one particular situation and which will be instantly obsolete should the standard empiric therapy ever change. This is especially true when there is no patent protection or other proprietary status for the assays.
It is very important to note that none of the above considerations apply at the level of the individual patient who has been pre-selected to receive a cell culture study by virtue of clinical situation and availability of tissue for sampling and testing. Thus, the fact that many patients with recurrent breast cancer only have disease in their bones or livers which cannot be reasonably biopsied is great relevance to the feasibility of a prospective clinical trial but is of no relevance to an individual patient who happens to have a superficial recurrence of a fleshy mass at the mastectomy site or in a supraclavicular node or malignant effusion.