The Cortazar/Johnson Review and the NCI Lung Cancer Studies

1. The Cortazar/Johnson Review

The Cortazar/Johnson review (J Clin Oncol 17:1625-31,'99) represents a substantial amount of work, and the authors took great pains to be comprehensive, accurate, and fair. The authors have accurately described the substantial logistical barriers to completing studies such as this and offered helpful suggestions for future studies. But the take-home messages come through loud and clear. The studies analyzed suggest a response benefit, but do not confirm a survival benefit. The authors conclude that prospective clinical trials "have not yet demonstrated a clear benefit of chemotherapy using drug sensitivity testing." In 2 less comprehensive recent reviews, Markman recently made the same points, as discussed above (Markman, M. Chemosensitivity and chemoresistance assays: are they clinically relevant? J. Cancer Res. Clin. Oncol. 121, 441-442,'95; Brown, E. and Markman, M. Tumor chemosensitivity and chemoresistance assays. Cancer 77, 1020-1025,'96). It is, however, extremely misleading to attempt a meta-analysis on the data. There are limitations to all studies in which one lumps together data from various studies and calculates overall means or medians. In this case, such lumping is particularly problematic. The data in the Cortazar/Johnson review which attract the most attention are those data comparing survivorship with assay-directed versus empiric therapy. Indeed, it is likely that the take-home message for most readers will be that, in a fairly large total dataset of 13 prospective studies and 569 patients (which was actually only 5 studies and 92 patients, omitting one study which was inappropriately included, see below) receiving DST-directed therapy, the median survival of DST-directed therapy was 8.9 months, compared to 10.9 months for empiric therapy and that "only one non-randomized study showed patients treated with [DST-directed therapy] had a significantly longer survival than the group of patients treated with empiric chemotherapy." Even more simply and directly, the take-home messages of the above would appear to be clear: 1. There have been a lot of studies and a lot of patients, providing a reasonably fair test of the hypothesis that clinical chemotherapy directed by DST may improve clinical outcome. 2. The data strongly imply that there is no benefit to clinical chemotherapy directed by DST.

The above are outrageously misleading messages, unsupported by the studies and data included in this paper.

It is accurate to state that it has not been proven that the use of these assays improves the treatment of populations of patients in prospective, randomized trials (which have never been carried out and completed). It is not accurate to state nor helpful to imply that the hypothesis has been adequately tested and found not to be true. It is also unfair not to consider at the same time the considerable data which show that when patients are treated with drugs active in the assays that they have vastly superior reponse and survival rates than when they are treated with drugs which are not active in the assays.

To begin with, in the Cortazar/Johnson review, it is not appropriate to include the study by Maenpaa, et al (authors' reference # 26). This was a study of the subrenal capsule assay, which is an in vivo assay which has never been utilized in this country or anywhere else in the non-investigational clinical setting. This assay was criticized in several publications by several different groups, who reported that the alleged tumor "growth" measured in the assay was mostly owing to host inflammatory response against the implanted tumor and was, thus, an artifact. To my knowledge, this assay has never been used by any of the private sector laboratories and has been abandoned as a research tool.

Remaining are 5 studies, 2 of which utilize the "human tumor colony assay" ("HTCA") and 3 of which utilized the "DiSC" assay. A total of 92 patients in these 5 studies actually received assay-directed therapy. The HTCA and DiSC assay are completely different technologies. The HTCA measures inhibition of cell proliferation. It utilizes very low drug concentrations. It is a relatively long-term assay. It suffers from a number of serious technical problems (which have been enumerated in a number of review papers, going back to the early 1980s). It has a very low evaluability rate. And it hasn't been used by any private sector clinical laboratory in at least the past 13 years, to my knowledge. In contrast, the DiSC assay measures cell death (delayed loss of membrane integrity, which is what is measured by the DiSC assay, has been shown to correlate with apoptosis). It uses markedly higher drug concentrations than does the HTCA. It is a short-term assay. It poses entirely different technical challenges than does the HTCA. It has a very high evaluability rate, as discussed below. It has been used by private sector clinical laboratories since 1987 and it continues to be used by the three most prominent private sector laboratories which are providing cell culture drug resistance testing in this country today.

Basically, the authors lumped together 6 vastly differing studies, none of which had a control patient population which would be accepted in any journal publication comparing different forms of empiric therapy (i.e. the typical oncology journal chemotherapy paper), and quoted the following median survivals for patients receiving assay-directed therapy (in months): 6.2, 7.2, 8.6, 9.1, 24, and 38.5. They took the median of this series of 6 papers of different technologies and came up with a number of 8.9 (months) to represent the overall median result of assay-directed therapy upon meta-analysis. Then they took the following median survivals for patients receiving empiric therapy (who did not represent scientifically valid control groups): 4.2, 7.3, 10.6, 11.2, 19, and 28. The median of this series came out to be 10.9 and this was stated to represent the median survival for patients receiving empiric therapy. Lumping together non-randomized data from widely varying studies of entirely different assay endpoints (cell proliferation versus cell death) which are not in any way comparable for meta-analysis to obtain parameters such as median survivals and then to make comparisons of these median survivals is not scientifically justifiable.

Had the median survival with this sort of fractured analysis showed a "signficant" improvement, there is no way on earth the Journal's editor would ever have accepted this for publication. Only because the non-significance of the finding fit in with the Editor's pre-conceived bias was it published. Since it was the authors' intention to evaluate the "efficacy" of DST-selected therapy, it would seem most appropriate to first evaluate the quality of the data which exist to make this determination. In order to be useful to the many hundreds of clinical oncologists who continue to use this DST in their clinical practices, it would also be helpful to note how the technologies evaluated in this review compare with the technologies which are currently being applied as a service to patients.

As the HTCA is not one of the technologies which has been used by the private sector laboratories, but as (technically-evolved) versions of the DiSC assay are used, it would appear that the only papers discussing survivorship which are relevant to present-day clinical practice are the three lung cancer studies from the NCI which ostensibly utilized the DiSC assay (authors' refs. 9, 10, 39, 11, 12). The problems at the core of these studies are described in the following section.

2. The NCI Lung Cancer Studies

The NCI (along with Cortazar and Johnson, in their review) has taken the position that the use of these assays is to be considered investigational. This position is not based on an examination the relevant literature reviewed in in the present paper, but appears to be based entirely on a series of highly-flawed studies published by the NCI lung cancer group (JNCI 82:117,'90; J Cell Biochem 24:173,'96; Clin Cancer Res 3:741,'97). I am very familiar with these three studies, which had their origins in a parking lot conversation which I had with Dr. John Minna at the San Diego ASCO meeting in the early 80s. Although I have tremendous respect for the above investigators and great appreciation for their efforts over the years, it must be noted that this body of work suffers greatly from a number of problems which are not relevant the application of these assays in clinical practice today and which are, more importantly, not applicable to the data which exist to justify the use of these assays in clinical practice today. The NCI's three studies were in (1) extensive stage small cell lung cancer, (2) advanced non-small cell lung cancer, and (3) limited stage small cell lung cancer. Study # 1 (JNCI 82:117,'90) can be described as being modestly "positive." Study # 3 (Clin Cancer Res 3:741,'97) can be described as being highly "positive." However, study # 2 in non-small cell lung cancer (Cancer Res 53:5181,'93; J Cell Biochem 24:173,'96) was certainly highly "negative."

As the NSCLC study was, by far, the most "negative" of the three, and it is worth considering in some detail, to illustrate the problems with drawing the conclusion that the "efficacy" of DST-directed therapy has been adequately tested and has been found wanting. I apologize in advance for the detailed complexity of the following critique, but it is extremely important to understand how irrelevant this study was to the use of CCDRT in the clinical management of cancer patients.

Shaw, et al studied assay-directed therapy of NSCLC. The Methods section (Cancer Res 53:5181, '93) stated that the assay "results included in this report were performed on fresh tumor tissue." This statement was added by the authors to the Methods section after I reviewed a pre-publication draft of this paper and noted the problems with testing sub-cultured cells (described below). In point of fact, the description in the Methods was incorrect and none of the results were performed on fresh tumor tissue. All assays were, in fact, performed on short-term cell lines (i.e. subcultured cells), which had at least one and often more passages in vitro. I confirmed this by speaking with Dr. Adi Gazdar (who was the investigator who actually performed the assays) and, in fact, the authors confirmed this with their own words in the Discussion section of the paper, in which they stated "subculturing also specifically excluded fibroblasts and benign epithelial cells." This is very important, as the assay used by the NCI was designed (by me) to be a "total cell kill" (apoptotic) assay and not an antiproliferative assay. All of the drug concentrations and assay conditions were designed for fresh tumors and to test the total (largely non-dividing) tumor cell population. This endpoint is much more robust for specific cell killing mediated through apoptosis than are cell proliferation assays which measure non-specific growth inhibition effects. Once you start working with passaged, short-term cell lines, you begin to measure tumor cell subpopulations and you begin to measure antiproliferative effects along with the more specific apoptotic effects. All of the other investigators who have reported excellent correlations with total cell kill assays (more than 1600 published assay/treatment clinical correlation in all with these assays) have used true fresh tumor assays and not short-term cell lines. Yet the NCI applied this technology to cell lines and Shaw then concluded that these assays don't work, rather than more appropriately concluding that passaged cell lines don't work (e.g. see Kruczynski,A. and Kiss, R. Evidence of a direct relationship between the increase in the in vitro passage number of human non-small-lung cancer primocultures and their chemosensitivity. Anticancer Res. 13:507-14,'93; Smit, et al. In vitro response of human small-cell lung-cancer cell lines to chemotherapeutic drugs: no correlation with clinical data. Int. J Cancer 51:72-78,'92.).

The purpose of the NCI study never was to determine if fresh tumor assays worked. All of the considerable literature which supports the use of these assays in patient management has been based on true fresh tumor (non-passaged) cell assays. The NCI used cell lines because the major expertise of the investigators who carried out the study was in the creation of lung cancer cell lines, and they wanted to see if they could perform assays on these cell lines to use in patient therapy. The results were that they were able to test successfully only 22% of specimens received, including only 7% of primary lung lesions! This contrasts with a 75% overall success rate reported by earlier investigators who used the same assay system in fresh tumors (Wilbur, et al Br J Cancer) and a routinely obtained > 95% success rate using improved methods available today.

By any definition, the NCI studied a sub-population of patients (22% of all patients and only 7% of patients with tumor obtained from the lung). What else is known of this sub-population of patients? It turns out that these authors had previously shown that patients whose tumor cells give rise to immortalized cell lines have a very poor prognosis, relative to patients whose tumor cells do not give rise to immortalized lines (Ann Intern Med 113:764, 1990). Fully 62% of the patients treated with assay-directed chemotherapy in the NCI study had tumors which gave rise to immortalized cell lines, compared to 24% of all patients and 12% of patients in whom assay results could not be obtained (the control group). Additionally, 35% of specimens from metastatic sites could be tested, compared to only 7% from primary lung tumors. The authors themselves noted that "the same factors associated with metastatic potential could enhance in vitro cell survival." Thus, the subset of patients actually receiving assay-directed therapy were largely members of a group of patients previously identified as having a particularly poor prognosis, relative to a larger group of patients, in a disease where even the most favorable group itself is known to be associated with a very poor prognosis. In a disease such as NSCLC, where the benefits of chemotherapy are well-known to be marginal at best, one might reasonably expect the patients most likely to derive benefit would be those patients with the most favorable prognosis and not the patients with the least favorable prognosis. And yet the patients who received assay-directed therapy were those patients with the least favorable prognosis and these patients were compared to the much larger group of patients who had a significantly more favorable prognosis!

What is needed in all studies, obviously, is a control group that consists of patients who have assays successfully performed, but whom do not receive assay-directed therapy. But such a control group is lacking in all studies published to date. Additionally, all of the work in the past 15 years in the field of total cell assays in epithelial tumors has been carried out largely on three dimensional clusters of cells. I cited the testing of cells in the absence of cell-cell communication as a serious flaw in a letter published in the NEJM in 1983 (Weisenthal, LM. Human tumor stem cell assay. New Eng J Med 308:1478-79, 1983). In my laboratory, we throw away the single cells and attempt to work exclusively with three dimensional, floating, tumor spheroids. What was used in the NCI study of NSCLC? Monolayer cell cultures (source: personal communication with Adi Gazdar). Why is this important? In some of the most important work in this field published in the last 10 years, Teicher at Harvard and Kerbel from MD Anderson independently showed that you can create drug resistant tumor cells in mice by repeatedly treating the mice with chemotherapy and then harvesting the surviving cells from the mice, transplanting them into new mice, retreating the new mice, and so on. After a while, you get cells which are many-fold resistant to the chemotherapy, when tested in mice, but absolutely no more resistant, when tested in monolayer assays in vitro. But if you test the same cells as three-dimensional spheroids, they are now many-fold resistant in vitro, just as they are in vivo. (The authors call this "multicellular resistance," and it may be due to any number of things. For example, cells can pass glutathione back and forth to each other (analogous to bacterial conjugation). Additionally, cells in clusters have been shown to be resistant to apoptotic stimuli). The fact that the NCI studies in SCLC were more "positive" than those in NSCLC may well be related to the fact that small cell lung cancer cells, in most cases, do not grow as monolayers but rather do not attach to the surface of the culture plates - instead growing as floating, three-dimensional spheroids (the way that all fresh tumors are cultured for testing in my laboratory). In contrast, non-small lung cancer cells grow as monolayers and the studies of Teicher and Kerbel show the biological irrelevance of monolayer cultures to tumor cell resistance to anticancer drugs.

Also very important is the fact that the NCI authors themselves note that patients who have tissue easily available for biopsy for testing represent a population with more advanced disease and a less favorable prognosis than patients who do not have such superficial tumor tissue available. Finally, the clinical impact of assay-directed therapy in the (generally positive) small cell lung cancer studies (JNCI 82:117,'90; Clin Cancer Res 3:741,'97) was certainly attenuated by the fact that the assay-directed therapy was not given until after 12 weeks and 4 complete cycles of empiric chemotherapy.

In short, there are absolutely no findings or conclusions in the NCI non-small cell lung cancer study which are in any way germane to the vast contemporary literature which strongly supports the clinical use of these assays, when they are performed by expert laboratories. Yet, within the academic community, it is not uncommon to hear, as I have heard repeatedly, things like "well, they tried that at the NCI and it didn't work. If they couldn't get it to work at the Mecca of Meccas, what makes you think that you can get it to work?" And that's the problem with these NCI studies. Good investigators. Good institution. Good journals. But vaguely-described methods, misleading analysis, and misleading presentation of data. And it mainly serves to reinforce the opinion held by many in academic oncology that cell culture drug resistance testing has been adequately studied, but has been found wanting to the extent that further studies are of vanishingly low priority and that clinical application is not to be supported.