Letter to Dr. Naomi Aronsen, Director of the BlueCross/BlueShield Association Technology Evaluation Center, in which I critique the BlueCross/BlueShield Association document, entitled "Technology Evaluation Center: Chemotherapy Sensitivity and Resistance Assays," which was offered in evidence by Wellpoint Blue Cross during the trial of "Doe" vs. Blue Cross (California SNA013292). This document was stated by Wellpoint Blue Cross to be available on the Web, but I have been unable to locate it to provide a link.

May 4, 2002

Naomi Aronsen, Ph.D.
Technology Evaluation Center
BlueCross/Blue Shield Association
225 N. Michigan Avenue
Chicago, IL 60601-7680

Dear Dr. Aronsen:

I am enclosing a number of documents relating to Cell Culture Drug Resistance Testing (CCDRT). Additional information may be found at the Human Tumor Assay Journal website http://weisenthal.org. This information should be self-explanatory.

The documents include:

1. A rebuttal to the BlueCross/BlueShield Association document, entitled "Technology Evaluation Center: Chemotherapy Sensitivity and Resistance Assays (hereafter referred to as TEC:CSRA)." This is a letter to San Diego Superior Court Judge Yuri Hoffman, in support of the plaintiff (a Wellpoint Blue Cross subscriber) in a lawsuit against Wellpoint Blue Cross, for non-payment of her medical expenses relating to CCDRT.

2. A copy of a review I prepared in 1999, entitled "Cell Culture Drug Resistance Testing (CCDRT): misconceptions versus objective data." This appeared to be a major source for the disputed TEC:CSRA review. It is published only on the Human Tumor Assay Journal website htt://weisenthal.org

3. A copy of a review I was invited to write for the journal ONCOLOGY (NY). This is, I believe, the most up to date and comprehensive review ever written on the topic of CCDRT. Since I wrote this paper, there have been two additional papers of relevance: one relating to cell proliferation assays and the other relating to cell death assays. Both of these papers reported "positive" results and are being including in my revision of this manuscript (in progress). Note that the ONCOLOGY reviewers found no fault whatsoever with any of the factual material in this review (no errors, omissions, or misleading descriptions). I also enclose a copy of the single written review of this manuscript which was provided to me, along with a brief response written by me.

4. A "frequently asked questions" summary which, I think, provides a nice overview of the relevant issues, technologies, and laboratories currently providing variations of CCDRT services.

5. A collection of data and discussions relating to ovarian cancer, which is, I think, a very good example of how 30 years of clinical trials research has produced absolutely nothing of any importance. We have yet to prove, for example, that any form of therapy is superior to the single agent, oral melphalan which was being used in the 1960s.

In most other areas of cancer chemotherapy, the few things conclusively proven by prospective, randomized trials have represented truly minuscule improvements, which achieved statistical significance only through the enrollment of thousands of patients, in the primary and confirmatory trials. And the true advantage to patients in these situations is still highly debatable. For example, adjuvant chemotherapy of post-menopausal breast cancer may produce trivial degrees of improved survival, but we are only now learning that such chemotherapy may produce cognitive dysfunction ("chemo brain") and it will take probably another two decades of clinical research to document the magnitude of this problem and to figure out whether chemotherapy in this (and other) situation(s) is worthwhile, afterall.

What needs to be clearly understood when scientists such as yourself interact with medical oncologists such as Drs. Armitage and Henderson of your TEC review committee is that there is a disconnect between the illusion promoted by academic clinical oncologists and the reality of the published medical database. Drs. Armitage and Henderson would have you believe that oncology is a true scientific discipline, and that the clinical practice of oncology is based entirely on scientific principles which have proven in the proverbial "well-designed clinical trials." This is by no means the case.

As only one example, I have related the example (cited in my letter to Judge Hoffman, P. 16), of "following" disease progression versus response by means of expensive, serial radiographic imaging studies. The cost of doing this is quite substantial; the true value to the patient is doubtful, yet no one has ever challenged it or proposed a prospective, randomized trial to determine if it is necessary or even helpful. Do you know why? Because academic institutions count on insurance reimbursement to pay for these studies, which are deemed necessary to obtain clinical trials data for such parameters as response rates and time to disease progression.

The point I am making here is that the Dr. Armitages and Hendersons of the world have a double standard (and I believe it to be a truly cynical double standard) of not demanding proof of things they do believe in (e.g.,

(1) the value of expensive serial imaging studies to "follow" disease;
(2) the concept that an overall 2% survival advantage proven in a meta-analysis of tens of thousands of patients establishes a standard of care, without a clear understanding of issues such as quality of life and potential cognitive dysfunction;
(3) the superiority of platinum + Taxol over single agent platinum, etc.,),

but demanding proof of things that they don't believe in (e.g. that CCDRT doesn't work; can never work; and will never work)

It is because of the belief that CCDRT doesn't work, can never work, and will never work that allows for the unwarranted certitude of dismissing a very rich but complex field of clinical science inquiry on the basis of the superficial and shoddy scholarship which went into TEC:CSRA review.

It wasn't worth the effort to try and learn about the technology -- to learn the differences between cell proliferation assays and cell death assays and monolayer assays and cell cluster assays, so that it was possible to analyze the existing clinical data in an intelligent manner, and to learn the complicated Bayesian statistics which apply to the use of the technology, and to try harder to understand how CCDRT may be appropriately used and is being used in the real world where patients must be treated today, on the basis of imperfect knowledge in many areas of clinical oncology.

To provide just two general examples, consider the following typical clinical situations:

Example 1. "Common" (high prevalence) situation: Patient with stage III breast cancer still has sizable (minimally responsive) lesion following neoadjuvant chemotherapy with cyclophosphamide + epirubicin. What, if anything, is to be used for post-operative adjuvant therapy? Ask Drs. Armitage and Henderson the following question: In their heart of hearts, do they really think that a prospective clinical trial randomizing patients to one of the following regimens would really identify a clear cut "winner?":

Single agent paclitaxel, single agent docetaxel, high dose ifosfamide + carboplatin + etoposide with stem cell/growth factor support, high dose mitoxantrone + melphalan + carboplatin with stem cell/growth factor support, high dose cyclophosphamide + cisplatin + carmustine with stem cell/growth factor support, high dose cyclophosphamide + thiotepa with stem cell/growth factor support, paclitaxel + mitoxantrone + 5FU/leucovorin, high dose melphalan + etoposide, high dose melphalan + thiotepa, paclitaxel + cisplatin + 5FU/leucovorin, vinorelbine + mitomycin c, vinorelbine + mitoxantrone, mitomycin c + mitoxantrone + melphalan, irinotecan + 5FU/leucovorin, carboplatin + taxol, gemcitabine + cisplatin, gemcitabine + cisplatin + vinorelbine, + cisplatin, irinotecan + cisplatin, topotecan + cisplatin, gemcitabine + vinorelbine, docetaxel + vinorelbine, paclitaxel + vinorelbine, gemcitabine + ifosfamide, and I could go on and on and on.

In my opinion, every living, practicing oncologist could easily pick out a half dozen of the above regimens which he/she would view as being absolutely equivalent, with regard to probability of clinical efficacy. In my opinion, I think that all of the above regimens (and many others) would produce equivalent results, administered to randomly-selected populations of patients. Yet, they are certainly not equivalent on an individual patient basis. Some patients will have lifesaving benefit from particular regimens, whereas other patients will experience only toxicity (not infrequently quite morbid and occasionally fatal) with the same regimens. Some patients who experienced only toxicity and not benefit would go on to experience substantial benefit for other of the above regimens, administered as so-called "salvage" therapy. These patients should have been given the correct regimen the first time around, for a variety of good reasons with which Drs. Armitage and Henderson would agree.

Example 2. "Uncommon" (relatively low prevalence) situation:

Esthesioneuroblastoma is a very uncommon disease. There is no good clinical trials literature to guide drug selection. Furthermore, there never will be a good clinical trials literature to guide drug selection.

In between the above two extremes, there are a hundred other situations where "existing knowledge" does not provide a clear-cut choice as to which form of cancer chemotherapy is most likely to help an invididual patient (and insurance beneficiary) with a life-threatenting disease.

Now, I don't want you to ask Drs. Armitage and Henderson the following question (they are hopelessly compromised by pre-existing biases), but I am going to ask you to ask yourself the following question. First, read the enclosed review paper by Weisenthal and Nygren. Second, ask yourself the following question: "it it credible that, everything else being equal, drugs which kill tumors (under rigidly-controlled, identically comparative conditions) in the laboratory are statistically more likely to kill the same tumors in the patient than are drugs which don't kill the tumors in the laboratory?

Now, if you answer that last question in the affirmative and if you understand the clinical examples (stage III breast cancer and esthesioneuroblastoma) given above, you will understand how unbelievably stupid was the statement appearing on page 11 of the TEC:CSRA review: "The PPV of 79.1% indicates that a positive test result would be false in 20.9% of cases; a rate that may concern practioners and patients."

Let me explain this in the way I explained it to Judge Hofmann: "Note that the unknown authors of the BC/BS Association document raise a very misleading concern on page 11 of their document: 'The PPV [positive predictive value] of 79.1% indicates that a positive test result would be false in 20.9% of cases; a rate that may concern practitioners and patients. The NPV [negative predictive value] of 87.4% shows that a negative test result would be false 22.6% of the time.'

Firstly, their math is wrong. An NPV of 87.4% actually shows that a negative test result (in the theoretical population of patients under consideration) would be false only 12.6% of the time, and not 22.6% of the time. Of much greater importance, however, what this means is that patients treated with a "positive" ("sensitive") drug would respond 79.1% of the time, while patients treated with a "negative" ("resistant") drug would respond only 12.6% of the time. Once again, there would be a huge advantage to the patient to receive a "positive/sensitive" drug, compared to a "negative/resistant" drug."

This really gets to the heart of the matter. The two most important questions which relate to the justification for using these tests in clinical oncology are the following:

1. Is there a legitimate choice between treatments of otherwise equally-probable efficacy, absent the performance of CCDRT? (in most situations in clinical oncology, this is indeed the case, as in the example above).

2. Is there credible information to support the concept that clinical drug activity is significantly associated with laboratory drug activity?

Now, one can not merely ask the above questions in a vacuum. They must be asked in the context of the following considerations:

1. The fact that life or death medical decisions must be made on the basis of existing knowledge and not on the basis of ideally perfect knowledge (i.e. particularly in cancer medicine, the appropriate standard should be preponderance of existing evidence and not proof beyond reasonable doubt, which is a standard which currently applies to almost nothing relating to the efficacy of cancer chemotherapy).

2. The fact that there is a pre-existing standard relating to the use of medical tests, radiographic tests, and laboratory tests, such as estrogen receptor; Her2/neu; expensive panels of immunohistochemical stains to assist in the diagnosis and categorization of cancer, which, in turn, influences drug selection in chemotherapy as much as CCDRT may influence drug selection; and the serial radiographic imaging tests described above (which function, in this context, also to influence drug selection, by determining if a given treatment should be continued or changed).

The pre-existing standard has (appropriately, in my opinion) always been documentation of a useful degree of accuracy, and not proof of "efficacy," as explained in my letter to Judge Hofmann.

Lastly, I would like to try and explain to you (a layperson as regards cancer medicine, but also a Ph.D. scientist [I presume; I don't know in what discipline you obtained your doctorate]) why it is that such impressively credentialled academics as Dr. Armitage and Henderson can be so "wrong" and a simple private practioner working in what he calls the "Weisenthal Cancer Group" can be so "right." I don't blame you for "believing" "them" and not "believing" "me." (When I say "you," I'm referring to your entire technology assessment group; basically you guys approached your review with the strong bias that CCDRT was not currently supported in the general academic community, and therefore the superficial level of scholarship which went into the TEC:CSRA review reflected the fact that you knew that you were preaching to the choir and that strong scholarship with fresh, original thinking were not required).

So why are Armitage and Henderson so certain that this doesn't work?

1. Syd Salmon and Dan Von Hoff convinced the world that "The Human Tumor Stem Cell Assay" was the next great thing. It wasn't (the New England Journal giveth and the New England Journal taketh away, referenced elsewhere in my writings on this topic). Then Dan Von Hoff basically went around telling everyone that, based on his own considerable personal experience, the field of CCDRT wasn't going anywhere and that the future would be with tests based on molecular biology. The academic community said "hear, hear," and unconditionally refused to support work in the field of CCDRT, from basic laboratory studies to clinical trials.

2. The National Cancer Institute made a sincere, but fatally-flawed attempt to study CCDRT in lung cancer. This is extensively discussed in my "CCDRT: Objective Data vs Misunderstandings" paper which you referenced in the TEC:CSRA review.

In the case of every single patient, the NCI CCDRT tests were not performed on true fresh tumors, but rather on subcultured cells. The major expertise of the laboratory performing the studies was in the establishment of cell lines, and the laboratory was more interested in determining if its cell line amplification technology would work than it was in determining whether or not true fresh tumor (non-culture-selected cells) would work. Additionally, in the case of their study in non-small cell lung cancer (by far the most "negative" of their studies), the tests were performed on cells growing in monolayer cultures, rather than on three dimensional cell clusters.

This sounds like a trivial, arcane distinction, but, if you go to the National Library of Medicine's on-line PubMed and enter multicellular [titl] AND resistance [titl] and then just browse the 30 or so abstracts, you will clearly understand the problem with the NCI NSCLC studies. Example: create a line of murine cancer cells which is 100-fold drug resistant by passing the cells in mice, while simultaneously treating the mice with chemotherapy. Test these cells in vitro, in monolayers, and there is no difference between the wild-type (non-resistant) cells and the 100-fold drug resistant cells. But test these cells in three dimensional clusters (as has been done in all the true fresh tumor assay papers I reviewed and as done in my laboratory), and they are 100-fold resistant in the cell cultures, as well as in the mouse.

Then there is the whole issue of proliferating cell assays versus cell death assays. I have supervised the performance of more than 10,000 fresh tumor assays (i.e. assays performed on tumors removed from more than 10,000 human patients) using proliferating cell endpoints and nearly the same number with cell death endpoints. I have had hundreds of specimens in which I had data comparing cell proliferation endpoint results to cell death endpoint results, and thousands of cases where I have made head to head comparisons of different cell death endpoints (most extensively cell death by morphology on stained microscope slides versus mitochondrial Krebs cycle reduction of the MTT reagent versus cellular ATP content). What I can assure you is that cell proliferation correlates poorly with cell death (as just a single example, the drug concentrations used in cell proliferation assays are an order of magnitude lower than those used in cell death assays; this relates to complex factors such as "generationally-delayed cell death" and "multi-log kill," on which I've published extensively, but won't review in this letter). I can also assure you that the different cell death endpoints, with only rare, isolated exceptions relating to specific drugs, for which there are perfectly understandable reasons why one cell death endpoint may be preferred over another, correlate very well with each other (justifying the sort of meta-analyses of the different cell death endpoints which I describe in the enclosed papers).

To make things very simple, however, the case I am making is for the clinical use of true fresh tumor assays, not tested in monolayers, using the following cell death endpoints (which have been documented in the peer-review literature to correlate very well with one another) DISC, MTT, ATP, and fluorescein diacetate. There is a very extensive literature relating to these assays. And the only clinical trials literature which is relevant is the literature which uses true fresh (not subcultured) tumors, not tested in monolayers, and using one or more of the above endpoints.

In the TEC:CSRA review "Review of Evidence" section (beginning on P. 19), 6 of the 7 reviewed studies used proliferating cell assays, or subcultured cells and/or monolayer cultured cells. In addition, you did not include the two most relevant studies. I critique your "Review of Evidence" on p. 13 of my letter to Judge Hofmann. I really did not intend to go hashing and rehashing so much in this cover letter. It is all covered in the enclosed material. And there is much more (such as our own, to-be-published data relating to patient survival). Then there is the whole issue of cost-effectiveness and how the whole issue of CCDRT relates to the various parties involved in cancer treatment - from insurance companies to practioners, as briefly summarized in the enclosed appendix.

I would like to close, however, with one addional criticism of your work. It is utterly incomprehensible to me that a technology assessment committee would have the audacity to put out a document such as your TEC:CSRA review, without making a minimal effort to have it peer-reviewed by scientists and practioners who actually know something about this very complicated area of medical inquiry. Had you sent me a draft of the document before it was discussed by your illustrious committee, I could have, in advance, pointed out omissions, incorrect conclusions, and other problems and weaknesses. In the end, you might or might not have arrived at different conclusions, but at least you would have been vastly better informed and, I think, you could have done a much better job of it. As it is now, you have done a real disservice to hundreds of thousands of suffering and dying cancer patients who are Blue Cross/Blue Shield Association member subscribers.

Do not take your job too lightly; I hope that you didn't get your Ph.D. just to follow the path of least resistance and tell your employers and the medical establishment what you know in advance they want to hear. You should be a real scientist; do real research; ask real outside-the-box questions; ask the right questions ("How do our standards for judging existing treatments and tests square with the standards we are currently applying to this new type of test? What is the risk/benefit ratio to the patient? Is this reasonable? Is it potentially helpful? Is it potentially life-saving? Would I want this for myself? Do I really understand the technologies well enough that I've chosen the correct dataset of clinical papers to examine?"). And so forth.

My last criticism is that your TEC review process and procedures are needlessly closed and opaque. Why not do it the way the Blue Shield of California (and Medicare) does it? Invite input. Arrive at and pre-publish preliminary conclusions. Invite all concerned parties to attend an open meeting, where all points of view can be discussed and debated. Then have an open, non-secret ballot vote. This level of scientific rigor and transparency would give your process the credibility which it currently (so-deservedly) lacks.

Sincerely yours,

Larry Weisenthal

cc: Christopher Loftin, Corporate Legal
WellPoint Blue Cross, Woodland Hills, CA