Evaluating Anecdotes and Case Reports

Evaluating Anecdotes and Case Reports

By Linda L. Isaacs, M.D.

Click here for PDF version

Stories are more memorable than statistics. Newspaper or magazine articles about some new chemotherapy for cancer will typically include an interview with a patient doing well on it. Web sites of alternative practitioners and clinics sometimes feature stories by or about patients describing their successful treatment. These stories can be very persuasive to readers, but some scientists dismiss such stories scornfully as “anecdotal” and therefore meaningless.

Case reports, which have been published in the medical literature for decades, are also stories — they are descriptions of individual patients with unusual presentations or outcomes. What distinguishes an anecdote from a case report? As one author puts it, “The term ‘anecdotal evidence’ connotes secondhand or poorly documented fact and should not be confused with case studies of individual patients that involve careful observation and recording of detail”. (1)

Anecdotes and case reports cannot be used to definitively prove a therapy is effective. But case reports cannot be dismissed entirely. As a recent article stated, “Case reports and series have a high sensitivity for detecting novelty and therefore remain one of the cornerstones of medical progress; they provide many new ideas in medicine.” (2) Case reports are a way to reveal the unknown or describe the previously unrecognized. They were critically important in the discovery of new diseases such as AIDS (3) or Lyme disease. (4) Descriptions of sick birds and sick people helped scientists figure out that West Nile virus had arrived in New York City (5), far from its usual location.

Case reports are the first steps in discovering unexpected drug effects, both good and bad. (6) For example, sildenafil (Viagra) was developed initially as a treatment for angina, but a side effect observed in individual cases led to its marketing as a treatment for erectile dysfunction. (7) Case reports led to the discovery of heart problems caused by drugs prescribed to suppress appetite (8), or muscle problems caused by contaminants in L-tryptophan. (9)

A well-written case report should provide clear evidence of the patient’s problem or condition and its treatment. In addition, it should provide a clear explanation of why the reader should be surprised by the outcome of the case, with appropriate references. (10)

Clear evidence of the patient’s initial problem or condition

Because I work primarily with a nutritional therapy for cancer, I will use cancer as an illustration throughout this piece, though the principles can be applied to other illnesses. It is not uncommon to read patient testimonials that claim a diagnosis such as cancer was made, but do not provide evidence that cancer was present. For example, I have seen cases where a patient found a breast mass, was told by a doctor that it was “clearly cancer,” refused surgical removal or biopsy, proceeded with some form of alternative therapy, and subsequently claimed miraculous cure.

The problem with this scenario is that cancer can only be diagnosed by a pathologist viewing a tissue sample under a microscope. It requires removal of tissue by biopsy or excision, or collection of samples for cytology. A doctor’s clinical impression of cancer or a suspicious test result does not prove the presence of cancer. As an example, in one study on the predictive ability of mammograms, lesions described as “highly predictive of malignancy” had an 81% chance of being found to be cancer when biopsy was performed – so 19% of the time, no cancer was present. (11)

In a case report, then, it should be clear exactly how the diagnosis was made. It should also be clear what treatment a patient might have received before embarking on the treatment that is being given credit for an unusual outcome.

Why the outcome is unusual

For a case report to be worth reporting, the outcome of the patient in question must be remarkable or unusual in some way. In the case of cancer, unusual results can be prolonged survival, or stabilization, shrinkage or disappearance of the tumor mass. Cancer by its nature grows and spreads; stabilization over a prolonged period, shrinkage or disappearance are all unusual for a biopsy proven cancer.

To evaluate survival, the patient’s case must be compared to a historical “control group” of the expected outcome of patients with their condition. For prolonged survival to be meaningful, it must extend well beyond the expected outcome — for example, if the usual prognosis is death five months after diagnosis, survival for six months is not impressive, but survival for 24 months is.

Expected outcome can be difficult for a layperson to assess, since they do not have the medical background to know what the usual outcome is for any given condition. A well written case history should describe the typical outcome, and the reference from which this information was obtained.

I have seen testimonials which state, “My doctor told me I would be dead within a year if I did not get the chemotherapy.” This statement is the only evidence given that the outcome is unusual. Unfortunately, statements such as this may or may not be correct — patients may not hear what their doctor said correctly, or in some cases, the doctor may be overstating the benefits of chemotherapy. For example, consider the following hypothetical scenarios:

A 60 year old woman has a lumpectomy and lymph node removal for breast cancer. She has a 1.5 cm tumor; the cancer has spread to two lymph nodes. She is told that she needs chemotherapy to prevent a recurrence. Her doctor tells her that “refusing chemotherapy is like committing suicide.” She refuses and instead decides to pursue an alternative treatment. Ten years later, she is alive with no evidence of cancer.

Another woman has a lumpectomy and lymph node removal for breast cancer. She has two positive lymph nodes, receives chemotherapy, and one year later is found to have recurrent disease with tumors in the liver and brain. She refuses further chemotherapy, decides to pursue an alternative treatment, but has great difficulty in following it because of symptoms from her cancer. Three years after her diagnosis of metastatic disease, she dies.

Which outcome is unusual? In the first scenario, to assess her probable outcome after surgery, I went to the website Adjuvant! Online. This site provides information to help health professionals and patients with early stage cancer discuss the risks and benefits of getting additional therapy after the cancer has been surgically removed. This patient had a 52% chance of being alive and without evidence of cancer after 10 years with surgical treatment alone. (12) So her good outcome is not unusual. Had she gotten chemotherapy and hormone therapy, according to Adjuvant Online, her chances of being alive and without evidence of cancer in 10 years would have improved to 63%.

In the second hypothetical case, while review of the medical literature cannot answer precisely how long a patient with metastases to both the liver and brain might live, one article reports that breast cancer patients with metastatic disease to the brain have a median survival of 29 weeks (about 7 months). (13) In another study of patients with liver metastases from breast cancer, the median survival was 14 months. (14) In both cases, unlike our hypothetical patient, the study subjects received orthodox treatment for their illness. Even though the patient died, living three years with metastatic disease in the locations listed is unusual. Her case report is of interest, even though she died.

Limitations of case studies

Case studies are good for picking up novelty, but they have limitations. Generally speaking, a case report cannot prove that the treatment described is actually what created or caused the desired result. And a case report cannot indicate if the experience described is typical; only statistical analysis of a larger treatment group, compared to a clearly defined control group, can do that.

There are some situations where case reports are accepted as definitive evidence — a single case report can prove that a drug caused an adverse reaction. (15,16,17) The case report should include the following:

Challenge — the reaction occurs when the drug is given
De-challenge — the reaction resolves when the drug is stopped
Re-challenge — the reaction recurs again when the drug is given

In the case of the treatment of disease, a similar scenario can create a stronger case report. For example, a patient under my colleague Dr. Gonzalez’s care presented with metastatic breast cancer which resolved while she followed his recommendations, recurred when she quit, and resolved again when she resumed her nutritional protocol. This would argue that her nutritional protocol was effective against her disease.

Case reports can point more strongly to the treatment as the cause of the outcome when more than one is presented simultaneously. Sometimes individual cases of resolution of cancer are discounted as “spontaneous remissions.” Spontaneous remissions, in the medical literature, refer to complete or partial resolution of cancer for no reason that the physician can discover or credit. Spontaneous remissions are not common. The author of a book (18) on spontaneous remission of cancer, Warren H. Cole, M.D., said in a 1974 interview: “The phenomenon is extremely rare. Some investigators estimate the incidence is as low as one in 100,000 cancer patients.” (19) Given their rarity, it would be unusual for a single practitioner to see even one case of spontaneous remission during their career, let alone two or three. This would suggest that if an alternative practitioner can provide more than a few case reports with clear-cut unusual outcomes, there may be something worth investigating in that treatment.

The outcome described in a case report may not be the typical experience for patients pursuing a particular treatment. As an example, the drug Iressa created great excitement when it was first introduced for lung cancer, because some patients in initial case reports had amazing resolution of their disease. (20,21) The Food and Drug Administration approved it for use outside of research studies in May 2003 under its accelerated approval regulations. But when the drug was more extensively tested in controlled clinical trials, it was found that very few patients actually had any response. (22) Overall, there was no improvement in survival. (23)

Case reports as basis for further research

Case reports ideally provide the basis for more definitive research. In studies such as randomized controlled trials, patients selected according to very specific criteria are assigned to treatment groups and followed prospectively. This can demonstrate whether the treatment is causing the desired effect, and show in what percentage of patients the desired effect is achieved. Clinical trials are relatively simple (though expensive) to implement in the case of a drug; for interventions such as dietary changes or frequent doses of nutritional supplements, problems with patient compliance can make it hard to generate useful information. Numerous studies have shown that patients tend to overestimate their compliance with medications (24) and dietary advice (25), or misrepresent it in an effort to gain the approval of the physician. (26) Because of the challenges of funding and of study design and implementation, very few trials of complementary or alternative therapies have been completed, and the results of almost all of those completed are debated. (27,28)

As a recent book on clinical case reporting states: “because a case study is the first link in the chain of evidence, other steps do not necessarily have to follow for some time. A single case or case series (with all their inherent limitations) may long remain the only evidence available. If that happens, single cases or case series must provide the best evidence in their contexts.” (29)