Advocacy Articles


Evidence-Based Practice: What role should research play in our decision making?

Davin Youngclarke, MA
Katie Ramos, PhD
UCSF-Fresno Family Practice Residency Program

The work of child welfare requires a myriad of decisions to be made that directly affect the lives of children and families. If you are involved in making such decisions, you have undoubtedly considered that your recommendation could be right or wrong, and recommendations you don't make may have been right or wrong. The bottom line is whether things go better or worse for the child, and you certainly hope that your recommendations play a role in encouraging the better outcome. This essay is about how we make decisions that impact children.

Decisions, Decisions... Have you ever had to make a decision, or give someone advice, that turned out to be dead wrong? Perhaps you have thought about a recent decision you've made and wondered if things are going to turn out like you want them to. We all start with the best intentions, yet at times we may feel a sense of anxiety or worry that we didn't get all the information or maybe we made the decision the wrong way. One thing that can go wrong is that someone else, or our own behavior and expectations, puts us in the position of being an expert on a particular topic or type of situation. People then begin to expect us to know what to do all the time and they expect our recommendations to be effective, all the time.

Odds are that if you're reading this essay a significant part of your formal role with children is to evaluate problems and make recommendations for what ought to be done to improve kids quality of life. We need to find some way to participate in a prudent manner and make our judgment count without being painted into the expert's corner.

Experts are fine for certain things. Consider the attorney as an expert in law. That is, she knows the proper codes and subsections and can apply them to a variety of situations. That's fine, it works. However, try to put one of these folks into the role of an expert in foster care and let's see if it holds water. In order to live up to this title he should be able to make decisions that will always result in the desired outcome. So do 100% of children in foster care go home and live happily ever after? Of course not. How about 90%, 80%? Keep dropping your expectations and you'll get closer to the real outcome of foster care intervention. But it's complex so don't blame anyone just yet.

We've got some good news for you. There is a model for dealing with this problem of over-reliance on experts. Medicine is facing the same dilemma, where the presumed experts (physicians) are expected to know everything about health, disease, treatment, etc. They make many decisions every day, decisions that often have serious consequences for their patients. It is critically important that they consistently make good decisions. In order to be consistently effective, physicians are learning to move out of the role of "expert" by systematically using clinical research to make decisions. The practice of evidence-based medicine provides a useful model for others for whom consistently good decisions are critically important.

Assumptions of Evidence Based Practice

Evidence-based medicine is based on two underlying assumptions. First, doctors can't know everything. There are limits to human knowledge, even among those who are both intelligent and well educated. Physicians cannot know everything, so it's wise to accept that up-front. Accepting human limitations is a necessary first step toward using curiosity, nagging doubts, and special concerns to pursue evidence. When a physician is comfortable with her limitations, she is able to ask specific answerable questions, and pursue those questions using strategies most likely to yield a useful answer.

Second, not all evidence is created equal. When a doctor acknowledges his limitations, and sets out to find an answer to his question, he wants to find "proof" that conclusively answers the question. This temptation is widely shared...wouldn't it be nice if there was a "Book of Answers" out there with the right answers about all of life's decisions? Well, there is often evidence to help answer questions, but that evidence must be interpreted, analyzed, and applied in the context of professional judgment. An evidence-based practitioner must be able to evaluate the quality and generalizability of research evidence. Rarely does science provide absolute proof of anything. Instead, our evidence is typically biased or limited in one way or another. Researchers must deal with patients who lie about how often they exercise, the impact of smoking in addition to the disease being studied, and subjects who are "lost" because they died or moved. All of these complications can affect the quality of the evidence.

Physicians must make decisions based on the best evidence available, and weigh the risk of the unknown against the known risk of doing something else, or nothing. Evidence-based practitioners must analyze the level of evidence, taking into account all such limitations. They also must decide if findings from a study on teenagers can be applied to a younger child, or whether research on White subjects can be applied to their Latino patients. This is the work of Evidence-Based Medicine, work that is being replicated in other disciplines as well.

Evidence Based Practice is Crossing over into Social Work and Psychology

There is also a growing movement in the academic areas of social work, and prior to this in psychology, that use this process of determining 1) what is known and 2) what is the quality of the evidence. This is especially important for the "soft" sciences like social work and psychology. In fact, psychology ratified a "scientist/practitioner" model in Boulder Colorado decades ago and has since wandered far afield from its stated purpose to produce professionals who are first and foremost scientists and after that, practitioners (i.e., therapists). In short, psychology was an area in which objective truth was to be determined through judicious use of the scientific method and then these truths would be used to provide the best mental health care free from subjective, untested beliefs.

Another way of saying this is that there is a growing movement towards the concept of "expertise" rather than the "expert." The "expert" is a person who ostensibly knows everything about a broad content area (impossible!).

Now consider the concept of expertise. Expertise is knowing specific things based on objective and systematic reality checks, i.e., confirmation via the scientific method of testing hypotheses or predictions. With proper training and knowledge of what is objectively known can we be good at some things and not others? Of course we can. Consider a hypothetical foster care intervention team consisting of a medical doctor, a licensed therapist, a social worker and a CASA. You have to assume that they take the time to take an evidence based approach; this means that they work from objective bases of evidence that certain interventions will be more effective with certain families and children. The physician can contribute expertise in the form of medical advice, the therapist could recommend psychological treatments that are consistent with known effects, the social worker can make placements and arrange support services based on effective mitigation of needs. The CASA could provide the support and supplementary service to facilitate success.

The model of expertise rather than the expert is important for a number of reasons. First, it moves us from control to cooperation. Second, it values the real contribution each individual can make and sets boundaries so that one person (e.g., the medical doctor) doesn't dominate the process. Third, it makes each member accountable to the others that their information and recommendations are based on fact rather than traditions, beliefs or other unvalidated ways of knowing.

The Effective Combination of Judgment and Research Findings

Another example of this comes from criminal justice. Parole boards have in the past functioned from the basic premise that you assemble experts who can predict who will re-offend and who will not. For a long while this is how it presumed to work. But when you apply basic statistical principles with just a few variables the panel of experts is beat hands-down. It turns out that simple statistical predictions using type of offense that led to imprisonment, number of past convictions and the number of non-criminal violations of prison rules had superior prediction when compared to the expert decision making process. This prediction is even stronger when you add the simple variable of past heroin use. In fact, other studies have shown that clinicians' predictions about future behavior get worse, that's right worse, with experience.

According to Evidence-Based Practice concepts, the issue is not which method is better; but, rather how do we combine the two methods of actuarial (statistical) prediction and clinical (expert) prediction. Let's face it, humans make the ultimate decisions and our ability to do so is not as good as we think it is, so we need all the useful tools we can gather. The specific tool of discussion here is evidence-based practice: That is knowing what is known and what is not, and determining the strength and limitations of the available evidence.

To sum it all up there are two potential major downfalls of social service and court intervention. First, that we fall powerless to the feet of the expert who has been proven wrong so many times in the past but who we desperately want to believe in. Or second, that we fail to use the best, most recent evidence to assist us in the very human process of making decisions about very needy children and families.

Take home message:

Evidence-Based Practitioners function by taking the best from science and judgment to make good and honest statements about what they know and what they don't.
Evidence-Based Practice is the conscientious and judicious use of current best evidence from clinical care research in making decisions

  • Conscientious and judicious (based on clinical judgment)
  • Current best evidence from research
  • Decisions are made considering client values and preferences
  • You can contribute to the treatment team by fostering an environment based on fact rather than theory or belief.

Don't fall victim to the domination of group processes by an "expert" who controls and devalues you. Simply say "and how is that known to be true?" If he answers that he knows based on extensive experience then he is acting like an expert. If he says that he has studied the problem in light of existing evidence understands something about it and can offer a tentative recommendation, then he is working from a basis of expertise.


 

< Go Back

Home
California Court Appointed Special Advocate Association 660-13th Street, Suite 300, Oakland, CA 94612
(510) 663-8440  (800) 214-CASA   Fax: (510) 663-8441   E-mail: staff@californiacasa.org