INTERPRETING DATA FOR TREATMENT OUTCOMES: CLINICIANS AS CRITICAL CONSUMERS

Herbst appliance

BY WILLIAM R. PROFFIT

As orthodontics moves toward data-based rather than opinion-based treatment, clinicians may increasingly find themselves evaluating treatment outcomes in statistical terms. There are two key things that a clinician needs to know when treatment outcomes with alternative treatment approaches are presented —and often neither is presented, to the detriment of clinicians who are trying to interpret what the results mean in terms of appropriate patient care.

Identifying the Level of Clinical Significance
The first key item is a whether a statistically significant difference in treatment outcomes is clinically significant. Only if the difference is clinically significant should you consider a change in clinical treatment. An excellent illustration of this point comes in the evaluation of growth modification, especially with appliances that aim to increase increase jaw growth. It now is widely accepted that functional appliances can produce an acceleration of mandibular growth. Does that mean the patient ultimately will have a larger mandible?

There is about a 50-50 split between the many studies of this issue that say yes or no. It helps a lot to realize that those in both camps report a possible increase of 1-2 mm in mandibular length over what it would have been without treatment. Is that statistically significant? Perhaps. Is it clinically significant? Almost surely not—if you want to correct a skeletal Class II relationship, your method had better include a decrease in maxillary growth and some compensatory tooth movement if you are only able to increase projection of the mandible by an average of 1-2mm. Improved mechanical devices are not going to change that. In the future, gene therapy or some other biologic modulation might. In the meantime, clinicians must be sure to understand the clinical significance of a mean increase of mandibular growth of 1-2mm, not just be taken by the enthusiasm of a statistically significant p-value.

---

Another prominent illustration of the importance of distinguishing statistical from clinical significance can be seen in the recent report in the AJODO that early (preadolescent) Class II treatment reduces the chance of injury to protruding maxillary incisors(1). That finding was reported in one of the first published clinical trials of preadolescent vs. adolescent Class II treatment(2), and a follow-up paper from the same study pointed out that the typical injury to incisors was only chipping of the incisal enamel, with obvious fracture of a crown rarely observed(3). So, the effect was statistically significant but probably not clinically significant. The recent paper, amazingly, did not consider the magnitude of injury in reporting the results—and is misleading because it didn’t.  In this example, the clinician who looks to the statistical outcome alone will again overestimate the clinical importance of the result.

---

Cevidanes et al. Angle Orthodontist 2010

Understanding the Variability of the Clinical Response
The second key thing clinicians need to know when evaluating statistical outcomes is important because we must be able to rationally apply statistical findings to individual patients in the clinic. Historically, the results of human subjects studies in orthodontics were almost always reported in the mean / standard deviation format. Treatment outcomes, however, usually are not normally distributed — a few patients have most of the changes— and now, non-parametric statistics not based on the normal distribution often are presented, with findings in the median / interquartile format. The size of the standard deviation, or better the interquartile distribution, tells you something about the variability in responses within a group of patients. Even with nonparametric statistics, there is a strong tendency to focus on the median and to think about the responses as being normally distributed – even when you know they were not. Is the median change is what my patient will get? The greater the variability within the group that were studied, the less likely that is to happen. It’s always true that some patients respond to any treatment better than others. Understanding the variability in patient response to a treatment is at the heart of the decision to adopt a new clinical procedure, and it is also the critical component in obtaining informed consent.

To understand new data from a clinically useful perspective, the nature of the response must first be defined. Then, the individual responses—not the group response—must be examined to put the patient in the proper sub-group so that the percentage chance of favorable clinical changes can be determined. Let’s look at another growth modification example to clarify this important point.
                 
Does Class III treatment with bone anchors and Class III elastics during adolescence produce forward movement of the maxilla? On the average, the answer is yes. The mean change in the position of the maxilla is a little over 4 mm, twice the mean amount of change with facemask treatment prior to adolescence(4). But it’s more important to know that with a patient of northern European descent, like those who have been studied, you can expect 80% to have forward movement of the maxilla and one third of that group to have an increased prominence of the midface as well. 20%, however, do not have a positive maxillary response at all(5).
                 
Knowing that, would you suggest this treatment method to a maxillary-deficient adolescent? In fact, this is the most effective growth modification method that orthodontists have ever seen. What should you tell the patient and parents about this during informed consent? They need to hear about the success rate: that there’s an 80% chance of a good response and a 33% chance of an excellent response—but they also need to understand that there’s a 20% chance of no forward movement of the maxilla. Will this method decrease the chance that jaw surgery ultimately will be needed? It will take more long-term data to be sure about that, but it seems likely that it will.

The Bottom Line
We are in an era when orthodontists need to be critical consumers who question the information provided about advances in clinical treatment. That goes for appliances and other hardware; it also goes for treatment concepts.
                 
If the advocate of a new treatment approach cannot provide good answers when asked about 1) clinical versus statistical significance or 2) the variability of treatment success across different patients, skepticism is in order. If he or she can respond well, accepting the new information and acting on it should be the clinician’s response.             

References

1. Thiruvenkatachari B, Harrison J, Worthington H, O’Brien K. Early orthodontic treatment for Class II malocclusion reduces the changes of incisal trauma: results of a Cochrane systematic review. Am J Orthod Dentofac Orthop 2015; 148:47-59.
2. Tulloch JFC, Phillips C, Proffit WR.  The benefit of early treatment: progress report of a two-phase randomized clinical trial.  Am J Orthod Dentofac Orthop 113:62-72, 1998.
3. Koroluk LD, Tulloch JFC, Phillips C. Incisor trauma and early treatment for Class II division 1 malocclusion. Am J Orthod Dentofac Orthop 2003; 123:117-125.
4. Cevidanes L, Bacetti T, Franchi L, McNamara JA Jr, DeClerck H. Comparison of two protocols for maxillary protraction: bone anchor versus face mask with rapid maxillary expansion. Angle Orthod 2010; 799-806.
5. DeClerck HJ, Proffit WR. Growth modification of the face: a current perspective with emphasis on Class III treatment. Am J Orthod Dentofac Orthop 2015; 148:37-46.      

---