Noise: A Flaw in Human Judgment

The authors argue that the expensiveness of noise reduction strategies has been exaggerated, especially when they have been proven to improve performance and save money. However, “the belief that it is too expensive to reduce noise is not always wrong,” and people need to weigh the benefits and costs of noise reduction (384).

Stith and Cabanes argued that the noise-reducing sentencing guidelines which minimized judicial discretion would lead to greater rather than fewer errors. They campaigned against the positivist view that the world was a puzzle that can be solved, and that human solutions are required for human problems—especially where harsh life circumstances rather than innate bad character have contributed to a crime.

However, the authors argue against putting all noise-reducing strategies into the same boat and making blanket statements that if one has failed, another could not improve circumstances. For example, more complex assessments in various disciplines reduce error and eventually noise.

Noise-reducing algorithms can sometimes express bias according to gender or race. Thus, the advantage of eliminating “unwanted variability” could be compromised if it also reinforces damaging prejudices already existing in society (389). The authors recommend careful testing of algorithms to ensure that discriminatory factors are adjusted as much as possible. Still, algorithms are easier to scrutinize for biases than human judges, and there is evidence to suggest that they are better at selecting more diverse talent pools.

People generally like the sense that they are being seen and treated as individuals rather than as part of an undifferentiated mass. Noise-reduction strategies can disregard individual circumstances and thereby make blanket judgments that severely limit human potential. Moreover, they can enforce outdated norms and inhibit the changes that are already reflected in society. A noisier system where there is more judicial discretion might, for example, shorten jail sentences for drug possession and extend the ones for rape. However, the authors argue that where a noise-reduction strategy is “crude […] the best response is to try to come up with a better strategy – one attuned to a wide range of relevant variables” (396). Moreover, noise-reduction strategies can respond to changes in society: The authors write, “[I]f people use a shared scale grounded in an outside view, they can respond to changing values over time,” and if judgments are aggregated, they can better reflect emerging views (399).

Other oppositions to noise-reduction strategies include the prominent view within organizations that they damage creativity, motivation, and engagement. A noisy system engages people because they can use their own initiative in responding to problems; without noise, their jobs become almost robotic. Thus, a degree of noise should be tolerated if employees are happier and coming up with fresh ideas. Moreover, bosses should emphasize that even where there are rules, they can be challenged and rethought. Overall, people should find noise-reduction strategies that do not diminish the values they hold dear.

It is useful to distinguish between rules and standards in noise reduction strategies. While “rules are meant to eliminate discretion by those who apply them; standards are meant to grant such discretion” (406). As rules reduce the need for human judgment, noise can be reduced whenever they are in place. Standards can help diminish noise when they are precise as opposed to open-ended. Sometimes the politics in a particular organization may be so fraught that only standards can be imposed instead of rules. In this case, providing clear definitions can eliminate conflict and control equivocation. Rules need to be continually monitored to check that they are working as intended and are not producing more noise unintentionally.

Rules are strict or relaxed according to the extent to which the rule-makers trust their agents. However, rules are also useful when a large number of similar cases are being considered. This is true of dermatologists who evaluate numerous types of moles and rashes.

While the authors would not go as far as saying that noise reduction should be part of the Universal Declaration of Human Rights, it can be counted as a “system rights violation” in some cases, and all legal systems should be making an effort to reduce it. Just as organizations see “bias as a villain,” they should also consider noise that way, as it produces high costs and “terrible unfairness” (417).

This chapter summarizes the arguments made in the book as a whole in defining noise and describing the different types present. Noise, defined as “the unwanted variability of judgments” is too prevalent (419). Judgment is a type of thought that requires taking the measure of a situation and coming up with a conclusion. Often, the outcome of judgments is unverifiable, meaning that their quality can only be measured by the thought processes responsible for them. While organizations have come to see bias as an impediment, they are slower to recognize noise as equally problematic.

In their defense of noise-reduction strategies, the authors maintain that “the goal of judgment is accuracy, not individual expression” (430). They realize the unpopularity of this statement in a world where individual expression is championed. However, where judgment is concerned, individual expression is a source of noise. Instead, taking the outside view and thinking statistically about the case, rather than treating it as a purely individual phenomenon, can reduce bias. Similarly, intuition, that staple of the self-help world, is best applied at the end of a judgment process when all relevant information has been gathered.

In a noise-reduced world, noise audits would be regularly performed in all public institutions and measures put in place to control unwanted variability. Interestingly, “overt disagreements would be both more frequent and more constructively resolved” (437).

This appendix offers advice on how to conduct a noise audit in a company, including the players who must be involved. These individuals are the project team, clients, judges, and the project manager from the administrative team. Course materials include questionnaires with open questions that call for an outside view of the problem. The authors advocate calling the noise audit a “decision-making study” and branding it as an exercise in which “the organization is interested in how [decision makers] reach their conclusions” (442).

This is a checklist for an external decision observer and is intended to “serve as an inspiration” for decision observers who should tailor their own to the needs of the situation (445). The checklist, encourages the decision observer to be aware of different types of noise, such as people’s tendency to substitute a difficult question for an easier one of their own choice.

This chapter informs the reader on how to avoid the intuitive matching errors discussed in Chapter 14. It instructs them how to evaluate the quality of the information received in cases where it might be tempting to match statistics and create a story of cause and effect. It argues that the right answer lies somewhere between a person’s intuitive guess based on unreliable information and what the statistical mean would have been if a person knew nothing about the case. Most times, the correct answer is closer to the statistical mean than to the intuitive guess.