Wine competitions, especially those that are properly run and have skilled judges, benefit consumers who seek expert advice in their quest to get great wines or good values.
And properly strategized competitions also help wineries determine whether their output meets current high standards and assist wholesalers in determining whether they have priced their latest offerings appropriately.
Judgings also can assist others in the industry who don’t trust number-driven single reviewers whose tasting strategies are completely unknown, flawed, or who may know little about a particular terroir or grape.
The term “properly strategized” means that the organizers understand the pluses and minuses of the chosen format. It may sound like a good idea, for example, to have a panel of 10 judges evaluate 100 wines, rank them by numerical score, add up the scores and divide by 10, and come up with rankings.
But there are many drawbacks to that, including logistics and that 10 great judges are hard to find. And if a competition has 1,500 wines, it would require 15 days of 100 wines daily to complete.
I have coordinated wine competitions since 1982 (to date I have organized 53 major competitions) and can tell you that the best panel size is four, which one might think would lead to votes tied 2-2. Such as two golds and two silver.
However, panels that use an odd number of judges (3, 5, etc.) can fall into the pitfall of “majority rules” and mediocre results.
The benefit of an even number is that whenever ties occur, the best way to break deadlocks is for panel members to discuss the wines and reach an amicable consensus. Professional judges work to compromise, and ties can almost always be resolved.
Uneven-numbered panels that rely on majority rules often lead to terrible results.
Some judges (occasionally younger judges) like to display their toughness by refusing to compromise. This means it’s hard to get gold medals, and thus the event must have rules to make gold medals a likely scenario.
Three-person panels pose a uniquely tricky situation when two persons have valid reasons to like a wine, but the third member sees his or her power to deny the wine any medal, for whatever reason (typically the egotistical ability to simply wield power).
At a recent wine competition in Calaveras County at which I was a judge on a three-person panel, a wine received votes of gold from two of three judges, me and another judge. The third, having heard the other two vote Gold, said, “No award.”
Had the voting stood that way, G, G, NA, because of the way the judging was organized, that wine would have gotten a silver medal. Not a gold.
It seemed the third judge simply was on a power trip to deny the other two judges the gold medal they had, independently, agreed on. (Had the third judge voted a bronze medal, the wine would have been awarded a gold, based on the event structure.)
The best three-person panel judgings I have ever witnessed (and participated in) have been 11 events I have been asked to judge in Australia. That nation runs the world’s finest competitions, by far, partly because the judges all respect the spirit of the event -- that all judges must respect the votes of others.
(In the Calaveras County event, one judge clearly was disrespectful of the other two judges and spitefully attempted to hijack the results. This happened more than the once.)
Moreover, the present method of three members per panel in this country is far too simplistic for a major, world-class competition. It assumes two (mediocre) palates, not skilled in seeing aberrant styles, are “better” than one person who has a valid argument, if not always mainstream.
I have faced situations many times where I voted a wine a gold medal, and two others voted no award, and the wine in question wasn’t considered for a bronze. It’s as if a valid argument for a gold medal is ignored by a “majority rules,” which ignores the possibility that a slightly aberrant style is distinctive enough to warrant a medal, to reward a winemaker’s courage and adventurousness.
The critique I often get from those who prefer the “majority rules” system is, “What do you do with ties?”
Tie votes (such as two silver votes and two bronzes) are results devoutly to be wished for. That’s because the four-person panel is thus required to initiate a discussion that can lead to consensus. And the two-silver, two-bronze vote should in most cases be compromised to silver. The only scenario in which a bronze is the better result is when a silver voter sees a valid argument against the wine, or where a bronze voter realizes his or her vote was weak and should be dropped.
The GGBB sort of split should not be an automatic silver medal. One side or the other should try to make a compelling case that the consumer is either benefited or to be warned.
It is also crucial to assess the number of wines each panel judges.
Assume 40 Chardonnays must be judged. Is it too much of a burden for one panel to judge that many? Usually not. How about 55? Perhaps. But one goal of the competition should be to have each category judged entirely by one panel, with that panel advised that they should look for an appropriate percentage of golds.
In major competitions, the average percentage of gold medals is 7% to 10%, so with 40 Chardonnays, there should be between 3 and 4 gold medals – at the very least. It is feasible to have six or more if the wines display various styles. (This admonition before the judging helps the judges to understand that awarding only 1 or 2 gold medals to a class of 40 is probably a poor result.)
Consequently, it pays, when assessing the results of any competition, to ask who the judges were and how many sat on each panel.
This story was originally published May 22, 2017 12:00 AM.