A little over a week ago, I was skiing.
A little over a week ago, I was recruiting Mt. Rose instructors for a women’s test that would have rolled out yesterday.
A little over a week ago, I was pulling together a dream team of Non-FIS Race ski testers, to convene the second week of April.
Then it all cratered. With a few, swift strokes, all possible venues shuttered, all potential testers pre-emptively sidelined.
I know, a First World problem if ever there was one. It is, however, a preoccupying concern for this correspondent, as evaluating the current crop of skis is my stock in trade.
Fortunately, before the ski world went dark I’d already received several hundred digital “test cards,” so I had a fair pile of data in the bank. I still had holes to fill, but nothing a few more days on snow wouldn’t cover. Oh well, once obsessive concerns shrivel to inconsequential when a pandemic comes to town.
It turns out there’s nothing like quarantine to restore the inconsequential to center stage. Having been locked up with a few hundred lines of fresh data for several days, I’ve had time to reflect on how to present this year’s results.
The problem with limited data is a single entry can skew results, and numbers are what we use to quantify results. Because numbers appear more objective than words, people tend to have more faith in them, even when those numbers have zero statistical significance.
The dirty little secret behind all ski reviews that provide scores or rankings – Realskiers.com included – is that none of them use enough data to achieve statistical significance. The real value in a ski review lies not in the numbers, but in a narrative that takes the data into account along with other knowledge about the ski and who it’s made for.
The late, great John Fry compared ski testing and rating to wine tasting. As was often the case, Fry’s analogy was spot-on. We expect wines to be judged by a small panel of trained experts, not rated by an army of amateur imbibers. So it is with skis: five or six scores from veterans of years of disciplined testing are worth more than a shoebox of test cards from average recreational skiers.
What the veterans can do that the neophytes cannot is capture the experience of the ski, the behavioral nuances that give a ski its character. They search for the ski’s performance boundaries where its limitations are exposed, as well as indulging in its purported strengths. A talented tester knows how to outline a given model’s total performance envelope, which is what you’re really paying for when you buy a new ski.
The best all-around skis are the ones that are very good at what they weren’t designed to do. Here the talented Marcus Caston tests his skis’ ability to find the bottom of the pristine snowfields of Engelberg.
The silver lining in this year’s dearth of data is that it will induce more of my Dear Readers to recognize the primacy of narrative over numbers. The test scores help to sort through a diverse cast of candidates and draw clearer distinctions among similar models, but they are never the entire story.
Because it is numeric, test data exudes an aura of authority once it has been recorded, totaled, averaged and combined in various permutations. The narrative, in contrast, seems contrived. In fact, when data is scarce it is also far easier to manipulate than the written word. The data, after all, is provided by the same humans who help fashion the narrative, and as such it’s just as subject to error.
This is why it helps to know more about the ski than what one learns from tallying tic marks on a test card. It helps to know how it’s made, what sort of skiing it was designed for and how benchmark skis in the same category perform in similar conditions.
So despite a ski season that wrapped more abruptly and bizarrely than any episode of The Twilight Zone, rest assured your trusty ski gear guru will continue to coin sparkling turns of phrase and stockpile stunning metaphors in the service of guiding my Dear Readers to a perfect ski companion.