During my tenure as Editor of Realskiers.com, one of the benefits of membership has been a long-form ski review, accompanied by test scores complied by our small volunteer army of testers. Long-form reviews are still a principal component of the Realskiers’ membership bundle of benefits, but the data will no longer be a part of the presentation for the simple reason the paucity of entries results in data that is more detrimental than beneficial.
Now for the not so simple reasons. The Realskiers Test Team has always been primarily comprised of shop personnel, from owners down to seasonal salespeople. Once upon a time, this produced hundreds of what were then paper test cards, often with 20 or more entries per model. The test cards are digital now, which creates its own set of challenges – pencils don’t suddenly go haywire for no apparent reason, but apps do – but the much greater problem is that shops are already understaffed; sending a manager and a coterie of testers off to test skis for 2-3 days is not in the cards.
There are also fewer demo events to attend, in part because suppliers are imposing early order deadlines, drastically shrinking the window between the first appearance of demo skis and the drop-dead date for submitting an order. And that’s assuming the weather cooperates, but last year at Mammoth, it was in no mood to trifle. The event lasted all of one day before being weathered-out.
With fewer testers getting fewer opportunities to test, getting any kind of feedback has become a season-long chase. The end results are wildly imbalanced, and so few in number that if you start tossing out anomalies, pretty soon you don’t have many numbers to work with. It seems apparent to this ski test maven – I’ve been at this game since the winter of 1986/87 – that the numbers can’t be trusted. Nor is there some formula by which they can be “fixed.” When numbers beg for fixing, you know their value to the end user is damaged beyond repair.
To put the present circumstances in perspective, allow me to cite my friend Bob Lefsetz – whose copious output you can read at (https://www.lefsetz.com/), a prolific social commentator – and passionate skier – who happens to be the most influential blogger in the contemporary music omniverse, on the frailties of data, to wit: “Not all data is created equal.” To which he trenchantly adds, “But even worse, all data analysis is not created equally.” Lefsetz is pontificating about today’s political punditry, but his observations pertain to any realm where numbers posture as the distillate of reality.
Despite its air of substance, data can distort reality even before it’s entered. Any tests that use a 5-point scale will always end up cramming nearly the entire field into a de facto 3-point range, a phenomenon I first observed when reviewing skis for Snow Country Magazine. When I started with a 5-point scale, the better testers were often putting an “X” on the line between enumerated boxes, to create some separation in a field that might extend to 20 models. I realized then that if I really wanted to identify the best in any category – in a market with far more brands than we have today – I needed a 10-point scale. This allowed me to then rank the results in order of finish, helping drive consumer interest in our start-up publication.
The reason I published our detailed test results then and have done likewise for Realskiers subscribers to the present day, is that people place a disproportionate trust in data, as if the numbers being published weren’t made by the same people who wrote the text. My point is that numbers aren’t inherently truthful, despite being composed of numerals instead of letters.
From my perspective, the biggest issue with data and its interpretation is that it’s an arid way to describe the ski experience. It potentially still has some value when comparing one ski to another, but whether a ski earned a 3.7 versus a 4.0 for edge grip still doesn’t tell you much about how it feels to ski on it. Even this limited attribute goes out the window when one ski result has 3 entries behind it and another has 30. There’s no way to equalize the imbalance in the number of entries, period. Not to mention that treating all results equally is based on the generous assumption that all skiers are equally qualified to submit entries in the first place.
It’s reached the point that numbers are so problematic they are as likely to distort as to describe a ski’s performance envelope. Testing skis on snow is still a valid exercise, particularly when asking a specific question about a specific feature, such as a change to a single core component. For testing to have any validity beyond this limited scope, there has to some rationale to the composition of the test team, just as there has to be an educated, common understanding of each test criterion. These requirements beg for a measure of control over the data collection process, control that I can’t exert from where I sit.
One unequivocal benefit to sharing ski test data with my Dear Readers: doing so provided a system for ordering each category’s reviews based on merit rather than the alphabet. Despite the absence of published data, I will continue to divide each genre into Power and Finesse camps and rank them according to my estimate of how they measure up against their peers.
If there’s a redeeming aspect to the demise of dependable data, it’s that its absence places the emphasis of any review squarely on the narrative, where it has always belonged. The qualities that real skiers seek in a new ski can’t be conveyed by numbers, and aren’t so easily fashioned into readable prose, either, as a romp through almost any ski website will sadly make plain.
The goal of any ski review is, or ought to be, to help the prospective ski buyer sort through a maze of options and end up on the best choice for him or her. The average price for a quality ski/binding set-up will easily exceed $1,000 this season, so it pays to do a little research. There is no better place on earth to do this research than Realskiers.com, if for no other reason than our site has one asset no other can simulate, imitate or copy: me. One of the privileges of membership is that I answer all queries personally, and I stay with you until we get your problem solved.
I look forward to hearing from you.
Related Articles
The Making of a Skier, Part X: The Mechanics & Managers Workshop Tour
When I left Salomon in the spring of 1987, my motivations could be distilled into three principal components:
• The parent company declared it was moving its Reno-based North American HQ back from whence it came. Neither I nor my family had the slightest desire to return to New England.
• I felt I was spending more energy battling factions within my own company than I was out-flanking our competition. I’d worked more or less without a break since June of 1978. My thin veneer of patience cracked.
• I wanted to write screenplays. Not that I had demonstrated any talent for creative writing or had any training in the field. I’d written reams of technical swill, brochure copy, training manuals and memos which created the illusion that I could at least write something, so why not screenplays?
Note that none of these factors involves finding a new job. At the time, I didn’t want to resume wearing the shackles of employment as they would interfere with my ludicrous screenwriting ambitions. Then the stock market went into a tailspin in October, crippling what little equity I’d managed to accrue on my minimalist salary. Oops.
The Five Stages of Ski Finish Awareness
Don’t let the title fool you. Although it sounds like it, this Revelation isn’t about the degree to which skiing is top of mind among Helsinki society. The purpose of this exposé is to shine a light on a subject about which almost all skiers are woefully uninformed, namely the condition of their skis’ bases and edges.
Just in case you don’t imagine this subject worthy of your attention, I hasten to point out that how well your skis are tuned and maintained isn’t just a factor, it is the factor that determines how well your skis perform. A properly tuned ski, regardless of brand or type, is a delight for skiers of any and all abilities while an untuned ski is such a detriment that even supreme skill cannot overcome its liabilities.
