At the risk of occasionally re-stating the obvious and often repeating points made previously by various ski industry savants, I want to revisit the merits and demerits of ski testing generally and expertskiers’ methodology specifically. The whole point of the on-snow exercise is to pinpoint the behavioral matrix of any given ski make-up so skiers can try to match its performance to their needs and expectations. This process requires a skier as well as a ski, requiring all data to be filtered by a fallible human interpreter. I doubt there are enough qualified skiers in the US to create a sufficiently massive sample size to allow test results to rise to the level of statistical significance; in other words, it’s impossible to assemble a test team large enough to rely on the sheer weight of numbers. Achieving a meaningful result therefore depends on several qualitative factors:
- The technical and communication skills of the individual tester. Ideally, a tester is proficient to the point he/she can imitate virtually any technique and has the language skills to identify and describe behaviors.
- How skis and/or skier types are categorized. When skis are tested in predetermined groups, as they are at all magazine tests, how any one ski fares depends in part on the coherence of the bundle. If a collection is a hodgepodge going in, the results coming out will be an indecipherable hash.
- The behaviors captured in the data. There should be a mix of criteria the testers are asked to quantify, including behaviors preferred/required by lower skill skiers as well as edge-of-the-envelope behaviors that help identify skis with the highest performance range. Verbal descriptions of the ski’s personality should complement and expound upon the model’s attitude and aptitude as depicted in the data.
- The balance of the test team. In some cases, such as testing race skis or pipe and park skis, it’s desirable to have a preponderance of highly skilled and highly specialized practitioners or else the testers won’t be able to put the ski through its proper paces. In all other categories – i.e., over 90% of the market – the team should reflect the composition of the world of experts, including ex-racers and current jibbers, but also instructors, coaches, former product developers, specialty retailers and a couple of local hotshots who can turn a ski and a phrase.
- The ability to mine the data to create viable comparisons across brands, meaningful behavioral snapshots by model and correlate each ski’s behavioral matrix to specific skier types.
If there is a single argument that validates the ski test enterprise, it’s that manufacturers do exactly the same, usually with a smaller test panel, when deciding which of several prototypes to anoint as the next new model to join the line. At the end of the day, there’s no way to tell how a ski is going to perform without skiing on it. (Most manufacturers also take the opportunity to ski their competitors’ products whenever possible.) All their R&D investment isn’t worth diddley if, when the ski hits the snow, the results are indifferent.
It’s not that ski designers can’t predict what will happen when they add or subtract a feature from an existing template, but they can’t be sure, since changing any one thing in a ski can effectively change everything or result in just a minor fluctuation, like adding a hint of cinnamon to sangria. The reference to wine is not an accidental simile, for the business of testing skis is a close relative to the rituals of wine tasting. Both are very experiential exercises that nonetheless presume to produce a numeric result, i.e., test scores and attendant first place awards. Their credibility rests on their methodology and the authenticated expertise of the test panel.
Test Panel Composition
Manufacturers get by with very small test teams because their composition is super-select, usually from the R&D staff and a handful of trusted athletes they’ve used for decades. The major print magazines try to achieve what balance they can in an era of drastically reduced budgets. There’s less recruitment of great resumes and more assembly by convenience, the unfortunate consequence of having less to work with every year. There’s no shortage of skills and dedication to the task at hand – every tester is an expert skier giving his or her all to the cause – but there’s been a notable decline in the marquee quality of the major test teams over the last decade.
I should know. I’ve been on one side or the other (manufacturer or tester) of ski testing every year since 1986. When I orchestrated all alpine product testing for Snow Country Magazine in the 1990’s, I got the go-ahead to assemble an all-star cast. I imported a manufacturer’s methodology to the enterprise and, with the backing of the editor and publisher, devised several innovations that have since become standard: ranking winner skis in order of finish, showcasing the test team’s talents (which allowed readers to personalize the results), showing all scores by letter grade, along the way contributing instantly adopted terms such as “shaped skis,” to the sport’s lexicon. We used to test skis for a month, and we paid our crew; no print publication can afford such luxuries in today’s market.
I’m not saying that magazine buyer’s guides aren’t (usually) a valuable reference, only that they face certain inherent limitations. Like all ski tests, they’re dependent on the vagaries of the weather, with only a small window in which to match their bundles of skis to the snow conditions. At expertskiers, we dodge that bullet by testing at multiple venues over the span of several months. We capture data on many more models, from both mainstream suppliers and microbrands. Our crew, both male and female, is recruited from the ranks of some of America’s best specialty retailers. Most are lifelong skiers, dedicated – one might say, addicted – to catching as many hours on-hill as they can snag. Because their job is to match skis to skiers every day, they are particularly adept at visualizing who belongs on whatever ski they’re testing. They are elite skiers, but not elitists: they know there’s a ski for every skier and it’s their mission to identify it and communicate its benefits.
Winners and Rankings
Once a test director has loaded all the scores into a database, the biggest question of all must be addressed: how to parse and present the results? Organize the market by skier type or by ski category? Identify the top-scoring ski in each category or toss a blanket over all the reviewed skis and declare them all winners? Sift out the weakest models or show everything submitted by everybody? Present the reviews in alphabetical order by brand or list them in the order of finish? Show every score or no scores? Create a dizzying pile of symbols and pictograms and use them in lieu of prose to describe each ski? Present fine-grain categories with strict criteria or toss a mishmash of models into a pot, give it name like “Rippin’ Riders” and present the results as if the skis were all somehow related? It’s all been done, and more.
Here’s our solution at expertskiers. First, the completed test cards are sorted by brand, model and category. The categories are defined by where and how the skier intends to use the ski, which we tie with realistic rigor to the ski’s width at its narrowest point underfoot. All of our category terms apply equally well to the skier, the ski and the venue where they are going to play together. For example, the “All-Mountain East” skier wants terrain and condition versatility for the eastern US resorts where he or she habitually cavorts.
We ask our testers to give us a numeric score (on a 1 – 5 scale) for eleven behavioral criteria. Five of the criteria focus on technical properties that favor more proficient, fast and/or powerful skiers: Clean Carve, Accuracy, Stability at Speed, Quickness and Rebound. We bundle these five behaviors into our “Power” score. The six others are “softer” criteria that reflect the concerns of less aggressive skiers: Smooth Drift, Not Heavy, Ease at Slow Speed, Forgiving, Relaxing and Builds Confidence. We pool these numbers to create our “Finesse” score. We present the best Power skis separately from the best Finesse skis in each category as they represent two very different expectations on the part of the skier making the buy.
Here’s what we don’t do: show raw numeric scores or identify a single “best” ski in a category. We don’t show the raw score data because insignificant differences may be interpreted as important when they’re not. A 5-point scale only allows so much parsing, which means scores can be bunched, separated by only slivers of a point. Since at the very top we’re talking about differences in flavors rather than fluctuations in quality, we convert the numeric scores into letter grades like we all received in grade school. If we end up with a lot of “A” students, remember that we’re skiing preponderantly high-end, expensive skis: they ought to be good. But in every class, a few students stand out. These favorites – there is never only one – we distinguish by putting at the top of each list, but any ski reviewed in our Find the Right Ski cross-section of the market we consider worthy of our readers’ consideration. If you want to see the full range (US market) of any brand’s ski collection, please refer to our brand-specific catalog of results.
Keeping your Eye on the Prize
We endeavor to keep our focus on the ultimate purpose of the enterprise: help skiers find the best ski for them to buy this season. We’re not trying to track down and describe every ski being made in the world as this has become not only very difficult but at the end of the process, not very useful. There are more than enough extraordinary skis being presented here to make searching elsewhere unnecessary. Don’t turn the search for your next ski into a fetish; leave that to us. Believe me, you don’t want every ski day to be a test day. It’s better to fall in love and remain faithful, at least for a couple of seasons. We want to be your dating service.
Perhaps the best reason to look closely at ski tests is that they have sufficient influence on the market to create scarcity before consumers ever have the chance to try them. As I write this in the spring of 2013, the Blizzard Bonafide represents a case in point. For two years running, SKI magazine (and other oracles) declared it the best ski in its class, resulting in all Bonafides being gone before many natural snowflakes had appeared. You might think Blizzard should learn to make more skis, but all ski companies have banks looking over their shoulder, limiting their risk exposure. Overall, the ski market is hurting mightily, and the success of one model doesn’t insure huge financial gains. But having star products certainly helps a brand’s profits and helps build a reputation that can carry it through a dark period like the one being endured now.
If skis were only $199, nobody would be terribly concerned about making a buying booboo. But with a good ski and binding hovering around a grand, skiers feel more like they’re committing to an arranged marriage rather than indulging in a brief dalliance. People don’t want to live with a mistake for a decade. So people are going to search for some authentic validation that will give them the security to make a buying decision. There’s nothing more authentic than one’s own experience, which is why savvy skiers demo any model they fancy before diving into the deep end of the pool. In this case, a ski test like the one conducted by expertskiers provides the skier with a means of thinning the herd down to a practical number of candidates.
How to Demo
Since our site may well be used for just such a purpose, allow me to digress to address the subject. There is a right way to go about the process and all kinds of wrong ways. To do it right, begin by reading our reviews and background articles to get a feel for the market and where you fit in it. Narrow your choices down to no more than 5 skis. This is, in fact, too many, but we’ll narrow the field in the next step, which is finding a shop that carries the models on your candidate list. You probably won’t find all 5 under one roof, so there’s your next cut. Ask the shop what their demo program is and when you’ll be able to try them. Don’t be surprised if it isn’t until January as no sane shop sends their demo fleet out when coverage is light.
So let’s assume it’s January and you’ve found a benevolent shop that has 3 skis on your 5-ski hit list and they apply the $40/day demo charge to purchase for up to 3 demos. Ideally, you’ll ski the 3 contenders back-to-back in the same, appropriate snow conditions, or as close to this ideal as you can get. Then act: pick your favorite, plunk down your credit card and settle into your new relationship. Do not find 3 more skis to try, then another 3, and so on, hypnotized by a sort of demo narcosis. Remember that ski testing is not the same thing as free-skiing, and you probably haven’t had a lot of training at the former, so adding to your testing duties (and costs) may not improve the result. There is also the likelihood that one or more of your demo skis will be in sub-optimal tune, which makes achieving a coherent result virtually impossible.
The issue of tuning will most likely decide the outcome of your on-hill evaluations. How well a ski is tuned has an inescapable effect on its performance, whether that ski is being demoed by you, Dear Reader, or tested by one of our crew. A bad tune dooms any hope for a favorable impression, while a superb tune can make a dog at least respond to a couple of commands. Remember this when you finally adopt a ski and take it home: it needs constant love and reinforcement in the form of wax and edge polishing to achieve at its highest level. Don’t hold your skis – and yourself – back from achievement by shackling your babies with an indifferent base tune. Be nice to your skis and they will continue to be nice to you.
– Jackson Hogen
