MLB

Sabermetrics can and should be questioned, but do so correctly

When you are in this business (I mean, technically I am a member of the media…) you leave yourself open to a lot of criticism, debate and questioning. Some of it is positive and constructive; if our notions aren’t challenged, we have no reason to rethink them, and can easy go on assuming things to be true that really aren’t, or at the very least could be questioned and improved upon. The rest of it, though, can be described as blind, ignorant hate that serves to purpose other than to stir the pot, or try to reaffirm what one already believes.

All writers go through both sides some degree, with the big-name, mainstream guys like Jon Heyman and Ken Rosenthal constantly bombarded with hate on Twitter from fans wanting them to release their sources, or telling them they are wrong about their favorite team or player. But they will also receive plenty of criticism from the analytical side (some of it admittedly more hostile than necessary) for their continued use and defense of pitcher wins and RBI-like statistics that we should know by now aren’t all that great when evaluating a player. It seems, at times, that they are where they are more because of their writing background and ability than they are for their knowledge of their game, yet we still rely on them as experts and analysts.

On the other end of the spectrum, you have more statistically-inclined writers of all levels of popularity — from Jeff Sullivan to myself — receiving criticism that is both alike and different. It is alike in that some is good and some is bad; it is different in every other way. The good kind here, the kind that I love and invite and seek out, is people who question my methodology or the conclusions drawn from them by using methodology of their own. They make us better as writers, offering perspectives we didn’t consider, and that could very well be better than ours. We like this stuff, on account of entire MO being based in data and which requires dissent in order to improve.

But unfortunately, we (or at least I) seem to receive much more deconstructive criticism than anything else. People who choose not to delve into the world I have, and will outright deny anything that has to do with sabermetrics. Most of the time, these people have not done the research necessary to adequately question the metrics or processes used to create them. They just don’t like anything that disagrees with what they have always “known” and assumed about baseball, and if it doesn’t agree with them, it must not be true. And therein lies the disconnect.

There is often a sentiment that floats around describing stat guys as conceited or cocky, thinking they know more than anyone else because they use spreadsheets. But isn’t the other side* guilty of the exact same thing? They feel so strongly in their preconceived notions that anything that suggests they are wrong is automatically cast aside. There is no effort to examine the other side and come up with the most accurate conclusion possible.

I think this leads to us looking arrogant because we can show them the information that proves them wrong. We did the research and learned about these truths. We understand the traditional numbers, but we also know why they aren’t as important as some others are. If you have not bothered to study the methodology behind WAR, or wRC+, or defensive metrics, how can you posit that they are wrong? I know why I think RBIs are useless in evaluating an individual: they are completely dependent on his teammates getting on base in front of him. They don’t always show us who was “more clutch” or better at producing runs. They often tell us who had more opportunities with runners in scoring position.

If someone can explain, using data and evidence, why WAR is massively flawed, we/I will be more than happy to listen and reconsider. Take Jeff Passan’s critique of WAR earlier this year. It caught a lot of criticism, as expected, but it also created a legitimate discussion. I wrote up my thoughts on it. Dave Cameron and Passan discussed the issue in a podcast at Fangraphs. Hell, Cameron posted a piece questioning a major part of WARs makeup himself, and polled the readers on what they suggested Fangraphs changes (if anything). He made some legitimate points about where the metric falls short, even if he said some things I disagreed with. His intent was to improve a statistic he sees massive potential in, not to condemn the use of it because he disagrees with or doesn’t understand it. If someone enters into a conversation about statistical analysis without an iota of understanding, they can’t expect to get to far. They can’t offer any real criticisms, and moreover, often don’t want to.

His intent was to improve a statistic he sees massive potential in, not to condemn the use of it because he disagrees with or doesn’t understand it.

All of this isn’t to say anyone who uses traditional numbers, or chooses not to use advanced ones, is wrong, or that they automatically fit into the group I rallied against here. Plenty of people use them in addition to sabermetrics, while others use them exclusively but will not then turn around and purport to “know” that WAR is wrong or that sabermetricians never played or watched the game. They can coexist in some cases, or they can be in two separate worlds. That’s fine. I can’t say I agree with the choice, or that I think you have as much information as you would if you chose to study up, but it’s fine.

But if you choose to debate the methodology without having looked into it, know that things will get ugly fast, and ultimately get nowhere because as I said in my piece on Passan/WAR, “In order for actual progress to be made, everyone involved in the process must fully grasp the metric.” These discussions should be had with the intent to improve, or constructively criticize, not to insult the metrics and those who use them simply because you don’t agree.

Everything should be questioned. Without that, we remain stagnant in whatever it is we are dealing with. Dissent is how we get better, and expand our knowledge. But that dissent has to be intelligent and informed in nature, or it just becomes a pointless back-and-forth. The fact is, these numbers are a part of the game now. Every team utilizes them to some degree, and it seems those who do so to a great extent are able to find more consistent success (like John Mozoliak pictured above).

*I am not implying that everyone who isn’t a saber-minded guy does this, so please do not take this as a personal, generalization of an attack. I just feel this was in important post to write, and can be linked out to give my opinions on these kind of debates in the future to avoid unnecessary conflict.