In my Volleyball player efficiency ratings post I referred to a short paper proposing a way to evaluate setting. That paper is A Reconceptualisation of Traditional Volleyball Statistics to Provide a Coaching Tool for Setting by Alexis Lebedew. The paper’s basic idea is to combine a pass/dig rating (like the kind mentioned here) with a 0-3 rating for the outcome of a spike to get a set rating (0 for error, 3 for kill). The pass rating is effectively inverted. Good results off poor passes are rated higher than ones off good passes. This accounts for set decision and/or execution.
Conceptually, I like the idea proposed by the paper. There are a couple of issues with this particular ranking process, though.
No real new information
There was an evaluation of set rating system to see if it related to the probability of winning a set. It does, but that’s no real surprise since outcomes heavily influence the ratings. Specifically, the highest ratings coming on kills. We already know that getting lots of kills generally results in more winning. We also know that both hitting efficiency and pass quality relate to scoring points and winning sets. As a result, we don’t really have any new information from this rating in that context.
Problematic comparisons
Hitter skill is a major factor in attack outcomes. Good hitters can make up for poor sets. Poor hitters, though, struggle even with great sets. Further, the quality of the block and defense the hitters face is a meaningful factor in attack outcomes. As such, it is problematic to compare setting across teams. A great setter on a poor team simply won’t rate as well as a setter on a good team.
So what to do
If the idea of rating sets is to be able to compare setters directly then I fully support factoring in pass quality. It is the primary control variable determining what can be done with any given setting opportunity. Including it allows us to look at setting on an even playing field.
In order to really assess the quality of sets, though, we need to also be controlling for the outcomes in a similar way we do for passing. If a passer puts up a perfect pass, we don’t punish them because the setter mishandled the ball, nor do we bump up their pass rating because the setter was able to put up a good set off a poor pass. I think we end up circling back to the idea of rating sets similarly to how we rate passes. The problem, though, is we really should be factoring in play calling and decision making in the process, which is no easy thing.
All that said, though, I do think the type of set rating proposed in the paper could be useful in comparing setters on the same team or in a situation where teams and players are quite similar in level of play. In that case, hitting quality and the block/defense of the opposition would presumably be the same for each, allowing for a level assessment.
6 Steps to Better Practices - Free Guide
Subscribe to my weekly newsletter today and get this free guide to making your practices the best, along with loads more coaching tips and information.
6 Responses
Really, play calling and decision making are the most important qualities to rate in a set, as you suggest. I would hate to factor in the result of the attack, as a setter who wishes to have a higher statistical rating will tend to favor the better hitters more than they should, thereby decreasing the decision making component.
Could you factor in the number of blockers vs the attacker? At its most simple, a 0 for a triple block, and a 3 for no block? The assumption is that the set itself is quite hittable.
Hi Chris – You could make the case that setting the better hitters is kind of the primary point of set decision-making, especially at lower levels of play, but I get what you mean. You don’t want the ball forced to them when a higher percentage option is available.
In terms of including the number of blockers, the pass/dig will have a lot to do with that. A bad pass increases the chances of a triple, while you’ll almost never seen one when the pass is good since the block won’t be able to commit. Plus, the ability to score through/over/around a block is heavily influenced by hitter skill, which brings in the whole concern I have about basing a rating on the outcome.
The Quarterback Rating in football keeps popping into my head as representing something akin to what we’re after here. But of course the quarterback doesn’t have to deal (much) with variability in the way the ball comes to them and is working from a static start. Plus, the number of potential outcomes to a given play is much higher.
Hi John – its nice to know that someone read my paper!
The essence of the rating linking the pass and hit is that the primary outcome for a setter is to set a ball to a hitter who kills it. (As I mentioned in the paper, this is based on communications I had with Olympic volleyball medal winning coaches from different countries.) The primary outcome is NOT to set against a single block or a double block. As we know, a bad set to your best hitter can often be the best option!
The rationale, which I understand not everyone agrees with, is that ALL volleyball stats are already subjective. They are just subjective in relation to the opponent’s skill, rather than your own team’s. We don’t judge a ‘perfect pass’ lower because the skill of the server was low. We don’t judge a kill lower because the skill of the defender was low. We solely judge the outcome. For a hit the desired outcome is a kill, for a set the desired outcome is not a ‘perfect set’ it is one step removed from this, it is for the next action to be successful.
The last important point I tried to make in the paper is that, by using this tool (or something like it) you can now provide feedback to a setter with data you have probably already collected. And, even an admittedly imperfect tool like this, will provide the setter with useful feedback that no setters ever get.
Hi Alexis – Once I get done earning my decidedly non-volleyball related PhD, I fully intend on developing my own academic-type of volleyball paper. On what, I have no idea! 🙂
Your last point concurs with my own final thought. A system like the one you’ve outlined would indeed be good for giving a setter a useful reference point for their own performance because essentially all other factors are held constant (more or less).
I do very much agree with your rationale that subjectivity is involved throughout volleyball statistics. It’s funny you bring up judging passing based on server skill as that’s something I’ve long thought would be a good control – if you really want to do things properly from a statistical perspective that is. It’s the one part of the game where you can say the second contact execution is based on a fully player-controlled first contact, removing the random elements involved in other contacts.
The argument I would make in regards to the desired outcome of a set being a kill (or at least a non-error swing, depending on circumstances), is that it can skip over the actual set quality because of the influence of the skill level of the hitter. As you suggest, a bad set to a great hitter may be the right option, but then effectively all we’re rating the setter on is decision-making, not actual set execution. Of course this might be exactly what we’re trying to evaluate, which would be easy to see for higher level teams where set execution is less a concern. At the lower development levels, though, variability in execution is something coaches are very much concerned with.
In any case, the set scoring system you’ve outlined is like any other measurement tool. It can be useful as long as the coach understands the strengths and weaknesses and applies it in the proper context.
This post was from a while ago but somehow it popped up on my screen. I reread it because since writing it and now you have gone from a random person I’ve never heard of who writes a volleyball coaching blog, so someone I’ve met and had long conversations with who has a project with my brother!
While rereading it I had I thought of another story. I’ve gone through the paper with a number of beach volleyball and indoor volleyball coaches, with a number of Olympic medals between them. One of the indoor coaches said that it was a really interesting idea but is probably more suited to beach volleyball than indoor. One of the beach coaches said that it was a really interesting idea but is probably more suited to indoor than beach volleyball.
In the end it reminded me of a standard line I have used in relation to performance analysis for more than two decades now: don’t look to data analysis to give you the right answers, look to it to give you the right questions.
That’s interesting on the beach/indoor thing. My initial thought is that on the beach you have fewer variables involved (e.g. just one hitter), making comparisons cleaner.