DISCLAIMER: Ok, this is a speculative post based on idle wonderings rather than hard evidence based maths. Hopefully we'll get a page of decent replies before it descends into people shouting at each other about statistics and who's qualified to say what.
Right, so the NAF rankings are based on ELO (tactically ignoring Glicko because I don't knowhow that works). ELO works by comparing each coach's ranking and predicting the outcome. The change to each player is weighted by how unexpected the outcome was (so a rubbish coach beating a grand master would get a lot of points, a grand master beating a rubbish coach gets few points).
That's how it works in Chess but Blood Bowl has, by design, more variety than chess. We all know that Goblins vs Wood Elves is not an even match irrespective of who the coaches are.
So, should the NAF ELO formula take this into account? There's enough data to have some confidence about what win rates each possible matchup has (right?) so could we use that to modify the ELO formula?
NAF Rankings - ELO and Tiers
Moderator: TFF Mods
-
PercyTheTroll
- Veteran
- Posts: 254
- Joined: Thu Aug 12, 2010 8:11 pm
-
Purplegoo
- Legend
- Posts: 2256
- Joined: Mon Jan 14, 2008 1:13 pm
- Location: Cambridge
Re: NAF Rankings - ELO and Tiers
As a general point, yes, the formula could always be cleverly tweaked or 'improved'. But it's never going to be perfect or any more than a fun thing to look at as a general, rough approximation of performance. I think there is some value in considering quite broad changes (see: the conversation on the NAF forum about 'Masters Series' events getting an increased k-value, for instance), but I think we'd only gain a relatively low amount of additional value (compared to ongoing effort) from the sort of focused change you suggest.
Global win percentages are quite a blunt tool. A heavily tiered ruleset probably makes a mockery of a global percentage - if a coach makes out like a bandit and benefits from a very favourable multiplier to get a huge number, is that any better or worse than where we are today? Yeah, Raveen is number one with Vampires, but did you see the rulesets where he got all of those wins? Would you back date such a change, or bring it in tomorrow? How often would you review the numbers as the meta changes (we are currently becoming ever more intricate with our tiers, who is to say next year there won't be a reversion to a skill a game)? What if GW re-write a roster? All in all, I think there are some advantages to our ELO rankings being unconcerned about the relative starting strengths of each race and being quite general, both practically and in terms of accessibility for coaches. That latter point is quite important I think; while the ELO numbers are a blunt tool, we all sort of understand and appreciate them. No-one needs a maths degree to understand how it works in the terms you outline above.
I think the only practical way of producing anything close to near-accurate rankings is to have a variant that only accepts results from a single ruleset. I doubt there would be that much interest in a 'competitive' or 'true' ranking - but I await being told I'm wrong!
Global win percentages are quite a blunt tool. A heavily tiered ruleset probably makes a mockery of a global percentage - if a coach makes out like a bandit and benefits from a very favourable multiplier to get a huge number, is that any better or worse than where we are today? Yeah, Raveen is number one with Vampires, but did you see the rulesets where he got all of those wins? Would you back date such a change, or bring it in tomorrow? How often would you review the numbers as the meta changes (we are currently becoming ever more intricate with our tiers, who is to say next year there won't be a reversion to a skill a game)? What if GW re-write a roster? All in all, I think there are some advantages to our ELO rankings being unconcerned about the relative starting strengths of each race and being quite general, both practically and in terms of accessibility for coaches. That latter point is quite important I think; while the ELO numbers are a blunt tool, we all sort of understand and appreciate them. No-one needs a maths degree to understand how it works in the terms you outline above.
I think the only practical way of producing anything close to near-accurate rankings is to have a variant that only accepts results from a single ruleset. I doubt there would be that much interest in a 'competitive' or 'true' ranking - but I await being told I'm wrong!
Reason: ''