NTBB: Stats

Got some ideas for rules? Maybe a skill change or something completely different!!! Tell us here.

Moderator: TFF Mods

Post Reply
dode74
Ex-Cyanide/Focus toadie
Posts: 2565
Joined: Fri Jul 24, 2009 4:55 pm
Location: Near Reading, UK

Re: NTBB: Stats

Post by dode74 »

Don't conflate the arguments, Koadah, and don't mix up the issues. You are, as ever, simply trying to stir the pot.
where are all the statistics and analysis that the BBRC used to decide that their job was done?
You would have to ask them. Thing is, they did have official backing to make the changes they made. They didn't claim anything other than aiming for the tiers they were aiming for, and whether they achieved it or not is a matter of debate.
Where are the stats & analysis that you used for Khorne Demons?
I would have truly loved to have been able to run rigorous playtesting with Khorne before they were released, but being limited in that playtesting was one of the things we had to deal with. Martin does not have that limitation.
That said, what stats I do have seems to put Khorne in the tier 1.5 bracket so far.
Unless you are going to prove that they are already within Plasmoid's tiers then yes, I'm for him to call it narrow tiers.
Who are? And I'm questioning it based on the data he provided, the stats he used to justify the changes. Those stats don't show the teams he said were outside the tiers to be outside the tiers except on a very basic "this is the mean and this is what matters" (which we know not to be true) basis.
Darkson wrote:You already killed one discussion with your demands that people quantify their feelings with cold hard facts.
You may think that base assumptions aren't worth challenging; I do. It's the base assumptions which most often lead to disagreement.
I'll be sitting in the corner playing a game with people who "feel" something
I'm sure you will ;)

Reason: ''
koadah
Emerging Star
Emerging Star
Posts: 335
Joined: Fri Mar 25, 2005 5:26 pm
Location: London, UK

Re: NTBB: Stats

Post by koadah »

dode74 wrote:Don't conflate the arguments, Koadah, and don't mix up the issues. You are, as ever, simply trying to stir the pot.
Yes stirring pot. ;)

All I said was that Fumbbl has new options added. Very useful IMO if anyone wants to do some testing of the CRP+ part.

VoodooMike then comes in with some unnecessary horse poo. ;)

I'm happy to say that I don't think Plasmoid has convinced me that all his changes are correct or necessary.

I still think all this whining about the name 'narrow tier' is crap. :)

If you don't like the rules don't use them or just take the parts that you like.

Reason: ''
dode74
Ex-Cyanide/Focus toadie
Posts: 2565
Joined: Fri Jul 24, 2009 4:55 pm
Location: Near Reading, UK

Re: NTBB: Stats

Post by dode74 »

I agree with most of your last post there, Koadah. All I'm saying is that to call it "Narrow Tier" and to claim that it "pushes the T1 races into the 45-55% bracket" heavily implies a statistical basis for the changes (i.e. that some of the race are outside that bracket) which currently doesn't exist - it's misleading. Either do the maths or remove the implication. I believe plasmoid is doing the maths.

Reason: ''
koadah
Emerging Star
Emerging Star
Posts: 335
Joined: Fri Mar 25, 2005 5:26 pm
Location: London, UK

Re: NTBB: Stats

Post by koadah »

dode74 wrote:I agree with most of your last post there, Koadah. All I'm saying is that to call it "Narrow Tier" and to claim that it "pushes the T1 races into the 45-55% bracket" heavily implies a statistical basis for the changes (i.e. that some of the race are outside that bracket) which currently doesn't exist - it's misleading. Either do the maths or remove the implication. I believe plasmoid is doing the maths.
It might imply that to a handful of stats nuts but I suspect most of the average punters think he'll just eyeball some results data pretty much as we would. ;)

As the only real practical use for all this is applying it to our leagues it's all going to come down to our gut feel in the end anyway.

For me 'Narrow Tier' should stay. It declares the intent. Whether it achieves it or not. ;)
"pushes the T1 races into the 45-55% bracket". Hmm, well those zons look pretty good to me. The BBRC were concerned with full life cycle. Plasmoid isn't. So is there any evidence to suggest that the do fit within the brackets?
I think I'd allow that too.

Plasmoid may be doing some maths but I doubt if it will convince anyone ether way.

Gut feel, witchcraft & hand waving FTW. ;)

Reason: ''
dode74
Ex-Cyanide/Focus toadie
Posts: 2565
Joined: Fri Jul 24, 2009 4:55 pm
Location: Near Reading, UK

Re: NTBB: Stats

Post by dode74 »

If you're going to go with gut feel, witchcraft and handwaving then don't try to claim it's justified statistically. Just because most people don't understand it doesn't mean that perpetuating the misunderstanding is justified.

Reason: ''
koadah
Emerging Star
Emerging Star
Posts: 335
Joined: Fri Mar 25, 2005 5:26 pm
Location: London, UK

Re: NTBB: Stats

Post by koadah »

dode74 wrote:If you're going to go with gut feel, witchcraft and handwaving then don't try to claim it's justified statistically. Just because most people don't understand it doesn't mean that perpetuating the misunderstanding is justified.

Where does it say that this is justified statistically?

Reason: ''
dode74
Ex-Cyanide/Focus toadie
Posts: 2565
Joined: Fri Jul 24, 2009 4:55 pm
Location: Near Reading, UK

Re: NTBB: Stats

Post by dode74 »

The claim of tier 0 teams.

Reason: ''
koadah
Emerging Star
Emerging Star
Posts: 335
Joined: Fri Mar 25, 2005 5:26 pm
Location: London, UK

Re: NTBB: Stats

Post by koadah »

dode74 wrote:The claim of tier 0 teams.
Link?

Reason: ''
dode74
Ex-Cyanide/Focus toadie
Posts: 2565
Joined: Fri Jul 24, 2009 4:55 pm
Location: Near Reading, UK

Re: NTBB: Stats

Post by dode74 »

Plasmoid's sig.

Reason: ''
koadah
Emerging Star
Emerging Star
Posts: 335
Joined: Fri Mar 25, 2005 5:26 pm
Location: London, UK

Re: NTBB: Stats

Post by koadah »

I searched that page for "stat".

It only found "statline" and "status". Could not find "stats" or "statistics" at all.

Where does he "claim it's justified statistically"?

Reason: ''
dode74
Ex-Cyanide/Focus toadie
Posts: 2565
Joined: Fri Jul 24, 2009 4:55 pm
Location: Near Reading, UK

Re: NTBB: Stats

Post by dode74 »

Did you read the bit about tier 0 teams?

Reason: ''
dode74
Ex-Cyanide/Focus toadie
Posts: 2565
Joined: Fri Jul 24, 2009 4:55 pm
Location: Near Reading, UK

Re: NTBB: Stats

Post by dode74 »

Now that I'm back at a proper keyboard rather than the tablet...

You're being deliberately obtuse, koadah, a form of trolling in which you revel and excel (probably due to practice). However, to humour you:
1. http://www.plasmoids.dk/bbowl/NTBB.htm talks of Tier 0 teams:
The BBRC seem to have managed to get all the tier 1 teams into the 55-45% win zone that they wanted. But a handful of teams start out stronger than this, then fall down into the tier 1 zone in prolonged league play. In tournament play and short league play these teams are at a notable advantage - so I've introduced some minor changes to lessen their short term power without weakening their long term performance.
It lists those teams as: Dwarf, Wood Elf, Amazon, Undead and Orc.
2. When asked what stats were used to assess this statement, these were presented (second table, combined stats), but with the caveat that the Necro LRB6 data was ignored due to the fact that Golem prices changed by 10k, and the Necro(Old) stats from this table were used. The actual numbers he is using are referenced in this post.

This is where the issue is. Without teaching you to suck eggs, looking at those leagues alone does tell how well races have done in those leagues at that time, but that is all the bare win%s tell you. If you want to assume that those are representative of early-game league play in general then you have to take them as a sample of a larger (effectively infinite) population, and you will therefore have a range of values and a confidence interval (as VM explained so well earlier), and that will depend on the sample size in each case. You also need to make a fairly large number of assumptions about the sample itself, since to be considered valid the sample should be randomly taken, which these leagues are not: there are selection biases involved as well as biases towards coach skill, league structure and league composition. Glossing over all that, and assuming that the sample is considered to be a random (or at least "good enough") sample of the population (which, if only the dice were the difference between each matchup, they would be), we can then look at the size of the margin of error to 95CI. When we do that using very basic techniques we see that the range of values for all the teams does reach into into the 45-55% region. This means that the statement "a handful of teams start out stronger than this" from the quote above isn't justified at all by the stats that were used.
Even ignoring that, and let's say that we just look at the win%s pulled up by those leagues and ignore the CI ranges, neither Amazons nor Orcs are actually above the tier. Two of the 5 teams in the Tier 0 section whose changes are justified by the fact that they "start out stronger than this" can be seen not to in the very stats used to justify the changes.

Now I'm not against Martin's house rules at all: I'm trying them out on PBeM and enjoying it. But to claim that they are justified in the manner stated is just incorrect. I've been helping Martin to try to shore up the stats somewhat, but the issue remains for the moment: the stats do not say what is being claimed.

Reason: ''
koadah
Emerging Star
Emerging Star
Posts: 335
Joined: Fri Mar 25, 2005 5:26 pm
Location: London, UK

Re: NTBB: Stats

Post by koadah »

You know what Dode. I can't be bothered. Have it your own way.

Reason: ''
User avatar
Shteve0
Legend
Legend
Posts: 2479
Joined: Thu May 07, 2009 10:15 am
Location: Wellington, New Zealand

Re: NTBB: Stats

Post by Shteve0 »

dode74 wrote:Now I'm not against Martin's house rules at all: I'm trying them out on PBeM and enjoying it. But to claim that they are justified in the manner stated is just incorrect. I've been helping Martin to try to shore up the stats somewhat, but the issue remains for the moment: the stats do not say what is being claimed.
This. Plasmoid deserves a great deal of credit for his work publicising (and crediting) the CRP+ rules, as well as compiling data for the BBRC way back when. Some of the NTBB rules ideas are cute. Unfortunately, I believe the rules pack carries an implied legitimacy that its method of derivation simply doesn't support. If plasmoid can get to the bottom of this, then absolutely great.

Good decision making processes are underpinned by the availability and recognition of relevant information. In business, in life, and no doubt in game design, you can stumble through on guesswork and intuition; but you radically improve your chances of success if you learn from the past and actively seek out evidence to influence your decision process.

Essentially, the NTBB ought to be (and is no doubt quite capable of) attracting people on the basis of its value as a ruleset, not because they've misunderstood its source, intention or legitimacy.

Reason: ''
League and tournament hosting, blogging and individual forums - all totally free. For the most immersive tabletop sports community experience around, check out theendzone.co
User avatar
Pluisje
Rookie
Rookie
Posts: 24
Joined: Fri Oct 09, 2009 2:08 pm

Re: NTBB: Stats

Post by Pluisje »

Ok, I have some numbers for Amazons only. I took the 250.000+ games in the file kindly supplied by Koadah and struck all following games:
- games not involving Amazons;
- Amazons vs Amazons games (making the W/L/D spread bigger)
- games with teams other than the 24 base races;
- non League, Box or Ranked games.
After that I have 15,608 games left. 2,506 League, 8,344 Box and 4,758 Ranked. I couldn't get the statistics programs to run (probably lack of knowledge) so I used Excel for Average and Standard deviation calculation.

First a shocker on the League results. The straight average of Amazons in the League games is 0,519354. :o In no way significantly above 55%.

For Box and Ranked I ordered the games on date played. Then I took the average of 400 games in 400 game steps. For Box this gave me 21 values of win%, for Ranked 12.
Then I took the average of 200 games in 200 game steps. For Box this gave me 39 values of win%, for Ranked 24.
The results:

Code: Select all

 
	       400-400    		       200-200		   
	       Box	    Ranked	     Box	    Ranked	   
Sd        0,04225	0,04354	    0,04753	0,02756	   
Av	     0,59073	0,59942	    0,59140	0,57938	   
Av-2*Sd	0,50624	0,50436	    0,50433	0,52426
So it looks that Amazons don't come above the 55% in Box and Ranked, statistically speaking. Now show me the many faults in this calculation. :)

edit: corrected Av-Sd to Av-2*Sd

Reason: ''
Post Reply