Quidditch GM: Winning Probabilities

Hyper-Simple Model

Let's start with a very simple model: The home team has possession, and the score is tied.

Consider W_H(poss, d) to be the probability the Home team will win if the poss team has the ball, and if they have a score differential of +d.

We'll assume later based on mini-maxing, that the best decision was made at each point, but this is a crude decision tree, where the team with possession can decide to shoot a 2 or a 3 at each possession, there are no turnovers, and possession goes back and forth every time.

Let's further assume that Home and Away have the same odds for taking a 2, or a 3. G, for goal, is the chance of making a 2, S, for snitch,  is the chance of making a 3. For now, exclude the idea of free throws.

Generally (if we assume ties when the snitch is caught are 0.5 wins in value, and toggle(poss) is the function from home to away and away to home):

W_H(poss, d) = [3] (S (sgn(d + 150) + 1) / 2 + (1 - S) W_H(toggle(poss), d)
                           [2] (G W_H(toggle(poss), d + 10) + (1-G) W_H(toggle(poss), d)

  

The (sgn(d+150) + 1) / 2 is just a part that returns 1 whenever d + 150 > 0, 1/2 when d + 150 = 0, and 0 when d + 150 < 0, corresponding to winning, tieing, or losing).

The sign function is awkward for doing math with, but we can approximate that expression  with (tanh(k (d+150)) + 1) / 2

Choosing K:

K = 1 is probably too sharp. It goes from 2.06E-9 at -160, to 0.5 at -150 to 1 at -140.
K = 0.1 might be reasonable. 0.12 to 0.5 to 0.88

I choose K = 0.05 because it goes 0.11 at -170, 0.27 at -160, 0.5 at -150, 0.73 at -140, 0.88 at -130, bnut still converges nicely.

Call A_H this approximate version:

A_H(poss, d) = [3] (S atanh(0.05 (d + 150)) +1) / 2 + (1 - S) W_H(toggle(poss), d)
                          [2] (G W_H(toggle(poss), d + 10) + (1 - G) W_H(toggle(poss), d)

For sub-problem 1: Let's assume Home always shoots 3s, and away always shoots 2. Then we don't even need to mess with this formula, because of a simple fact: Since Home always shoots 3s, they will never get their scoring differential closer. So once Away takes it to a 150 point lead, they can only tie, and once Away takes more than a 150 point lead they win.

So what is the probability that Away makes N goals before Home makes a snitch?

This is getting to a simpler combinatorics problem:

Start with N = 1, Away has to make 1 goal before Home gets the snitch, but Home gets to go first

Paway(1) = (1-S) G

Then consider arbitrary N

Paway(N) = (1-S) G Paway(N-1) + (1-S) (1-G) Paway(N)

Collecting terms:

(1 - (1-S) (1-G)) Paway(N) = (1-S) G Paway(N-1)

This is a simple geometric recurrence.

(1-(1-S-G+SG)) Paway(N) = (1-S) G Paway(N - 1)
(S+G-SG) Paway(N) = (1-S) G Paway(N-1)

Paway(N) = (1-S) G / (S+G -SG) Paway(N-1)

So Paway(1) = (1-S) G
Paway(2) = (1-S)^2 G^2 / (S+G-SG)

Paway(15) = (1-S)^15 G^15 / (S+G-SG)^14

Probability of the away team winning or tying is thus this value.

Let's assume S = 0.01

P15 = 0.99^15 G^15 / (0.01 + G - 0.01 G) ^ 14

Let's look for G where
P15 = 0.5

There are 20 pages of players with 3P % > 1% in the current season of QGM. So that is a very low mark. Some have as much as 6.8% 3PM.

0.865 G^15 / (0.99 G + 0.01)^14 = 0.5
G^15=0.578 (0.99 G + 0.01)^14

Real solution:

G = 0.6278

So that's lower than I thought.

What about for S = 0.03 (top 20 in the QGM are at this level or higher)

0.633 G ^ 15 / (0.03 + G - 0.03 G) ^ 14 = 0.5
0.633 G ^ 15 / (0.97 G + 0.03) ^ 14 = 0.5

G = 0.85033

General case

Prob Curve: Roughly speaking, 1% better in 3P% requires 10% better in FG% to match up.

50% line -

if 2P% = 0.55 + 0.1 3P% then V(2P%) ~ V(3P%)

If we change the 3P line for PER from 28 to 9 (9 + 1 = 10, 10x value of 2s) will it make more sense?

The negatives are happening through other parts of the formula.

Let's make our own Monte Carlo model and conclude the value of attributes.

Variables:

Pr_2 : propensity to shoot 2
2%
Pr_3 : propensity to shoot 3
3%
FR : rate of free throw chances
FT%
B% : Block %
S% : Steal %
DRB%
ORB%

For each side play hundreds of games, and see how many wins with different combos of stats.

Let's get a range of values for each of these variables to test within, and then I will write up the simulator in F# since I'm familiar with that, and it's a fun little test project:

From the player stats for QGM:

3 PT FGA / all FGA = Pr3 ranges from 0% to just over 40%

Let's model from 0.0 to 0.5 for Pr3 (Pr2 = 1 - Pr3)

FTA per 2 PT FGA : FR ranges from 0.08 to 0.53

Let's model from 0 to 0.6 for FR

FT% ranges from 15% to 50.8%

Let's model from 0.10 to 0.60

2 fg% ranges from 0 to 0.2
Let's model from 0 to 0.3

3 fg % ranges from 0 to 0.068
Let's model from 0 to 0.08

B%

We have block stats, but not enough data for a perfect block% stat

Let's look at FGA / minute on average. 0.284

Consider B% to be Blk / 0.284 Min
Consider S% to be Steals / 0.284 Min

How do we estimate a player's ORB% and DRB% when these have to be balanced so that P1 ORB + P2 DRB = 1 and P1 DRB + P2 ORB = 1 for each matchup of players?

Scale ORB / Min and Oppo's DRB / Min so that these are one, and likewise the opposite.

Our EST B% goes from 0 to 0.23. Let's take it from 0 to  0.25
Our EST S% goes from 0 to 0.26. Let's take it from 0 to 0.30

ORB/Min goes from 0 to 0.21. Let's take it from 0 to 0.22
DRB/Min goes from 0 to 0.59. Let's take it from 0 to 0.65

0.284 FGA / minute * 48 minutes * 20 quarters max = 272 possessions each until the model considers it a time runs out scenario, and picks the player with higher score as winner.

Round to 280.

Model

The Monte Carlo Model creates N distinct samples within these bands, and pairs them all off, for X round robins, to generate X*N observations of for each sample. The number of wins is totaled for each and the samples and their wins are returned.

I was thinking 1000 samples, and 1 round robin to start.

Run 1 settings:

    let qparams =
        {
            Prop3Pct = Band(0.0, 0.4);
            ThreePTPct = Band(0.0, 0.08);
            TwoPTPct = Band(0.0, 0.2);
            FTPct = Band(0.15, 0.6);
            FTPer2FGA = Band(0.0, 0.6);
            BPct = Band(0.0, 0.25);
            SPct = Band(0.0, 0.30);
            ORBRate = Band(0.0, 0.22);
            DRBRate = Band(0.0, 0.65);
        }

    MonteOutput.runMonte qparams 1000 1


Still running after 6 minutes. I don't know if that's a bug in my model or just that running a 1000 * 1000 sims its taking that long.

I bailed out after 9 minutes, and decided to try 10 samples as a test to see if the model code was working at all.

    MonteOutput.runMonte qparams 10 1

When 10 samples took over a minute, I bailed out. I ran 2 samples, and then broke into the debug menu.

Possessions left is showing at 782041226, which is way bigger than 560. Somehow my possession tracking was screwing up.

I was setting netPoss to negative, but then subtracting it, so I outsmarted myself and possessions were going up.

Now, running with 2 samples finished within a second. Running for 100 samples finished in a few seconds.

1000 samples took about a minute and a half. So then I opened the results in excel.

I ran Excel regression on the results.

My model must still be broken because 3P% shows an extremely negative coefficient of -1913, while 2P% shows positive. My model was symmetric, and was accidentally crediting the opponent with catching the snitch.

After a fix to the recursive call in my model, I tried it again. Unfortunately this change broke tail recursion, so I wasn't sure if it would overload the stack, or take way longer.

It didn't seem to take any longer though, so once again I looked to the results. This time, the model made more sense.

	Coefficients	Standard Error	t Stat	P-value
Intercept	-262.61	14.04	-18.70	4.49E-67
3PA/FGA	846.70	21.00	40.32	4.41E-211
3P%	4807.28	105.96	45.37	5.23E-244
2P%	744.02	41.03	18.13	1.11E-63
FT%	92.96	18.06	5.15	3.20E-07
FTA/2PA	113.19	13.75	8.23	5.71E-16
B%	388.90	33.09	11.75	6.10E-30
S%	455.97	27.69	16.47	4.97E-54
ORBR	570.00	37.52	15.19	5.29E-47
DRBR	248.31	12.65	19.63	1.04E-72

New PER Model

Calculated Params:

2P% = (FG - 3P) / (FGA - 3PA)

FTA / 2PA = FTA / (FGA - 3PA)

B% = B / (0.284 * Minutes)

S% = S / (0.284 * Minutes)

ORBR = ORB / Minutes

DRBR = DRB / Minutes

3PA / FGA : self explanatory

uPER : -260 + 850 3PA / FGA + 4800 3P% + 750 2P% + 90 FT% + 110 FTA/2PA + 390 B% + 450 S% + 570 ORBR + 250 DRBR

PER = uPER / (avg uPER) * 15

This should result in an average of 15, the EWAs and baselines by position may need change later, but for now, leaving the replacement PER at around 10 makes sense.

Being a relative novice to javascript, I've made my changes, but we'll see if they crash or not.

Here we go:

>npm run build

and
>npm start


Running a month, I noticed all PER was being set to 0.

After debugging, It looks like there's no Three Point attempts stat being carried over.


We can run another model, and do 3 per FGA, 2 per FGA and FT per FGA, instead of using the per 3pa or per 2pa stuff.

A lot of the P values seem even stronger in this model:

Intercept	-51.94	10.54	-4.93	9.7E-07
3P/FGA	21847.30	327.85	66.64	0
2P/FGA	983.33	49.26	19.96	8.75E-75
FT/FGA	473.10	36.41	13.00	9.04E-36
B%	364.34	32.83	11.10	4.72E-27
S%	457.75	26.64	17.18	4E-58
ORBR	529.70	36.80	14.40	8.26E-43
DRBR	257.49	12.53	20.56	1.65E-78

So, I will do:

uPER = -50 + 22000 3P/FGA + 1000 2P/FGA + 480 FT/FGA + 360 B% + 460 S% + 530 ORBR + 260 DRBR

Building and trying once again:

It worked! Or at least the PERs are not 0.

Let's test with Katie Jones, a PER leaderboard entry who is a Big (Wo)Man, and Leo Howe, the current PER 2nd place, EWA first place leader

Jones: 52.4 min, 3.6 FG, 28.8 FGA, 0.1 3P, 2.0 FT, 6.6 ORB, 19.9 DRB, 2.8 S, 2.7 B

Howe: 70.6 min, 2.8 FG, 30.4 FGA, 0.3 3P, 2.9 FT, 0.8  B, 4.3 S, 5.6 ORB, 14.5 DRB

Jones uPER: 498.6116

Howe uPER: 503.5859

Jones: 33.7

Howe: 34.1

So the scaling factor is around 14.76 uPER per regular PER.

It looks right!

EWA is also looking right. It is calculated based on PER and minutes played.

Offensive Wins and Defensive Wins are still screwed up, but I don't think those affect player valuation the same, so I can worry about that later.

Time to start a new league with the new valuations, but that's for another post.