Thursday, February 22, 2018

Transforming from Basketball Sim to Quidditch Sim

This is a new series, on my efforts to make a Quidditch league simulator from the code of Basketball GM.

Basketball GM can be played at: https://basketball-gm.com/
The Basketball GM codebase is on Github at: https://github.com/dumbmatter/basketball-gm
My Quidditch fork of BBGM is on Github at: https://github.com/apeterson-BFI/basketball-gm


Past Efforts


I had previously created a Teams JSON for import into the very customizable basketball simulator Basketball GM. The Teams JSON just changed the teams from Fake NBA team names, to Quidditch teams, both mentioned in Harry Potter, or made up from reasonable team concepts.

My initial teams covered the houses of Gryffindor (plus a team for Professors), the other Schools of Magic, the professional and national teams mentioned in the series, plus teams for species and teams for divisions of the Ministry of Magic.

Recently, after I started pursuing the full Quidditch GM project, I was made aware of the full league known as the British and Irish Quidditch League.

I have added those teams which I didn't already have.

The JSON project is compatible with basketball gm, just import my json file as a custom roster: permalink to this version.

The teams json is an important part of the Quidditch GM as a whole, as it gives me a flavorful base for the game. I have added to it customized names based on the top 100 boys names, girls names and surnames in England right now.

My next plan for names is to bring in the rest of the world, weighted towards Great Britain overall, to give it a Harry Potter vibe. Basketball GM's default names have always seemed too modern and basketball for this purpose.


Quidditch GM: Changes from Baseline BBGM


Quidditch GM is still basketball GM, just with a few simple changes to the BBGM code base at this point. My goal is to make it more Quidditch, less Basketball, one change at a time.

According to Github, I am currently 20 commits ahead of the BBGM code. That means, I've made 20 changes from it, and Dumbmatter, the creator of BBGM has not posted any new changes to the master repository since I started working on this.

  1.  Added the harrypotter_teams.json file, which should be imported when creating a new league.
  2. Added a now way out of date planning document (quidditch plan.md). I will do my planning here instead, so that's going away.
  3. Added logo files for the team logos used in harrypotter_teams.json. These were on imgur before when I was running the json through the main BBGM website instead of locally on my pc. Now they are in the repository.
  4. Nav Bar says Quidditch GM.
  5. GameSim.js: This is the meat of the real changes that make the game more Quidditch-like:
    1. Fatigue per possession reduced by 10. Previously fatigue was getting ridiculous, to the point where no one made any shots and a single game went on forever.
    2. Instead of ending regulation if the 4th quarter is over, end if the 21st quarter is over. This is a copout to prevent endless games where a snitch is never caught.
    3. After a possession, end the game immediately if either team has made a 3 pointer (conceptually: 3s are now Catching the Snitch).
    4. Eliminate free throws from 3 pointer attempts (catching the snitch doesn't let you score a quaffle). Allow only 1 free throw if fouled in a 2 pointer attempt.
    5. Reduce chance to make 3 pointer by a factor of 10. Catching the snitch is harder than making a 3.
    6. Reduce chance of making 2 pointer by half. Scoring a goal with the quaffle should be harder than a basketball 2 pointer. And scores were getting ridiculous.
    7. Reduce the benefit of assists to made shot % by half (to scale with the rest of the odds).
    8. 2 pointers are now considered scoring a goal with the quaffle. 10 points as in Quidditch.
    9. 3 pointers are now considered catching the snitch. 150 points as in Quidditch.
    10. Free Throws are still in the game, but worth 5 points (half a goal) now (shoot once if fouled). I want to remove FTs completely soon, but not yet.
    11. Free Throw make chance cut in half, like 2 Pointer make chance.
    12. Quarter length set to 48 (from 12), because games were running to that last 21st quarter too often. Works in conjunction with the fatigue reduction to make box scores more legitimate.
As a result of these changes, the game feels a bit Quidditchy, but the BBGM engine is now stretched and stressed in weird places.

Here are some box scores from a season I just simulated as the Ravenclaw Eagles:

Example Game 1

Lost 15 to 180 against the Wimbourne Wasps. My team made 1 goal and 1 free throw. Wasps made 2 goals (20 points), 2 free throws (10 points), and caught the snitch (150 points) to end the game in the 1st quarter. 


Example Game 2

Lost 1040 to 1550. The game ended on time with no snitches caught after the end of the 16th overtime period. One of my players went 0 for 142, trying to catch the snitch.



So it's completely ridiculous when interacting with a Fatigue Model from Basketball GM, that expects games to last 4 quarters, maybe even 7 quarters with Triple OT, but never 16 OTs, and never ending quick games in 1 quarter.

But it definitely has a Quidditch feel. If the snitch can't be caught, the game keeps on going. If someone catches the Snitch quick, you lose badly.


PER Ratings / Player Valuation / Free Agency


However, the main reason I wanted to post, is that these Quidditch changes completely break the Player Efficiency Rating (PER) advanced statistic, which is not only for display in the UI, but also used for player value calculations.

According to Basketball-Reference.com, for the 2017-2018 NBA season, these are the top PER players (https://www.basketball-reference.com/leagues/NBA_2018_leaders.html):

Crit
RkPlayerSeasonAgeTmPER
1James Harden2017-1828HOU30.5
2Giannis Antetokounmpo2017-1823MIL28.8
3Anthony Davis2017-1824NOP28.2
4LeBron James2017-1833CLE27.7
5Stephen Curry2017-1829GSW27.4
According to the same site, the lowest PER ratings for players with 30 games played minimum (to exclude oddball cases): 

CritCrit
RkPlayerSeasonAgeTmGPER
1Marcus Georges-Hunt2017-1823MIN344.2
2Semi Ojeleye2017-1823BOS514.3
3Dwayne Bacon2017-1822CHO354.5
4Arron Afflalo2017-1832ORL434.7
5Josh Huestis2017-1826OKC535.1

PER is calculated so that the average is 15. Usually the best are around 30, the worst at near 0.

Now let's look at Quidditch GM's PER tables for my simulated 2017-2018 season, after a full regular season:

Player Efficiency RatingPER
1. Joshua Johnston13BPsCHU151.2
2. Jack Howell3BPsSAL100.8
3. Adnan Rodgers3BPsBUL100.7
4. Jayden Taylor3BDpPsBAL100.0
5. George English10BPsBEA94.5
6. Patrycja Price53BPsMAG83.1
7. Aiden Paul2PsWIZ76.7
8. Safiyyah MoseleyBAPP76.6
9. Reece BairdAPP76.2
10. Julia Ashton3ABDpPsKOL72.5

I couldn't find a way to pull up a list of worst PER examples, but here are a few I just found on 1 team's bench:

Maya Poole: -35.2 PER
Philippa Gibbons: -46 PER

And I think the issue is showed most starkly between a Point Guard and a Power Forward, both with the same overall player rating in the game: 38 (awful, worst player in the league level).

James Knight - PF - 6.8 points per game: +/- of +25.9, but his PER is -31.0

Mikolaj Roberts - PG - 1.4 points per game: +/- of 23.6, but his PER is 41.1

Two players, both should be awful, one of them has better than James Harden PER, one has worse than Josh Huestis PER.

So, in summary, the changed length of games, and chance of making shots has busting the traditional PER statistic beyond repair.


Quidditch PER


How should a Quidditch PER work?

Let me go back to the basketball PER, and find out what it is doing. According to a Basketball Reference blog post, called Calculating PER, PER "is a per-minute rating developed by ESPN.com columnist John Hollinger. In John's words, 'The PER sums up all a player's positive accomplishments, subtracts the negative accomplishments, and returns a per-minute rating of a player's performance.'"

So PER is a per-minute rating of accomplishments in Basketball.

Let's work out Mikolaj Robert's ridiculous 41.1 PER in my QGM simulated season.

Roberts has a stat line of:






In Excel, I started working on the formula in Calculating PER, using Mikolaj's stats, team stats where the formula requires it. I noted almost immediately that Mikolaj takes only 0.5 3s a game, and makes none of them.

My league intermediate values for the calculation were:

Factor  VOP   DRB%
0.665605  1.469407 0.750613

Unadjusted PER is basically adding up 12 terms, some which are good things, some bad things, and then dividing by minutes played.

Mikolaj's values don't look impressive so far with the 12 sub sums.

#1 #2 #3
0.000 0.333 0.157
#4 #5 #6
0.000 -0.735 -2.757
#7 #8 #9
-0.167 0.073 0.000
#10 #11 #12
0.588 0.000 -0.067
These total up to -2.57488

We divide this by his minutes per game, which is 0.09901

So Mikolaj's Unadjusted PER is -26

How does he get to 41.1?


The next step is to adjust for pace:


pace adjustment = lg_Pace / team_Pace

According to the glossary referenced in the article, pace is:

Pace Factor (available since the 1973-74 season in the NBA); the formula is 48 * ((Tm Poss + Opp Poss) / (2 * (Tm MP / 5))). Pace factor is an estimate of the number of possessions per 48 minutes by a team. (Note: 40 minutes is used in the calculation for the WNBA.)

I don't have Time of Possesion or Opponent Time of Possesion. I tried to get pace, but for now I skipped it, because I have no idea how to do it. And the final PER calculation is more important

After taking my pace adjustment, then I calculate PER
PER = aPER * (15 / lg_aPER)

If League average PER is negative, and every Player's pace adjusted PER is negative, than PER will work in reverse. The more negative your performance, the larger positive your PER will be when you divide by average per.

Quidditch PER Answers


Even within this lightly modified basketball GM Quidditch, the PER formula is already way off. First of all, 3s are now worth 15 times more than a 2 point fg, rather than 1.5 times, even if you don't consider the Game Ending aspect of it. 

PER considers 3 Pointers made as one of it's factors.

FGs made are scaled by (2 - factor * team_ast / team_fg) * FG

For Slytherin, Mikolaj's team, that multiplier is (2 - 0.665 * 0.65) = 1.56

Since FG = 2P + 3P
These two summands can be rewritten as:

3.56 3P + 2.56 2P

If we rescale PER calculations so that 5 old points = 1 new point, then, instead of adding 1 more point to represent 3 pts vs 2 pts, we need to add 28 points to represent 150 points (30 * 5) vs. 10 points (2 * 5).

So the term 3P would be changed to 28 * 3P.

This is the easiest, smallest change we can make to PER. Let's try it out and see how it affects things. PER may still end up League negative in the average, because both 2 and 3 pointers have their chances reduced.

I replaced players[i].stats.tp in the code with 28 * players[i].stats.tp. Rebuilt the Node setup and launched a season, running it through the regular season.

Player Efficiency RatingPER
1. Joseph TaylorBPsBAL87.9
2. Noah BartlettBAL84.6
3. Frankie CookPUD76.2
4. Sophie BaileyPsBEA75.0
5. Lily Parry3BPsSAL74.7
6. Spencer Williams3PUD74.0
7. Theo AliILV73.9
8. Jessica Robins3BPsBAL73.5
9. Edward Cox3GOB72.2
10. Mariam Collins5WIZ72.1

There's a few 3 point shooters here, but mostly random guards again.

If I go into the Debug Console, maybe I can find the Average aPER in the league? Not really.

Seth Welsh: amazing player:

Seth Welsh3ABDiDpGF2417881$30.00M thru 20188074.365.426.81.439.4

Only 39.4 PER.

Let's look at the PER leader with 87.9

Overall rating 57.
Horrible player.

Redoing Advanced Stats for QGM v. 0.0.6, Completely


I need to put together a statistical model for QGM, that will assign reasonable PER values based on actual contribution to winning games.

I think it would be easier to target EWA and then back-correct that to PER, the opposite of how BBGM calculates PER and then translates to an EWA value.





No comments:

Post a Comment