TL;DR
There are a ton of updates in IVK Skill 2.0; this one covers how we went from comparing players to comparing sessions in determining player skill ratings (MMR).
- IVK Skill 1.0 compared players > IVK Skill 2.0 compares players’ overlapping match time
- Why? Increases fairness in skill ratings!
- Bonus: Accounts for teams ending up shorthanded, without rewarding opponents rage quitting
You can watch our YouTube video covering this update, or keep reading!
The skill rating algorithm update
Comparing players in a match against each other seems like “the thing to do” to determine skill ratings. It’s what every skill rating algorithm does, and that’s exactly what we were doing with IVK Skill 1.0.
But plenty of multiplayer games allow players to join matches in progress (backfill) or to rejoin after getting disconnected. Treating these players the same as those who played the full match doesn’t really make sense.
Instead, IVK Skill 2.0 compares players on if and when they were in a match together AND fairly compensates players in ranked modes when their teammates quit early.
Comparing sessions instead of players
Comparing players based on if and when they were in the match together means that we’re processing a list of sessions, with one or more sessions per player.
Fig. 1: IVK 1.0 processing players; IVK 2.0 processing sessions with timestamps
Here’s how IVK Skill 2.0 actually compares session data. Let’s say we have three players in a match with the following playtimes:
- Player A: plays the full match
- Player B: joins late and leaves early
- Player C: gets disconnected in the middle, just rejoins toward the end
Fig. 2: one match, three session histories
Step 1: identify events and encounters
The first thing we do is identify all the events, or state changes, where players join or leave the match. This allows us to look at every interaction period, or encounter, between every combination of players. This means that every interaction between any two players can be grouped and summarized into a relationship between the two players.
Fig. 3: every join and leave becomes an event
In our example, we can see that even though neither Player B nor Player C played the full match, we still have two overlapping encounters that allow us to define their relationship.
Fig. 4: the highlighted regions are all of Player B’s encounters with Player C
Step 2: create the relationship matrix
Once we’ve identified every player combination, we have a relationship matrix. This describes:
- the expected outcome of every player versus every other player — a bit like the probability of one player outperforming the other
- the actual outcome between those two players — usually a comparison of their scores per minute, normalized from zero to one, one way or the other*
- the importance of the relationship based on the data — things like overlapping time, uncertainty of ratings, different weights for same teams, parties, and bots
Step 3: determine the player’s new skill rating
After we have the full relationship matrix, we reduce each row into a single result. That tells us things like how we expected each player to perform, how they actually performed, and what their new skill rating should be.
Fig. 5: each row reduces to one result per player
*Comparing players against themselves
The matrix also includes a comparison of players against themselves. This makes sense in specific cases, generally based around the details of the underlying skill model.
Basically, it’s an easy way to account for different sample sizes, because being the best player in a match with 100 players should be rewarded more than being the best of two.
Quick example: let’s say Player A had 50 points per minute, Player B had 100, and Player C had 20. We can put that into a binary outcome matrix where one means a higher score, zero means a lower score, and 1/2 is a tie.
The weighted comparison adjusts like this:
- Zero weight → weighted average outcomes always span the full range from 0 to 1, regardless of player count
- Equal weight → outcomes land at the midpoints of equal-sized partitions
- Double weight → outcomes subdivide the 0-to-1 range at perfect intervals
It can make sense to normalize outcomes this way for modes with different player counts, like battle royales.
Fig. 6: self-comparison weight as a sample-size correction
How to model team interactions
When we compare session data, we also model team interactions. Just like with the player data, we determine the events — when the teams change — and use those events to partition the match into a series of encounters between teams.
Fig. 7: team encounters, built from the same session events
The process then is the same as with players: the encounters between every pair of teams are summarized into a relationship with net expectations, outcomes, and weights, and those form a relationship matrix that we can aggregate into per-team results. (Pretty straightforward with two teams, more complicated with… more.)
The team results are then combined with previously determined player results and ✨magically✨ turn into an updated player skill rating (MMR). The magic is so powerful that theoretically a player could switch teams mid-match and still get a fair skill rating update.
Fig. 8: the rest happens by magic
Adjustments for ranked play
Ranked play is different, and our updates treat it that way. With these changes, we’re able to adjust ratings appropriately if your teammate(s) leave early. On the other hand, we don’t want opponents rage quitting to sabotage your ratings. That’s why even if one player on the other team drops out, we evaluate their skill from your team’s perspective as though they were always at full strength.
Fig. 9: your teammates leaving is forgiven; their teammates leaving is not exploitable
Want a full rundown of IVK Skill 2.0 and how it’s THE skill rating system to base matchmaking on? Let’s talk.
