After testing less strict skill-based matchmaking in Modern Warfare 3, Activision found that players were more likely to quit, therefore justifying the system’s existence moving forward.
In April 2024, Activision shared an 18-page report explaining Call of Duty’s matchmaking process. The report revealed more information about skill-based matchmaking would be released in June.
However, after the devs missed that deadline, players voiced their frustrations. Activision heard the complaints and revealed a corresponding blog explaining the results of a test in North America with loosened skill consideration in MW3 matchmaking.
“The test showed that players with a wider skill gap were more likely to quit matches in progress and did not return to the game at a higher rate than players with unchanged matchmaking,” Activision claimed.
CharlieIntel posted a graph from the study showing that lower-to-mid-level players stopped playing more frequently than higher-skilled players.
Activision revealed the results of the MW3 MP test where skill factor was reduced for 50% of players in NA. This chart shows player retention % based on skill rating. Lower-to-mid skill players saw retention decline. Only the highest skill bucket had an increase. pic.twitter.com/68SCahiPsz
— CharlieIntel (@charlieINTEL) July 26, 2024
Activision added that the likelihood of a player quitting during a match increased by 80%, and 90% of players in a loosened skill group did not return at a higher rate than the control group.
Subscribe to our newsletter for the latest updates on Esports, Gaming and more.
“Playing a better opponent may push players to become more proficient over time, but being severely outperformed in their matches, our testing has shown, instead leads players to quit matches in progress or to not play multiplayer,” Activision argued.
There were also tests with tighter skill-based matchmaking. Results showed that the quit rate was down 90%, but it negatively impacted high-skilled players, and that idea was scrapped.
The Call of Duty development team has considered launching a multiplayer playlist that doesn’t include skill as a factor in matchmaking. However, testing showed that low and mid-skill players would be less interested in the playlist, resulting in primarily high-skilled players filling lobbies.
So, while ping remains the number one factor in matchmaking, skill is also a key component and will continue to play a role in future series entries.
For more on skill-based matchmaking, check out why the system is so controversial and why some FPS titles, such as XDefiant, ultimately decided against using it.