Bug Exploit data analysis

  1. 1. Introduction
  2. 2. Context
    1. 2.1. Exploit bug
    2. 2.2. League of Legends.
  3. 3. Pyke Special Build
    1. 3.1. Description
    2. 3.2. Usage
    3. 3.3. Effects
    4. 3.4. Consequences on the abusers
  4. 4. Global spell
    1. 4.1. Description
    2. 4.2. Methodology
    3. 4.3. Usage
    4. 4.4. Effects
    5. 4.5. Consequences on the abusers
  5. 5. Dark Seal Viego
    1. 5.1. Description
    2. 5.2. Methodology
    3. 5.3. Usage
    4. 5.4. Effects
    5. 5.5. Consequences on the abusers
  6. 6. Conclusion

Poppy

Ever wondered how bug exploit was handled by Riot Games? Here is an analysis by the numbers on three bug exploits.

Introduction

Recently, some bug exploits in League of Legends have been known publicly and exploited by players. These bugs can have different severity and impact on the game. The best way to understand how much it’s used and the impact it can have on this competitive game is by crunching numbers.

This study focuses on three bugs :

The analysis is based on data from the Riot Games API. The specificities of these bugs make it (sometime) easy to detect the abuse with only end of the game data provided by the API. We will see that these bugs are very different, both in terms of severity and impact, as well as how it is handled by Riot Games and the influence of Reddit over them.

Context

Exploit bug

Exploited bugs, in video games, represent the discrepancy between the game design and the program implementing the design giving an advantage, in short, something that the player is able to do, but shouldn’t. Most of the time, these bug are found and fixed by the devs before the game launch, but some can slip through the tests usually due to them happening under various and very specific conditions.

Speedruns, competitions about finishing a game as fast as possible, usually rely heavily on those exploits. There is a controversy about considering exploits as cheating. In the end, it’s up to the community and the game developers to decide, case by case, if an exploit is a bug or a feature.

.

League of Legends.

League of Legends (LoL) is a competitive video game, especially a MOBA (Multiplayer Online Battle Arena), launched in 2009 by Riot Games and in constant development until now, and derived from the Warcraft III mod DotA. A game lasts around 20-40 minutes and confronts 10 players in two teams of 5, the goal is to take over the enemy base. Each player controls a champion, an entity with its own stats, unique abilities, and customizable with items, themselves with specific characteristics and particularities.

Bug exploits in League of Legends come most of the time from unusual interactions between components of the game, such as champions abilities, items, runes, map…

Pyke Special Build

Description

The special build comes from the combination of two runes, Aftershock and Manaflow Band, and the item Catalyst. When these three elements are equipped and ready, the next Pyke hook will provide a huge amount of stats and healing.
These conditions make the bug not that easy to pull, and considering the Manaflow Band effect has only a limited number of uses (10 times), the impact is quite limited, yet can turn over a fight or even a game.

Usage

On the EUW server, over a few days before and after the first Reddit thread, this build appeared in 683 games, played by a total of 586 players.

Games type distribution

Figure 1 : Number of games with Pyke "special" build per game type

Figure 1 shows that the build was mostly played in normal games.

Number of games All Ranked
1 526 139
2 41 9
3 12 3
4 3 0
5 1 0
6 2 0
10 1 0
Table 1 : Number of "Special build" Pyke games per player, for all game types and ranked

Games per player

Figure 2 : Number of games with Pyke "special" build per player

Table 1 and Figure 2 show that one player used this build in 10 games, but only in normal games. In ranked games, no players used this build more than 3 times.

Effects

This bug had little to no effect on the competitive aspect of Pyke as a whole. Banrate stood at 18%~19%, pickrate at 10% and winrate at 48% for the whole timeframe studied in ranked games.

About winrates, as shown in Table 2, we can observe a drop of winrate in both normal and ranked queue.

Queue Special build All Pyke
Coop vs AI 1 0.987303
Normal 0.442231 0.477577
Ranked 0.403614 0.488687
Table 2 : Pyke Winrate, with and without the "special" build

This drop in winrate can be explained by the experience of the players on the champion. More than half the ones who tried this bug had only 3 games or less on Pyke, including the games abusing the bug.

If we take only the experienced players, with at least 10 games on Pyke in the timeframe, the winrate is 50%, slightly higher than the global Pyke winrate. However, as the number of concerned games is really low (only 28 games), as well as the winrate gain, we can not conclude in a massive advantage due to this bug.

On the side of interesting stats, Figure 3 show the distribution of healing done by Pyke in a game, on the left for all Pyke, on the right only by those using the special build. As expected, using the special build leads to generally higher healing.

Heal distribution

Figure 3 : Distribution of healing done per Pyke

Figure 4 represents multiple stats day by day, with the patch date and the first Reddit thread about the bug as reference. First, the blue bars show the number of games with Pyke special build. We can see that the build was only played once in the week before, a spike of uses after the Reddit thread, and then a massive drop, suggesting that most, if not all, uses of this build was only to abuse the bug, that was quickly fixed after a few days. The lines show mean healing done by Pyke, with and without using the special build. It makes it easy to see when the bug began to be used, and when it was fixed.

Games and heal per day

Figure 4 : Pyke healing and Pyke “special build” games per day

Consequences on the abusers

All players who have used this bug are still playing as usual, hence no ban has been received, and for the few contacts I got with some of them, they did not receive any warning either.

Global spell

Description

Due to an error in the calculation of an area under specific circumstances, spells with a rectangle area of effect coming from a champion on particular spots on the map would hit everything on the map. The bug appeared multiple times over the year, first with Kayn “fixed” in March, then Ornn (fixed in April), then again for Kayn in June. All these bugs looked very related, until the point when it came that it was the same bug affecting a whole type of spells (see that video).

This of course leads to a big advantage, being able to farm minions on the whole map and even killing other champions anywhere. This means an almost guaranteed win.

Methodology

As this bug abuse is not reliant on items or runes, it becomes more tricky to detect the instances of this abuse. However, it leaves traces that are findable though the Riot API. I started with finding the cases where a big share of team damages (>80%) is held by a unique champion, as well as farming share. This method leaves out a lot of games, but it allows at least to find players abusing, and by looking at their matchlist, it is possible to find more games where they abused the bug. On the other hand, it can also lead to false positives, and to avoid counting an innocent player as guilty, all games were manually reviewed to ensure the abuse. In the end, it is not possible with only the Riot API data to find all games, but at least an important sample, and the following analysis will only be about the games found.

Usage

Over a week, between 17/06/2020 and 24/06/2020, the bug has been abused in at least 614 games, by 194 players.

Game type distribution for global spell

Figure 5 : Number of games with global spell per game type

Games per player

Figure 6 : Number of games with global spell per champion

Figure 5 show the distribution of games per type.
We notice that the usage is far higher in Ranked than for the Pyke bug with more than 87% of the games being in Ranked, and that it was even used 3 times in Clash.

We also see in Figure 6 that Poppy was the most used champion for this bug, with Kayn and Illaoi being the two other champions used.

Number of games All Ranked
1 62 52
2 43 36
3 29 22
4 22 20
5 9 9
6 9 8
7 8 6
8 5 5
9 1 1
11 1 1
13 1 1
15 2 2
16 1 1
20 1 1
Table 3 : Number of Global Spell abuse games per player, for all game types and ranked 
Number of games All Ranked
1 1890 1638
2 56 45
3 18 14
4 11 10
5 5 5
6 6 6
7 4 5
8 2 2
9 3 2
10 6 6
12 1 1
14 4 4
16 1 1
18 1 1
22 1 1
37 1 1
Table 4 : Number of Global Spell assist games per player, for all game types and ranked

Table 3 shows the number of players who used the bug in a set number of games. We can see that a player used it 20 times, and when a player used it 8 times or more, it was only in ranked games.

Table 4 shows the number of players who were an ally of the player abusing the bug, for a given number of games. If being in a game with an abuser is not suspect, being in more than one might mean that player is queuing up with the abuser to get free LP. One player even managed to be 37 times in the team of an abuser.

Effects

Sitting at 96% winrate, this bug abuse is, not surprisingly, very efficient. But it also had indirect measurable effects.

Figure 7 shows the impact on Poppy banrate in ranked which was multiplied by 5 after the bug was posted on Reddit, from around 1,000 bans a day to more than 6,000 on 19/06, Poppy becoming for a short time the 72nd most banned champion (still quite low however), and then slowly decreased over a few days. Poppy pickrate also went up just a bit on that day.

Games per player

Figure 7 : Poppy pickrate and banrate, and number of games using Global spell bug

Consequences on the abusers

127 accounts among those which abused the bug (around 65%) do not show any game played after June 2020, most likely meaning they were banned by Riot. Moreover, I contacted accounts that were still playing, and they all reported that they were hacked and indeed got a ban that was lifted after they recovered the account. Most of them presented a few days gap in the games played, maybe the time needed to recover the account, which comfort that statement.

Interestingly, some of the accounts that were in games with an abuser but did not abuse themselves do not show any games since then, meaning that even indirectly taking advantage of the bug is in the scope of Riot when it comes to punishment.

Dark Seal Viego

Description

Dark Seal is an item that allows to collect “stacks” on kills and assists, up to 10, and lose some of these stacks on death, and these stacks provide bonus stats such as Ability Power.

After “fixing” a problem with Dark Seal and Viego in Patch 11.5, when Viego dies during possession, the number of Dark Seal stacks goes way higher than supposed. It is a case of Integer Underflow. To make it short, when Viego dies with Dark Seal, it loses 4 stacks, but when it has 2 stacks for instance, instead of going to 0 stacks, it goes to -2 stacks. The main problem here, is that the number of stacks is encoded in a way that doesn’t allow negative, and the underflow makes the value going to the highest possible, in this case, as a 16-bits integer, 65535. That’s a lot of stacks, and that provides way too much stats bonus, which lead to some abuses as we can see in the video.

At 15:11 (UTC) on 04/03, Viego has been disabled to hotfix that bug.

Methodology

In this case, a consequence of the way too high damages of Viego is that the reported damages at the end of the game were also way too high to be handled by the system, which gave out negative damages, a so unique case that it makes it easy to spot. Interestingly, Viego wasn’t the only champion concerned by this bug, but some champions interacting with Viego in some ways, such as Mordekaiser or Yuumi, also showed negative stats.

Usage

This bug appeared in 1032 games by 673 players in EUW between 04/03 and 05/03, and to be more precise, before the time Viego was disabled to be hotfixed.

Game type distribution for Dark Seal Viego bug

Figure 8 : Number of games with Dark Seal Viego bug per game type

In Figure 8 we can see that this bug has been abused in Ranked for a little more than half of the cases, one third of the cases for Normal and the remaining in Coop vs AI.

Number of games All Ranked
1 492 223
2 103 51
3 35 16
4 16 7
5 11 5
6 8 7
7 3 2
8 1 1
9 1 1
10 3 2
Table 5 : Number of Dark Seal Viego bug abuse games per player, for all game types and ranked 
Number of games All Ranked
1 3614 1809
2 160 57
3 34 23
4 7 2
5 3 2
6 6 6
7 4 3
9 1 1
Table 6 : Number of Dark Seal Viego bug assist games per player, for all game types and ranked

With Table 5 we can see that among the 673 players who used the bug, 181 used it more than once, up to 10 times which is a quite high number given how quickly the bug was fixed.

Table 6 reports the allies of the abuser, and 215 allies happened to be in the team of an abuser in more than one game.

Games per player

Figure 9 : Viego games, with and without Dark Seal bug, and Viego banrate over two days

In Figure 9 we can see that the vast majority of the bug cases happened after the first report of it on Reddit.

Effects

With 76% winrate, this bug gives largely an upper hand, but not as much as with the Global Spell bug, although it increases to 82% for players with more than one game, and 87% for 3 games and more.

In Figure 9, we notice the beginning of the rise of the banrate of Viego with the increase in number of cases of the bug. Viego banrate goes from 31% (despite some instability during the night due to low number of games) to 37% before being stopped by Viego deactivation, which means that it would have gone even higher without the deactivation. After Viego reactivation, the banrate was slightly lower at a quite stable 29%.

Consequences on the abusers

69 accounts out of the 181 which used the bug more than once do not show any game played since April 2021, which represents only 38%, far lower than with the Global Spell bug, a difference that can partly be explained by the biais in sampling, which did not happen for Dark Seal Viego bug, but also by how easy this bug can be activated by accident.

On the side of the “frequent allies”, only 54 of the 215 convenient teammates seem to be banned without directly using the bug, which comforts the theory that the bans are also given to this type of players.

In all cases, this give only a lower estimation of the number of bans, as an account can be recovered if the abuse has been made by someone else than the rightful user of the account.

Conclusion

The most impressive, yet not that surprising, conclusion of this analysis, is how high is the impact of Reddit on the abuse of a bug once it is published.

Then, we can better see that Riot effectively punished the abusers, even looking for indirect abusers, when the bug is game breaking. On the other hand, on non game breaking bugs, Riot does not act at all, despite a quite clear will to take advantage of the bug by some players.

The code for the analysis is available here : https://github.com/HextechLab/Bug-Abuse