8 cheated times were uploaded to the leaderboards on 2019-09-15. The times were set on a real console using CTGP Revolution and uploaded from the same console. The times were blatantly cheated being significantly faster than the current records and clearly using some sort of crude clip through wall cheats. Within an hour MrBean35000vr noticed the times and disabled all new time uploading to the leaderboards. Over the next few days a patch was developed to prevent the exploit used to activate the cheat code in CTGP and an update fixing this was released on 2019-09-18. Ghost upload was reactivated at this time, but given our inability to detect with certainty potentially cheated times, the decision was taken to disallow all potentially cheated times from the vulnerable version of CTGP. Potentially cheated times will still be uploaded to our server but will not be publicly displayed anywhere.
CTGP Revolution v1.03 was designed with security in mind. Unfortunately Mario Kart Wii's online services were plagued with cheaters and we wished to address this in our mod. The original aim of our security systems was to make it difficult for anyone to cheat in the live online racing between CTGP players. In June 2016, we added support for Time Trials to CTGP creating an offline competitive environment where leaderboards compared the fastest race times in the world. Similarly to the online racing community, Mario Kart Wii's Time Trial community suffered a long battle with cheaters, particularly cheaters who would subtly cheat to get the fastest times in the world. In Time Trials, cheating is particularly devastating because any temporary lapse in security could lead to the creation of cheated times that could be considered as legitimate records for many years, or even permanently. Supporting Time Trials therefore added a new burden on CTGP's security to avoid the same problems that Mario Kart Wii's leaderboards had.
Unfortunately it is impossible to make a mod totally secure against cheaters. This is a provable fact, not an opinion. Therefore the aim of CTGP's security is to make it very difficult to cheat. To that end we meticulously blocked all public tools and pieces of software that allow cheating on Wii from working on CTGP. This therefore means that anyone wishing to cheat on CTGP would need to build a new tool or a new technique. This restricts the number of people with the know-how to develop a cheat for CTGP to only the brightest members of the modding community. We hope that such people would be responsible enough not to do so, and to use their talents more productively.
Whilst developing CTGP's security we identified many potential new ways of making exploits that could attack CTGP. We categorised these into ways that we considered feasible for a novice to exploit and ways that we felt were complicated enough that they would never be used. We also considered how easy it would be to block such exploits and made security systems to block the easy ones. Unfortunately, one of the exploits we opted not to block (as it was too complex to fix) is the exploit used in this incident. This exploit has been present in CTGP v1.03 since release, but has recently been independently discovered by an attacker who used it to cheat.
At 2019-09-15T19:55:57Z five ghosts were uploaded to the leaderboards. These had just been created by the attacker who had used the exploit to install a simple cheat into their game. Over the next 40 minutes, three further ghosts were uploaded. We were first alerted to this by members of the community at 20:40. To quote MrBean35000vr at the time: "Oh dear.". For an example of the community reaction at the time see luke_gb's live stream.
At 20:49 all uploads to the CTGP Time Trial leaderboards were disabled. This is the simplest emergency measure we can take in order to prevent further damage. This means that, whilst anyone could still be Time Trialling offline, no one could add times to the leaderboard including the cheater. At the time Chadderz was asleep and so Bean woke him up to assist. Initially we investigated who the perpetrator was and quickly established their identity as a well known player named 'Star'. We then began to investigate the possible exploits that could have been used. We also contacted Star in the hopes he would assist. Unfortunately Star initially refused.
By 21:52 we had strong evidence that the exploit mentioned above was the one that was used. Happily this implied that 150cc original track times could not be attacked due to the extra security of those. Unfortunately as described above, the exploit was deemed to complicated at the time to fix. We were now faced with writing a practical fix on short notice despite the complexity. We knew this would take multiple days of solid work. MrBean35000vr worked for the rest of the night and made decent progress.
The following day we continued to work on the fix and had a workable solution by the end of that day. Without assistance from the perpetrator we were still worried that we had made some mistake and that fixing this exploit would not actually stop the cheats. Mercifully at the end of the day Star decided to help us and confirmed our suspicions about the exploit used. He also went further and sent us the exact code for the exploit that he used so we could test that our fix worked. We are very grateful for his assistance. Overall the fix took about 49 hours of work split between MrBean35000vr and Chadderz.
As the fix was very complicated, it required extensive testing which was conducted the following day. In the early hours of 2019-09-18 the update was finally released. We wish to thank all of the testers who helped us during this time. Once the update was released, uploading to the leaderboards was reactivated. However, due to the possibility that they were cheated, all 200cc Time Trials on all tracks and all 150cc Time Trials on custom tracks which were uploaded after 2019-09-15T20:50:00Z on the unpatched versions of CTGP will not show up on the leaderboards.
As mentioned previously an exploit that was known to us was used to attack CTGP. We had deemed this exploit too complicated to fix and too obscure for anyone else to find. Clearly we were wrong. For this I must personally apologise, people trust us to make decisions about security and so we must take responsibility for those decisions. We will be reviewing other decisions we have made and will look to improve CTGP's security further still. For the time being CTGP is 'secure' in the sense that we do not know of anyone who has broken its security, but clearly this incident shows us there are people who are potentially willing and able to do so.
As mentioned previously, it is impossible to make a mod that is totally secure from cheaters. All we can do is make it very difficult. In this instance we were lucky that the attacker was so blatant. It made it very obvious that we needed to investigate and that we needed to act. It is of course possible that someone could have cheated on CTGP in a more subtle way. This incident shows that such things would not necessarily be automatically detected. We will always try to increase the difficulty of cheating on CTGP and try to detect it when it does happen, but we will never know for certain that it has not happened. For example, people could have been using this exploit for years on CTGP to cheat subtly as it has been in CTGP since release. We have no way to know for sure.
In the interests of being as thorough as possible, we've decided not to accept any time trials set on vulnerable CTGP clients which were set after the cheat was known to us. This gives us greater assurance that the cheats could not have been spread or used by anyone else. We may consider making special exceptions to this rule if some times of particular interest were set, for example world records. Please do get in touch with us if you think a time should be counted. To facilitate this, we are allowing the potentially cheated times to be uploaded but they will not display publicly without manual intervention.
Although the cheats uploaded were obvious, it is possible that someone else was using the same exploit to install more subtle cheats. The exploit would have allowed them to install any cheat code into the game which means they could've created subtly cheated ghosts or cheated in the online multiplayer. We are not aware of this occurring, but we have no way of knowing for sure. Therefore all community members should consider CTGP players as potential cheaters until the fix was released.
As mentioned Star did eventually assist us with creating the fix. We are very grateful for this. However, in future, we would ask anyone who does discover an exploit for CTGP to tell us before using it publicly. We will then patch the exploit before anyone can use it and will happily publicly credit the creator with the discovery once the patch is released.
The only positive thing that can be drawn from this is that the security systems we have created for CTGP have stood the test of time. It has taken more than 5 years for an incident as serious as this to been found.