All in the Game with the Elisa Esports Development Team

Most esports operators have their own solution for gathering and displaying game data for the players and spectators, but until this year, Elisa lacked an all-encompassing system, despite the huge popularity of Elisa Esports. When the Elisa Esports development team took on the challenge of building a system of our own, they really upped the whole game – and by more than a few notches.

“The timeframe to have all the pertinent game data for the post-match analysis, for example, is about twenty seconds, so you really can’t do it manually”, says Jaakko Wallo, one of the developers in the Elisa Esports team.“Unless you have really fast typing fingers!”

The system the team has built is affectionately called Tupari, which comes from tuotannon pikku apuri in Finnish, meaning “production’s little helper”. Tupari is a centralised system that utilises game data to automatically analyse and display match, team and player statistics for the viewers, to generate graphics, and even to control the lights in the gaming arena.

Tupari consists of three sub-systems, with the web app Turska (“Cod”) acting as a public repository for all the data, Monni (“Wels Catfish”) in the background gathering data and statistics by monitoring the matches, and Ankerias (“Eel”) controlling the arena lights during the matches using data from Monni.

“Tupari runs on a Google Cloud virtual server, and one of the tricks is to have public IP visibility but have some limits to it. We use Terraform for system administration, and it works extremely well for our purposes”, explains Wallo.

Although the team only started the development work last January and had the first public version running in March, the system is all ready to go for this November’s Elisa Masters Esports Tournament. Through the usual trial and error, Tupari can now handle all the aspects needed to run a spectacular and spectacularly well-flowing esports tournament.

“Monni is coded with Go and it handles all incoming information while providing event data to Ankerias through GRPC. Usually, there would be several observer clients communicating to different addresses for this kind of operation, but Monni really simplifies things”, Wallo says.

“Turska is a CRUD system, with the backend running on Laravel and the frontend on React, and some of the data – such as the maps used in a single match and team information – is input manually”, continues Valtteri Palonkorpi, software engineer in the Elisa Esports team.“At the same time, Turska fetches data from Monni while, for example, keeping track of every event in the match automatically.”

Turska stores all the team and tournament data using Steam IDs and generates real-time statistics for match streams while displaying the tournament history for each team. All statistical graphics were already run on Turska in the previous Elisa Esports Tournament.

One key element in the system is Telegraf, which is an aggregate for time series data. Telegraf stores the data it has gathered in an Influx database for Turska to use.

“The system has the capability built in to receive two data streams at the same time from the same match”, says Wallo. “The primary stream is the one used during the match, and the other one is a backup with its own view of what is going on.”

While the match is underway, the system gets updates about ten times every second. However, that data in itself is not very valuable. The match data is automatically input to Influx, which generates summary data that provides the really interesting datasets for everyone involved.

When the Elisa Esports team started designing Tupari, there weren’t any notable, publicly accessible systems to use as a benchmark. That led to some development glitches that the team members now look back on with mild amusement.

“Last summer, we wanted to generate even more graphics from the statistics using PHP in Turska. It really was a mistake, as the server ran out of steam running thousands of lines of Influx code every minute, with some kind of memory leak also in the mix”, reminisces. “Fortunately, that taught us the lesson that you should plan well in advance before you go into production.”

Jaakko Wallo tells us a bit more about the lessons learned and the overall goal: “We didn’t necessarily make any wrong design decisions, but the needs of the system became clearer along the way. Sometimes you have to kill your code darlings. It’s just reality.”

“When developing graphics and interfaces, you should also have a clear view of what is happening in the control room of the live stream. We are in constant communication with the directors to build the best possible live stream for the matches and tournaments.”