Spectral Wrapped 2024
Every year about this time I write a blog post about whatever I did with my Dota/Esports related things throughout the year.
The major thing this time was not having enough time to complete the goals I set up for myself last time. Yet I was still moving in the direction of fulfilling the said goals. So I’d rather treat them as a roadmap, and in this sense it was a somewhat productive year.
My major goal for 2025 is to get back to making content (I wrote the same exact thing last time, didn’t I?), reflect on my experience in esports, work on some more personal projects and goals and start hosting community events.
In 2024 I was busy working on a completely new and unrelated project with a new team, accumulating some new experiences for myself.
Because of that, Dota projects became slightly less important. Plus, there was a gigantic productivity hole for a couple of months around the spring.
But let’s look a bit closer at what I’ve done with Dota this time and how the goals I set last time are holding up.
What’s been done
Had a lot of plans that had to be rescheduled, and there was a giant gap in productivity due to personal matters. But generally speaking, most of the changes this year came in form of backend improvements and prototypes, so not too many of them will be noticeable from the outside.
Throughout the year I’ve had some opportunities to work at events: The International 2024, PGL Wallachia Season 2. But also some small community events as a caster and even helped out with stats during some Deadlock events (though my friend Degaz filled in this niche for me in most cases).
Server-side stuff
The year started with the site analytics reports. I already used them for the last year’s blog post, and will use some fun bits and pieces from it below (in another section).
One interesting thing to note was fighting back bot attacks. Ever since 2024 started, there was increasing load coming from places which I suspect being AI crawler bots. Can’t say I was too happy with that, but only by November it got too much, so I started looking into it. After banning several botnets, the number of requests came back to normal (as opposed to ~150k requests in 5–15 minutes).
Another thing that turned out to be very important: stability improvements to the backup/unload/unpack tools, report descriptor cleanups and massive optimizations to items queries. With the increasing number of Immortal players and matches, this sort of changes became crucial to keep report processing times more or less in check. And REALLY massive datasets became much easier to offload from the server and process again on my home machines.
Because I am still stuck with a very specific stack, some of the tricks allowed me to compensate for some of the features not being present, and generally reduce the space requirements for the patch datasets. Plus, some of the improvements act as an adidtional compatibility layer, so I did not had to change too much of the existing code.
Another bit of changes came in the form of datasets “chunkization”. This became very useful for starting items and consumables specifically. This feature was in the works for a while, and I was making small adjustments to it throughout the year, but one issue that was bugging me was the processing times required for this feature and space required to “emulate” some of the more complex queries. Thankfully, this chunkization approach helped to compensate for this issue, and also improved items processing even further as a nice bonus.
With all these changes combined, as well as some internal improvements to the script working with Stratz (as well as the same kind of bans of malicious on Stratz, which resulted in significant increase in processing speeds), I managed to reduce report processing times by 3–4 times (from ~36–38 hours down to ~9–12). This still doesn’t take consumables processing into account, sadly, but this wouldn’t result in a major increase at this point. I have some ideas how to optimize the process even further, but this is not a huge priority right now.
Also finally implemented (or rather fixed, as they already existed) patch and date filters in the match fetcher.
As an bonus: implemented (and verified that it’s stable) Deadlock Game Coordinator mirror service for internal use (for my internal APIs and reports) and added a new Markdown engine for one of the upcoming projects.
Big releases
Now about the things that you could’ve noticed yourself.
First of all, it’s the new Roadmap and the new Donations page (using the new Markdown engine), heavily inspired by Thunderbird’s announcements.
While I am not too happy with asking for donations, recently it became more important than ever, especially with some some TOs and teams heavily abusing the site with massive requests surges.
Dota projects are all non-commercial and are all supported exclusively by donations and my own reserve funds (any collaborations are usually treated as big donations).
The page lists all the Patreon/Boosty tiers and what they offer, and it should be a good starting point for those who want to show their support for the things that I’m doing.
Prototypes and Upcoming projects
And this is where we are starting to talk about more exciting things.
First of all, here’s a new prototype for patrons: personal match stats in the form of custom reports, with some additional data.
Second, it’s upcoming starting items and consumables. It was in the “polishing” state for a long time, but as of now the only things left are translations strings and a release blog post.
Third, SuperNote prototype. It was initially aimed to be a technical base for talent sheets and datasheets for TOs, as well as a way to connect all the different APIs I have. It ended up evolving into something better and more “unified”. Markdown engine came with it, too.
First iteration of it added some neat stuff to the site
- better section navigation for the texts
- improved multi-language support for big texts
- a very convenient way to host documentation
- Markdown to JSON processing for datasheets and vice versa
As the end goal, I want it to be able to provide a giant cheatsheet as a PDF for every event. But also I want to use it to store documentation for various stuff (not just my APIs, but also results of my researches on how to work with Steam Datagram or things like that). And also I want to bring the full archive of all my old blog posts to the site, with all the translations available.
Currently, I have only some pages ready, but the technical base is already there, and I’d say 60% of the work was already done.
Fourth, Deadlock experiments. Internal Game Coordinator mirror/API service, some basic tooling for the reports, basic implementation of the replay parser (Degaz made another one, and in the end we both kind of stopped working in this direction, but still). Deadlock reports hub is in the works, too.
Fifth, scripts to process Dota GC output, metadata and private metadata. I already wanted to do this for a long time, and there were some replays missing for our scrims and community events. This led me through this rabbithole of learning, what can be found in the metadata, what can be found in the private metadata, how to decipher it and what I can do with it.
Currently I have a set of scripts to do the conversion from the net dumps and produce JSONs that can be fed into LRG2. And I’m pretty sure there is more to get from all this, but it’s not a huge priority right now. It’s a huge achievement, but the application is very niche.
Sixth, and probably the last one: Turbo Meta Trends.
This is the prototype I wanted to make for a very long time. Since there is no “Turbo Leaderboard” and most people who have ranks don’t play unranked (especially Turbo), it was a bit tricky to find out the “best of the best” of Turbo.
I ended up filtering matches based on average ranks in matches of known players and gradually applying it to “unknown rank” players. This approach worked well enough, but felt overcomplicated, so I replaced it with something much simpler and forgiving.
The main reason why I didn’t release it yet is how disappointing the findings were. One would probably expect SOME sort of differences with the meta proper, right? Well, while there are some differences, generally Turbo stats were so similar to Immortal Rank meta trends reports, that I decided it’s not worth the time. The prototype is ready, but I’ll probably let it to just marinade for a bit, until I find out something better to do with it.
UPD: …maybe “turbo leaderboard” will be the reason to release this later down the road.
Small updates
Match cards now have records for this match listed
Added Players Laning section, though it’s still turned off by default for now
Added groups of categories (to the API too)
Updated Interactive map to 7.35
Added Tools section UI (updated version of interactive map, hero grid, analyst dashboard, builds, etc)
Added shady events notice
Modals improvements
NerdsBuilds Improvements: facets, translations, new D2PT API
Guame filters improvements
Also checked all the existing reports for missing matches, will be slowly working through the reports to add every missing thing. Also started working on my Source 2 replays archive. Also started working on my historical stats thing. Also still working on some missing events that bug me.
Some fun bits from the site analytics
Don’t really want to think about some fun comments, so I’d rather just leave screenshots up to interpretations.
One funny bit is how I made a small update and accidentally broke analytics logging for a couple of months.
The big surge of users closer to the end of the year feels a lot like more bots. But I wasn’t really able to trace some more of them. And the sheer number of requests coming from Hong Kong is suspicious too.
This chart is quite funny as it shows the only instance where I didn’t clean up the bot requests from the database and accidentally unbanned one of the subnets for a couple of minutes.
Plans for 2025
Most of the plans for the next year will be pretty much the same as last year. The overall direction stays the same. For this reason I will copy the descriptions from the previous blog post, but update them slightly to fit the current goals.
- More community events. I want to start hosting more “spectral scrims”, along with Artifact and maybe Underlords events, and even some regular custom game brawls. (still holds true)
- Match stats explorer. While most of the matches in my reports are available elsewhere, there are still some cases when I either filled the data manually or parsed replays from unticketed lobbies. For these cases it’s pretty much impossible to find the data anywhere, and I usually spend a lot of time and effort to get everything right for my databases (e.g. rewatching the VODs to fill in as much missing data as possible). Match stats explorer is essentially a frontend for matches in my (currently live), aimed at giving access to some of the match stats (except for ranked reports). And it will be useful for my community events, hehe. (still holds true)
- Nerds Builds Improvements. I want to get back to this section at some point and implement a search bar at the navbar at the top. I would also like to make some adjustments to how skill builds and starting items work. And I was also considering adding 3rd party builds at some point: default in-game builds, as well as Torte De Lini and GoX guides. (not sure about comminity builds, but otherwise still true)
- Reports quick switch in a category. This feature actually already exists, but it’s hidden behind the early preview. One thing I would like to solve is figuring out which category to use for the report when rendering the switcher dropdown. (still considering this, otherwise still true)
- Redesign. The initial design was heavily inspired by the Dota Pro Circuit page and was intended to be used for “small reports” which were essentially pretty reddit posts. Since then this design made less sense and I wanted to move away from it for a while now. (still true, but more like a complete engine rewrite first)
- Replay Hunt. There were a couple of corrupted/remade matches throughout the 2023, as well as one particular tournament from 2022, which I can’t just ignore. It’s a lot of manual work, but I want to get it done eventually. (still true, have only one event left)
- Minimap Generator. I started thinking about this ever since 7.33 was out. Comparing current iterations of minimap images to the older ones is depressing, and I would like to do something about it. In theory it should be rather easy to “generate” a good looking minimap image, using the map metadata (same one that’s used by the interactive map) and a bunch of presets made as vector graphics (which I have to make first). (still true)
Bigger Things
- Skill builds, starting items, consumables, ability draft support. I was planning to finish this before 2023, but something went wrong (priorities changed). Almost ready at this point, just preparing localization strings and ready to go.
- Reports Lurker. The API is ready, but hidden. Will get back to this later throughout the year.
- Artifact logs parser. Still holds true, but it’s not a huge priority.
- Players/teams ranking for reports (still holds true)
- Player with hero stats in the reports and other Player Profile improvements (still holds true)
- More content: blog posts, videos, features, etc (still holds true)
Massive Projects
- Artifact stats hub. Still planned.
- Reworked reports engine. In progress.
- SuperNote. In progress.
- Live match recorder. Still planned, but not sure when I’ll get back to this.
- Heroes Archetypes. Still planned.
Closing words
I guess that’s it? The main plan for this year is going further along the path I set before. Hopefully, next year I’ll have more exciting new features to talk about!
Happy 2025! Enjoy Dota this year more than ever before.
Also don’t forget to follow me on Twitter/Bluesky, Telegram or join my Discord.
You can also support my projects by donating, this really helps a lot.
