Weekly progress posts is clearly not working so we’re going to try monthly now, surely that’ll work. So lets talk about what happened in the past 2 months.

March

In March, I started working on a massive new system for the osu daily challenge statistics site. It was requested a long time ago to add pp statistics, this item has been on my backlog for ages and I finally spent the time to actually work on this. The setup is to use Laravel queue workers that will run osu tools commands for calculating pp values.

The data I had already saved in the database was insufficient to calculate how much pp a score is worth. This meant I had to adjust the database to store more information but also that I have to read all information from the osu api again. Collecting all information at 1 request per second took a little over 12 hours, but I shouldn’t have to do this again, hopefully.

A lot of code had to be refactored in order to get everything to work correctly, but it is working now. There are a lot of things I could say about how I tried to optimize this calculation but that’s for another blog post. There’s a bit more about how I’m running the pp calculation for the first time, but you’ll see that in the April section of this post.

April

In March, I only worked on the osu daily challenge site but in April I worked on a few different things.

NPM broken

When AFuxy started reviewing my Trinity code and fixing the merge conflicts caused by them he noticed that the CI was failing. Turns out that the npm install step on one of our packages started breaking.

This seems to be a bigger problem that not only I am dealing with as reported by other people. Luckily it was reported that pinning the npm version to an older version does fix this problem. I applied this fix to a few repositories that were affected by this. This is not a good permanent solution so I’ll have to keep monitoring this so I can unpin the version once this problem has been fixed.

MiSide Speedrun Mod

This is not a change made by myself but by Stafil0. Stafil0 made a PR for the MiSide Speedrun Mod about a low FPS toggle. Turns out a basement skip has finally been discovered and was discussed on the speedrun.com forums.

In order to perform this basement skip you need to have very low FPS which you can’t really achieve in game without external tools. But after some discussion it was decided to allow this if this could be implemented in the mod. There is some additional work in another PR but I haven’t had the time yet to do a full review of this. This update will drop soon though so I’m curious what effect this will have.

osu daily challenge calculations

On the osu daily challenge website I can now calculate pp values but running this calculation for the first time can cause quite a lot of problems. According to some calculations I did it would take over 24 hours (potentially close to 48 hours) to run this calculation for all scores. There are definitely some big optimizations I can do still but I didn’t want to go all out in the initial implementation especially because this would cost time.

Since these calculations will take a long time (and also use a lot of CPU power) I did want to babysit the first calculation to ensure things wouldn’t break for weird reasons which would require a full recalculation. Because of this it has taken a lot longer to finish this calculation but I am approaching the end slowly. I am very glad that I monitored the calculation a lot because I discovered many bugs this way that would’ve otherwise caused problems numerous times.

I have also realized that there is something bigger wrong with deciding what scores should count towards weighted pp. For example this player has a pp score of 1.5k pp while on their osu profile they only have a 698pp score. The difference between 1.5k and 698 is massive so something has to be wrong here.

I suspect that mods have to do with this but I haven’t been able to properly diagnose the issue yet. Nevertheless this means I’ll have to make more changes and potentially rerun pp calculations. This is the progress I’ve made so far though.

Caturra Broken

A crazy bug occurred that I still do not understand at all but it is something that happened. While doing some routine infrastructure inspection after experiencing an outage from our hosting provider I noticed the docker containers that run the Caturra website were missing. The website was up and the containers were showing up with docker ps but not with docker compose ps. This could break updates in the future so just to be sure I restarted the containers so I could see the containers show up properly again.

To my surprise I saw that the website was online but displaying a frontend we haven’t used in months. AFuxy and I were very surprised and had no clue at all how this could happen.

At the time of writing I still have no clue how this happened. What I remember from the logs the containers were created from an image that was previously built which makes sense. However, I prune all old images every month to prevent them from taking unnecessary storage so this image shouldn’t even exist anymore in the first place. The proper fix is to rebuild the image and restart from there but this didn’t work due to the npm install fix not being implemented on this version of caturra.

We did manage to fix this quickly but I would love to know how this could’ve happened. If anyone has any idea how this happened feel free to message me anywhere.

Monthly blog posts

I’ve noticed that I don’t have time every week to work on projects and I also don’t have time every week to write a blog post so I’ll try to make it a monthly post instead. I spent about an hour writing this post today so I can hopefully continue making posts like this starting now.