Boologam

Server Report 12th Aug

This post is to address the recent server outages caused at exactly 23:00 on 12th august, our tps was dangerously low to the extent that it was unplayable causing some players to notice weird glitches and very slow server peformance

Timeline of events

August 11 22:00

One of our staffs noticed a dangerously low tps and decided to report it to the higher adminstration team, The staff was commanded to restart the server, However tps degradation was still present

August 11 23:00

Hour later higher administration staff implemented particular flags, While doing so the staff had noticed a really shocking bug the server which was supposed to use 30 Gb ram was only using 5 Gb due to incorrectly configured jvm flags, The staff replaced the flags and fixed it causing the tps to stablize again and get on a normal-ish level, The issue was then marked solved and then server was put off from maintenance mode, Yet the issue was still present just barely noticeable due to low player count

August 12 17:00

staff attempts to fix the unstable tps level by implementing chunk filters and many more configurations which for an unknown cause made the server tps go down even more than it did on 22:00 August 11th, Now the issue was notified to the whole team and everyone was working towards fixing the issue for the sunday player rush, But the maintenance was delayed due to an important decision making session of, whether to turn of the server and fix the issue or wait till the players go offline

August 12 19:00

It was decided to put the server in maintenance for a couple of hours for the greater good, By this time the vps hosting company joined in to fix the issue, They endedup increasing the ram count and the disk count to max level as they had speculated that the server was maxing out, It suprisingly helped a little bit

August 12 21:00

Vps hosting company finished their work, Staffs now downloaded the server backup and uploaded it with the intent of binary searching to find the lag source, Which would go on till hour 1:32

August 13 01:32

Issue was finally fixed after finding out the culprit

  • The server was not using proper flags
  • Too many unused and lag inducing plugins
  • Bugged plugins
  • Improper reporting

In summary

We sincerely apologize for the negative impacts these disruptions have caused. We understand the impact that outages like this have on players. We will design proper reporting and first information so that the players stay informed, We look forward to your contribution.