Editing
Archived:February 2023 lag incident
(section)
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
== Incident == === PixelPrinter Grief === On the evening of February 3rd, 2023, the server had been fairly active. Mod [[Serenity7321]] had specifically decided to paste the attic of player [[dwrr_]]'s with hundreds of PixelPrinter-generated item frame pictures of Chinese communist revolutionary [[wikipedia:Mao_Zedong|Mao Zedong]], which had previously been downloaded to the server by Admin [[RandomUser34]]. She had done this same picture-based grief before in the [[Mount Xavier Compound]] storage room a few days before, totaling 855 item frames. The grief this time, however, had totaled a significantly larger amount of item frames, causing the server to experience item-frame-based entity lag. All item frames had been removed shortly after via the /killall server command by Serenity. The server recovered immediately after this. === Decision to upgrade server memory === Server owner nc77812 had decided shortly thereafter that he would upgrade the server RAM from 5 to 8GB effective immediately, announcing his decision in the Attempt2 Discord server's #voice-general chat. He cited future performance concerns as the reason to upgrade. The server then restarted at approximately 2:00AM EST. === Beginning of Incident (2:00-4:00AM EST) === After the server restarted, lag issues immediately became apparent. As people joined, chunks around them failed to load. Server TPS (ticks per second) dropped from the regular 20 average to a near 1 to 5 average, rendering the server virtually unplayable. nc77812 and RandomUser34, joined with the feedback of dwrr_, attempted to troubleshoot the problem, by performing things such as mass killing of entities, and attempting to find a root cause of the lag by installing and using the Spark server diagnostics plugin. As it became clearer that their actions were leading nowhere, multiple things were suggested to try and solve the problem. RandomUser34 and dwrr_ suggested that the issue could be fixed by switching the server to the Fabric API, a move which was swiftly rejected by nc77812, who insisted the issue instead involved the server world, citing previous issues with PixelPrinter-derived entity lag. Eventually, nc77812 made the executive decision to delete the current server world and revert the server's world to its last backup of January 30th, 2023, explaining that it would be easier than switching the entire server and its plugins over to Fabric, which used a completely different plugin/addon system than Paper/Spigot. This action generated immediate controversy from the playerbase, but was nonetheless executed. === Continuation of Incident (Feb 4-5) === Server Admin RandomUser34 continued investigation into the server lag issue. After trying many things, including running Spigot, Paper, and Fabric APIs, and even running the server as a clean-slate, Vanilla server with a new world, it became clear that the issue was the fault of the server host. He immediately contacted host support thereafter. After the incident, it was discovered that he had accidentally wiped both the server's End and Nether worlds, of which there was no backup. === Support resolves issue (Feb 5-7) === Support was relatively slow to respond, due to the incident timing occurring on a weekend. After about 2 days, support acknowledged they had transferred the server to another node, which had been overwhelmed with high resource usage. This essentially had throttled the server's performance to a point of unplayability. The host then transferred the server to another node free of charge. The server immediately reverted back to normal performance after the transfer. The server reopened later in the day on 7 February.
Edit summary
(Briefly describe your changes)
Please note that all contributions to A2wiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see
A2wiki:Copyrights
for details).
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Archived
Discussion
English
Views
Read
Edit source
View history
More
Search
Navigation
Main page
Current events
Random page
About A2wiki
interaction
Help
Community portal
Recent changes
Upload file
Tools
What links here
Related changes
Special pages
Page information