04/22/2024 - Emergency Server Restart (4:32 PM - 2:00 AM EST, 9.5h Downtime) Due to Plex DB Corruption
- Atom 5ive
- Apr 22, 2024
- 2 min read
Updated: Apr 23, 2024
Both Plex Databases have become corrupt, I'm working to restore backup to fix this issue now. Full downtime was: 4:34pm-2:00AM Backup Restore Happening: Est. <4h 30m for [Monitored] Plex Restore Fix.
[Unmonitored] Plex will be after this, will update when I have an updated time.
11:48PM EST - Update: I'm truly sorry about the amount of downtime this has caused. One small thing spiral into a bigger thing, into a bigger thing. I'm working to fix this and get it back up and going as soon as I can. If this application restore doesn't work, the next step is updated the main project core. If that doesn't work, I'm falling back to an older OS snapshot. If that doesn't work, well, let's hope that works. Otherwise, it's going to be a long morning for me.
Then a complete rebuild will need to be done. <-- This is worst case scenario's, and I have backup of backups of everything. So it shouldn't come to that. But no matter what, I will get this server back up to its normal self again. Please bear with me as I do.
Plex Services restored. Ended up having to do a full OS Rollback. This is only temporary to get BOTH Plex Services up and going. It seems there's an update to Plex where they've changed how the DB itself is being read, which corrupted everything. Causing even my backups to not work with the new standard Plex roll out. I'm not alone, others are having this issue. For the time being, I'm sticking to what's working and not shutting the server down. Again I'm sorry for this downtime and the services should not be touched for a while. Please take the time to bookmark this page https://uptime.atom5iveosb.xyz/status/atom365status to keep a quick glance update to if services or down or not. I'm also in the coming weeks going to be rolling out an alert service, so people can get SMS or Email depending on their choosing for major outages like this. Please check into my site as well the Blog here for updates to then site and server major changes are coming soon. For the $15 a month subscription, I will be working on IPTV support, which will include paid per view packages and more. I'll loop everyone as I know more till then, again sorry for this outage, I will continue to monitor the server in the morning and throughout the day tomorrow. Till then, I'm now off to sleep. -Atom




Comments