|
Everyone, thank you again for bearing with us during the various site maintenances of the last week or so. You might not be surprised to learn that there were some missing pieces in SA's infrastructure. Perhaps most appallingly among those, there had not been a database replica server running for several years. Further, if the site had a backup process in place before, it must have lived on that missing replica; there did not seem to be anything configured. You read that correctly—for many years, these forums were one malformed shell command, bad SQL query, or hardware failure away from vanishing in a puff of magic smoke. Over the last week we've largely rectified this. We've taken backups, spun up three new database servers (2 live forums, 1 archives), imported all the data, set up replication (meaning changes to the main database server are sent to the replica database server), and after the cloud provider maintenance is behind us we'll work on migrating the forums from the old database servers to the new (timeline TBD). This will involve an easter egg hunt as we migrate all of radium's little cron scripts over, but thankfully Jeffrey already has a good handle on where these live. One of those new servers is already in production and handling database queries for things like displaying attachments, calculating the results for the "Who posted?" feature, as well as for various administrative tools. With this work—and more—completed, we have the utmost confidence that we'll be able to recover from several potential worst-case scenarios, both in the upcoming downtime for cloud host maintenance and as we proceed onward into the future. Thanks for loving the forums.
|
#
¿
Apr 23, 2021 07:56
|
|
|
|
| # ¿ May 22, 2024 16:58 |
|
|
Believe it or not, radium did have a replica server set up. In the post-radium years there were on-and-off issues with it, and at one point I believe they even had someone from the cloud hosting company come in and help recover the data on that server, but somehow it still did not survive long enough to make it through to the move to ~the cloud~ in 2016. GreenBuckanneer posted:You have offsite backups, right? Not just a local one? "3-2-1 rule" That is an essential part and should be done in time for the cloud host maintenance. jimmy mnemonic posted:Thanks so much for the enhancements! I think Jeffrey might've copied that from their own announcement.  Notorious R.I.M. posted:Ok the MySQL DBA in me really wants to stare into the abyss... ZDR converted just about everything to InnoDB, so thankfully very little.
|
|
#
¿
Apr 23, 2021 19:51
|
|
|
SniperWoreConverse posted:also dark mode fucks up so many smiles and tags lol I believe there will be a thread for cleaning up those things.
|
|
#
¿
Apr 23, 2021 21:43
|
|
|
Ignatius M. Meen posted:I love dork mode so much, thank you. Being able to peer into that alternate timeline, I can assure you it would have simply been a redirect to the frontpage while they get XenForo 2.0 set up, it's totally coming you guys, they swear.
|
|
#
¿
Apr 23, 2021 23:17
|
|
|
Before the 2016 cloud move, the hosting costs were, in fact, much higher, but my understanding is the forums also made even more money back then.
|
|
#
¿
Apr 24, 2021 07:27
|
|
|
spankmeister posted:This is great, maybe this place will survive after all. After all it's been through, it deserves to. This would be nice to have someday. quote:
We would eventually like to see Attachments 2.0, a modern system that would allow you to attach more than one image to a post,  not keep the images stored in the database , and have actual ease-of-use (drag-and-drop convenience, images not disappearing on post preview, can edit/remove them without involving a moderator, etc.).Not an arbitrary image host, though—just post attachments.
|
|
#
¿
Apr 24, 2021 08:39
|
|
|
Justin Godscock posted:Anyways, I support our own image uploading service as well. We unofficially use Imgur but it would be nice for sheer convenience to not have to visit an exterior site. Personally I'm more worried about their escalating anti-adblock masures. The forums have so many imgur images at this point that I would hate to see it go the way of imageshack, waffleimages, etc.
|
|
#
¿
Apr 24, 2021 09:01
|
|
|
Justin Godscock posted:All the more reason to have our own service. So many classic goldmine threads are there because of the images in them. A lot are now kinda pointless because the images are now lost when imageshack and the like died. The SA imageshack archive still exists, but whether it's hooked up properly to the relevant posts is another question. Some of the waffleimages still exist on what I believe to be their last-surviving mirror so there is hope of preserving many of those, as well.
|
|
#
¿
Apr 24, 2021 10:15
|
|
|
Jaded Burnout posted:I mean zero shade by this, but thank you for bringing the forums into 2006. It is but one stop on our journey to modern times. 
|
|
#
¿
Apr 24, 2021 19:07
|
|
|
Hope everyone has a nice [your local time of day].
|
|
#
¿
Apr 25, 2021 05:30
|
|
|
We're back.
|
|
#
¿
Apr 25, 2021 11:42
|
|
|
Incessant Excess posted:There are no plans for this I take it? If this happened it would likely be for 'optional' ones, like CCCC. Basically not for e.g. BYOB/FYAD, if that's what you're asking about.
|
|
#
¿
Apr 25, 2021 12:00
|
|
|
Ignatius M. Meen posted:I have one critique of Dark Mode and that is that sometimes there's a weird flash of white when I'm loading a new page on the forums. It could just be a tech issue with my browser and thus unfixable but when it happens I feel like I just got flashed by lightning (and their junk is extremely gross). That's indeed a browser thing. If you're on Firefox, there's a couple of about :config settings you can look up to toggle to fix that. Spinz posted:I have discovered attachments no longer work and I have made a thread asking about it in the technical questions sub forum I replied there but I'll post the summary here: I'll be able to start addressing this in about an hour, old attachments should still display fine, new ones won't appear correctly until it is addressed, and the new attachments have not been lost - it's just a display issue caused by the code not falling back gracefully enough. Sorry for the inconvenience!
|
|
#
¿
Apr 25, 2021 19:56
|
|
|
thotsky posted:Make sure to test that you can actually restore from those backups you've taken, or this thread could become a self-own. That was how we got the data to the new database servers in the first place.  But, yes, testing that your backups actually can be restored is a very important step in having a backup.
|
|
#
¿
Apr 25, 2021 21:10
|
|
|
Spinz posted:I have discovered attachments no longer work and I have made a thread asking about it in the technical questions sub forum Cross-posting: astral posted:A fix for this is pending; when deployed, the old attachments system will be more robust until we can replace it with something better. Thanks for reporting the bug!
|
|
#
¿
Apr 25, 2021 22:15
|
|
|
Spinz posted:THANK YOU VERY MUCH HOORAY The fix has been deployed.
|
|
#
¿
Apr 25, 2021 22:50
|
|
|
slowly lowering the "break glass in the event of infinite scrolling" emergency hammer
|
|
#
¿
Apr 28, 2021 09:39
|
|
|
Mr. Nice! posted:That's a cloudfare feature added by lowtax when he turned on a bunch of cloudfare stuff years ago. You're thinking of the gigantic obnoxious one that was there for a while. The little ones in the corners are added by forums JS.
|
|
#
¿
Apr 28, 2021 22:35
|
|
|
christmas boots posted:Was that when he enabled the "boost" feature and then claimed that it had immediately bankrupted the forums? That one was called "Argo" and it absolutely was on track to do so – he was billed over $500 for the first month during which it was on for only the tail end. The month after that, he was billed over $1500,
|
|
#
¿
Apr 29, 2021 00:13
|
|
|
Steadiman posted:What in the world is Argo? And why is it so expensive? I'm sure it sounded great without looking at the actual pricing. 
|
|
#
¿
Apr 29, 2021 00:32
|
|
|
I need to correct that; after looking into it, the $1500 was "it was on for the whole month" after all. Over 15,000 GB @ $0.10/GB. The $500 it cost him the previous month would have covered about a week and a half of use, and should have been enough of a red flag on its own.
|
|
#
¿
Apr 29, 2021 01:28
|
|
|
Zil posted:That would've required some proactivity on his part. Nah, that would've been taking the before-hitting-button step of estimating monthly bandwidth use based on current/previous numbers and punching this number into the very complicated cost formula.
|
|
#
¿
Apr 29, 2021 02:29
|
|
|
|
| # ¿ May 22, 2024 16:58 |
|
|
The upgrade is complete; pages might be a bit slow for a short period while the cache warms up.
|
|
#
¿
Apr 30, 2021 05:07
|
|