Speed issues today?


PDA



Rail Driver
October 17, 2012, 01:31 PM
Anybody else notice THR being extremely slow (albeit intermittently) today?

If you enjoyed reading about "Speed issues today?" here in TheHighRoad.org archive, you'll LOVE our community. Come join TheHighRoad.org today for the full version!
Derek Zeanah
October 17, 2012, 01:42 PM
Yep. I did that.

Basically: Every morning THR goes down for ~ half an hour for backups.
This is annoying.
Backups are important.
I'm trying to switch over to a new backup system that only makes the server unavailable for seconds/minutes.
Doing do required me to reboot the system, and may require me to reboot the database server later today.
Every time we reboot, the database needs to rebuild its query cache from scratch. So it's slower than normal for an hour or so while it has to look up everything from scratch rather than just remembering it.

Hope that makes sense.

Rail Driver
October 17, 2012, 02:44 PM
Makes perfect sense. What are the chances of changing to an offsite redundant server system that switches on when the main server is backing things up to avoid the slowdowns (and adds a third layer of backup in the event that the main server and the backup server both lose power)?

Derek Zeanah
October 17, 2012, 03:13 PM
Makes perfect sense. What are the chances of changing to an offsite redundant server system that switches on when the main server is backing things up to avoid the slowdowns (and adds a third layer of backup in the event that the main server and the backup server both lose power)?
The problem is database synchronization. When the database is being backed up, it needs to be locked, and since vBulletin writes all kinds of data (like the last time you visit) things lock up. We could run a master/slave database layout instead and back up from the slave to do away with the outages, but then when MySQL drops synchronization (and it will) then the backups will be missing all the newer data until the issue is resolved. Which brings up issues with monitoring two databases, and datebase defragmentation (a monthly process that takes a bit over an hour now) will double or triple in length, so there are some other costs.

I could change the database structure to allow things to keep running while backing up, but I'll need to triple the memory on the database server in order to cache things the way I'd prefer to (which will require a migration), and the conversion itself could conceivably mean that THR needs to be down for a day -- longer if we run into issues that require us to move back.

I'm doing what I think makes the most sense. :)

horsemen61
October 17, 2012, 03:15 PM
Its working just fine for me thank you for making this happen:D.

Rail Driver
October 17, 2012, 03:18 PM
Quote:

Makes perfect sense. What are the chances of changing to an offsite redundant server system that switches on when the main server is backing things up to avoid the slowdowns (and adds a third layer of backup in the event that the main server and the backup server both lose power)?

The problem is database synchronization. When the database is being backed up, it needs to be locked, and since vBulletin writes all kinds of data (like the last time you visit) things lock up. We could run a master/slave database layout instead and back up from the slave to do away with the outages, but then when MySQL drops synchronization (and it will) then the backups will be missing all the newer data until the issue is resolved. Which brings up issues with monitoring two databases, and datebase defragmentation (a monthly process that takes a bit over an hour now) will double or triple in length, so there are some other costs.

I could change the database structure to allow things to keep running while backing up, but I'll need to triple the memory on the database server in order to cache things the way I'd prefer to (which will require a migration), and the conversion itself could conceivably mean that THR needs to be down for a day -- longer if we run into issues that require us to move back.

I'm doing what I think makes the most sense.

no arguments there... the logistics with a site this size can present some major issues, especially where db sync is concerned. I am not as experienced with mysql as I would like to be, so looking into it is a learning experience for me. Either way, great job and keep it up!


Posted from Thehighroad.org App for Android

If you enjoyed reading about "Speed issues today?" here in TheHighRoad.org archive, you'll LOVE our community. Come join TheHighRoad.org today for the full version!