[J. Nicolaysen]

Hi,

I'm getting this message somewhat frequently, whether I type myArmoury.com or I click on a link to a subforum. If I reload or click on a different subforum, sometimes it works out. But it's a hiccup. What's up?

[Mark Moore]

I keep getting that also, but mostly when I switch forums or threads. .....McM

[Nathan Robinson]

This is caused by too many users / too much traffic on our old server.

We've had the same server for many, many years. It's old. It's tired. I can't afford to upgrade it.

[Nathan Robinson]

I've spent 80+ hours over the last week or so going through much of this site's code, trying to optimize many of the things causing some of these problems. This codebase is old and tired and so doing this type of work is a big pain in the butt. I've changed many things, but at this point it's really unclear how much it's going to affect things. I've got a bunch of logs going so I can monitor things and see how it goes.

Ideally, I'd put this site on a brand new, shiny server, but I don't have the money or the time to do that so it's going to have to live on this box until it dies. We'll consider this recent work to be some much-needed medicine to (hopefully) prolong that life.

[Christian Short]

Nathan I'd be happy to contribute to moving to a new server. If that's feasible

[Stephen Curtin]

Nathan, thank you for your efforts. Out of curiosity how much would a new server cost? I know you don't like the idea of asking for donations Nathan, but perhaps some fundraising could help. I'd be happy to make a small contribution, and I'm sure that I'm not the only one.

[Nathan Robinson]

An upgrade would be a couple thousand dollars. The big deal for me would be the the increase in monthly costs. I'm dialed into a fairly sweet deal now that gives me a lot of bandwidth without paying for overages. I wouldn't get that with any new deal and so it would be quite costly.

That means that my big worry would be upgrading the box itself and then getting stuck with a large monthly hosting bill. I just can't take the risk at the moment.

[J. Nicolaysen]

Thanks for the response. I don't envy the work and hard choices, but as always we are all grateful.

[Stephen Curtin]

I see. So it isn't as simple as a one time payment to upgrade. Well thanks again for the work you put in to keep this place going.

[Nathan Robinson]

I've made a ton of changes to the site lately.

Things appear to me to be improved. Feel free to give feedback of course.

I'll post an update about the changes soon.

[Stephen Curtin]

Well I haven't had the "critical error" message happen in a few days.

[Mark Moore]

I have not had it either. But, sometimes it takes a while to navigate. That could just be my slow crappy computer though. I need to do some house cleaning. ....McM

[Nathan Robinson]

Okay, so... an update:

I've put in a ton of work to optimize a lot of things that have been sort-of working for years but don't work so great with all the people viewing the site these days. When I built this site, I really had no idea how to build for scale. I wasn't expecting it to have 150+ people viewing all the time or what it would really mean for performance.

I had... really no idea.

So over the last couple weeks I went ahead and rewrote major parts of the site's infrastructure from the ground up. I also optimized a lot of the really poorest performing pages/functions and got those things working 10x better.

Much of what was slowing the site down and causing errors was the ongoing maintenance and cleanup stuff that has to happen all the time. Instead of looking for this stuff and doing clean-up along the way, the site now skips it and only does these things once every eight (8) hours or so depending on how busy the server is.

This means that every eight hours (not on the hour) we'll have 2-4 minutes of slowness and perhaps some database connectivity issues depending on the traffic the server is experiencing at the time.

I've had a lot of logs and other statistical stuff going on the last couple weeks, enabling me to run some reports to see if any of this made a difference. The server is showing a 100-300% improvement for its "load average" (a measure of how hard the server is working) and other metrics... so that's pretty good!

Along with that, I have been doing really late nights to build a whole new thing that I'll be launching tomorrow. Stay tuned.

[Nathan Robinson]

So it's been 4+ months with the new additions and as far as I can tell by logs and my own experience using the site, the errors are greatly reduced. We still get them while the maintenance cycle is happening (3 times a day, lasting 2-4 minutes max) but otherwise the site seems very stable. i hope this has been the experience of others as well.

[Mark Moore]

Nathan, I'm here on the site at all hours of the day and night, and I haven't seen that 'critical error' thing even once in a loooong time. Other than sometimes being a bit slow to load, no problems. Good job...thank you. .....McM