To N7LP:
Please see Andy’s last-but-one message. We need to keep the Database home page and the log-in page the same while the problem is investigated and fixed. Otherwise, there is the danger that the problem will seem to go away - so we then can’t investigate it, and furthermore, we could end up believing it is fixed when it isn’t.
Thanks Tom. That’s the point exactly, I need to see what effect some remediation has had before I change anything.
Interesting is that there were only 12524 attempts to logon yesterday. Now the average number of logons on a weekday is around 3400 and about 4800 on Saturday and Sunday.
Why does not every one simply NOT log into the data base for next 12hrs and let the people do there Job. and look again after 9am tomorrow see if its fixed or not.
Simples, up loading down loading can wait another 12hrs surely or they could just shut the whole system down and fix it that way
At least the alerts section is ok.
Karl
Hi Karl, the spots and alerts are on a different system - the sotawatch server, not the SOTA database server.
FOR CLARITY - this problem does not affect the SOTAWATCH Alerts and spotting systems.
Ed.
cool may be just as well thanks ED
Karl
One aspect: Before you posted your request not to log on about the average number of users tried to upload or view logs. As they were logged off after a very short period of time they had to log in several times just to get their job done.
Of course not to forget all the users trying out whether the issue is fixed already and those downloading their logs fearing data loss.
73 de Michael, DB7MM
The fact that a number of users have logged in and downloaded their logs when asked not to log in has made the situation worse for others. I can see who has done that. Their selfishness will not go ‘unrewarded’ because they have made things worse and secondly by not expecting the MT to make backups of the data they seem to be suggesting we don’t take running the various web services seriously.
Perhaps the above explanation 24 hours ago would have stopped the lemmings panicking.
That is a prime example of 20:20 hindsight! 24 hours ago all we knew was that the database was acting up.
Brian
Thanks for the explanation. I thought it was my computer.
Many SOTA enthusiasts don’t read this reflector, that’s a fact, so they are probably ignorant of the situation. Also English may not be the users first language.
73
Phil
I misunderstood Andy’s post. I was not aware that cosmetic changes would change the system functionality. My apologies.
That’s OK Lee, it’s a sensible suggestion and yes it is a cosmetic change so it shouldn’t break things. But it is a change and I don’t want to change things just yet.
You’re quite right Phil. Looking at the access logs recently, most of those still trying to log in are not native English speakers.
Run hamsters… run… bet some serious head scratching is going on… oh how I have been there… in my case 12000 people not able to log on to their PCs… good luck techs…
Sure, I’m still getting many queries from Japan SOTA lovers. Reflector is not the best place to annonce such important notice. I have announeced not to log on till next annouce in Japan ML. But not the ML not cover all.
Hi Andy thanks for the heads up no doubt it wont take long to get it sorted
after all it must get a pounding at times
the logs can wait till when ever
cheers 73
Rod VK2TWR
STATUS UPDATE.
It’s still broken and I am in contact with the hosting company.
The database is on a separate machine to the web server and it is the web server were we are seeing problems. The data is backed up every day and also transferred offs-site so we have 2 copies of data for the previous 24hrs available at for immediate restore. Offsite backups are maintained for the last 30 days and then monthly and yearly archives maintained. So there is less need to worry about making your own backups now. When we do get things sorted, it maybe we need to rollback to 10-oct-2015 but I don’t think that will be necessary. I’m simply pointing out it may be needed.
We were suffering from a flood of requests from Bing’s Bingbot which was thought to be happening fast enough to destabilise the web server. This flood has now been mitigated and Bing now spiders the web site about once every 12mins which is fine. However, this hasn’t improved anything yet. That was one of the reasons I didn’t want to change anything on our side as I don’t believe it’s anything we’re doing.
Further details about some issues I see as admin that you, the users, don’t see has been added to the ticket with the hosting company and that may get them to see what is broken. But for the time being the DB is still broken so don’t log in.
I’m sure this is an issue with the host and not anything to do with us as the last update was 27-sep-2015 and everything worked like a dream till 11-oct-2015. The codebase was running without errors for over 2 weeks.
I am seeing fewer attempts to login as the message “don’t” is getting about. Sadly one user emailed me with a really helpful error report “I can’t log in and I get some goofy message” which he didn’t include. Yes, I am bloody psychic and will know which error message he got without him sending it to me. I replied with a link to this thread. What do I get? Another mail from him telling me it still wasn’t fixed and he still couldn’t enter his logs.
Well that was news to me 
Checking the access log he’d tried to login and submit his logs 6 or so times since being told not to do that. I accept Phil G4OBK’s comment not everyone reads the reflector and not everyone reads English but this guy does speak English as a first language and was specifically pointed at information about the problem and he decided the request to not log in didn’t apply to him. Well I’m quite prepared to apply some sanctions to him when everything is fixed because that kind of stupidity is just completely out of order.
You’ll need to hang fire a bit longer. I wish it was fixed because we are getting many requests for help and worse, this week I am mad busy with customer problems at my day job and we’re decorating at home. So I’m just a little stressed out right now.
Andy, MM0FMF
Database manager
The world won’t stop spinning if the SOTA Database is down for a few days. I wouldn’t even bother opening the emails if I were you Andy.
I know that Tom. But the feeling of duty of care to the SOTA users is strong. Whenever people meet me at rallies, they come up and thank me for this work and offer to buy me beer etc. I don’t want to let these people down.
Two days into the problem, I see people are still being being caught out by this temporary inability of the system (and I recognize that it’s a problem at the hosting company) to facilitate logins - especially those who never thought of reading the Reflector to see whether some notice or other might, or might not, have been posted there!
For the life of me, I can’t understand why the login button can’t be overridden with a temporary piece of code which just pops up an alert to say “Server down, logins not possible at the moment. Please refer to the SOTA Reflector for further details” and nothing more. Five minutes work, tops. In this way, the people are being informed at source that there is a problem, and they don’t have to look anywhere else. Plus, no more login attempts are possible until the problem at the server is solved, and hence nobody needs to check the logs to see who is trying to login.
Or am I missing something really profound here?