Database updates are due for the CT3, VK1 and VK3 associations. In addition, the new W0N (Nebraska) association will be added to the Database.
Normally, I would do these ahead of midnight on the day before the updates are effective (ie this evening), but I am very tired after staff football. This has been compounded by having occasion to share a good bottle of champagne with my XYL Marianne.
I have decided to process the Database updates sometime over the weekend, with a bit more energy and a clear head. Apologies for any inconvenience, and thank you for your understanding!
OK, I believe all updates/uploads are now successfully completed. If you have activated or chased any of the new VK1, VK3, CT3 or W0N summits today, you should now be able to enter your logs.
Not a single person has contacted us asking why they can’t do so, so thank you for your patience everyone!
I have tried to look up the changes on the SOTA Summits listing for VK3 but can’t see them, even after hitting “F5”. I also tried to start a Chaser log entry but again the summit list has not changed.
Yes that works but why is SOTAwatch2, SUMMITS, VK3 still giving the old list?
Because SOTAwatch is not the database. It’s an additional resource that provides forum facilities, spotting facilities and alternate views onto the database. It lags the database by a variable amount of time based from when we decide that last database update was kosher to notification that an update of SOTAwatch can take place and Jon, who administers it, acting on that update.
The database update was delayed slightly as Tom was busy and Jon has not been asked to act yet. Like Barry, the awards manager, Jon is self-employed. The rest of the MT are either retired or still working. The rest of us try not to rattle the cages of those who are self-employed until we are sure we need their work. Requesting an update on SOTAwatch only to find we have snookered ourselves and made extra work for Jon is something we try hard to avoid. The last update (VK1, VK3, CT3 + W0N) was not large, not like when we have done 3500+ summit updates. Even though we check the data with various tools, we visually check it and sample summits, mistakes get in. It looks good so an update will be done.
And why can’t I enter a new summit from the relevant “submit” form.
Thanks for explaining that there are effectively multiple data bases. I had made an assumption that there was just THE DATA BASE (and many retired superseded ones now in limbo) and all other resources and things like SOTAwatch2 and the form used when submitting an activation were linked to THE DATA BASE by some cunning bit of code so that when THE DATA BASE was updated every other resource went to the updated version.
I quite understand you want to see if the update is OK before retiring the older version and adjusting the linkages. Fair enough.
Please do not take this as a criticism - I am self employed, nominally part time, so I am well aware of the time issues and am well aware through personal experience of the joys of voluntary work.
There was an invitation to give feedback on the update and that’s my intention.
I look forward to being able to use my “normal” methods of interrogating the data whenever it gets done.
Many thanks for completing the links 9or whatever needed to be done). It all works very well no matter which path (electronic) to the summits I take.
Your efforts are much appreciated. I like the feature where discontinued summits are still able to be logged if activated before the revision date and you cannot enter an activation for those peaks if it took place after.
In reply to SQ9OZH:
Log in into the database.
Click Summits -> List of all Summits.
You will see 9H/GO as preset region - doesn’t mind.
Scroll down to bottom to see "NOTE : You can download a CSV file containing a list of all summits by clicking here"
By clicking “here” you get the complete CSV-list of all summits.
In reply to DB7MM:
I am asking about QSOs database, not summits.
Now, when we need some custom aggregation over QSOs database, we depend on Andy, I think.
My idea is to expose the SQOs database for public access and leave initiative for SOTA community.
I’m not going to answer for Andy, but a few obstacles to public access to the QSOs database spring immediately to mind:
activators/chasers have to login to upload logs, view logs, and to gain access to their QSOs data. This is done because these are private data, not for public consumption, and access is granted only on login;
the possibility of cheating by unscrupulous activators/chasers is thereby greatly lessened;
the database is in any case HUGE - a CSV or any other format would be far too large (on the order of 100’s of megabytes): either to generate or to download on a regular basis.
If you have ideas for QSO data mining, you should contact Andy with your suggestions.
ad 1)
No problem for me, if downloading QSOs data will be available only for BASIC HTTP authenticated user (or any other auth. methods).
Maybe “public” is wrong description - available for any of SOTA community member will be better.
ad 2)
I don’t get it. In fact, all QSOs in SOTA database are available on sotadata.org.uk.
ad 3)
There is about 2.5mln QSOs now. Each QSO in CVS take less than 100bytes. So whole RAW QSOs database takes about 250MB. I don’t think that will be a problem for one-pass mining, per day.
ad ideas)
I have a lot of ideas.
by example:
filter by summit association for any result tables
sort by activation count descending, grouped by summit, band, mode
I think, there is more and more other ideas that can be implemented by community, without commitment from core sotadata.org.uk database.
basic HTTP authentication is a dinosaur, and is unencrypted unless conducted via HTTPS - I hope you would take better care than that of user’s credentials;
I can’t imagine this to be the case - when I login to “the database” (i.e. the QSOs database), I can only see my own results. Only summaries of others’ activities are available, not individual QSOs. I can’t see your QSOs, nor those of anybody else. Unless I’m very much mistaken, your statement that " all QSOs in SOTA database are available on sotadata.org.uk " is incorrect or at the least misleading.
you’re not taking into account the management aspects of such a quantity of data being made available on a regular basis. It takes considerable time for a database to create such a file as you envisage, with no guarantees that the db is available for the entire operation, even if it were an automated process. If it has to be done manually (perfectly possible/probable given the “amateur” [in the best sense of the word] status of those charged with maintaining the db), then that’s another possible obstacle to access to the data. Then again, given the slow rates at which the SOTA servers work to serve out files, and the fact that the servers go offline at strange times, means that the file transfer will very likely fail at some point. I have troubles enough with downloading the 8 or 9 MB daily dump of summits data - the problems would become much greater with a download of 250MB or greater.
In any case, the SOTA db is not the property of the community - it belongs to the MT, who can administrate it how they wish. And long may that remain so…
Many of us have a lot of great ideas, and yours are undoubtedly good. So, talk to Andy and plead with him to get some of them implemented. It hardly needs to be said, however, that a single voice won’t carry much weight. So get others involved in the process: state clearly what you would like to see, and see if others are interested. Teamwork!
Whoops Gerald beat me to it just as I was about to post the same, complete with Rob’s last QSO details!
18/Jan/2014 13:55 DM1CM 2E0YYY G/CE-004 Bardon Hill 14MHz SSB 1 1 s59 r59
Never mind - a quick edit …
Rob, as the database access issue isn’t going to be resolved anytime soon (I guess), how about you making it so that activators can upload their own activation CSVs to SMP in a similar way to GPX files.
This would start to populate the rather sparse activation data that is already there.
I’d be happy to upload mine as I find the plotting information useful!
As we’ve discovered I’ve nothing to hide as the details are already here for all and sundry to see on Sotawatch and if people are that desperate they can always query my online logbook too.
I’d even put up with the much hated Captcha!
On the subject of captchas, I was trying to book concert tickets the other day and gave up after 1/2 hour as I couldn’t get past the indecipherable captcha.
Luckily my sister helped me and got past it in only 25 minutes
I hate these things with a vengeance…
ad)"
In any case, the SOTA db is not the property of the community - it belongs to the MT, who can administrate it how they wish. And long may that remain so…
"
You are absolutely right. I’m just asking.
As long as whole database information is available through sotadata.org.uk this is only technical case, how MT can share this information with more “computer” friendly form.
Rob, as the database access issue isn’t going to be resolved anytime soon
(I guess)
Sadly you guess wrong!
After presenting a talk at my radio club this week, I get some DB time and the next database update for SWL logging will be rolled out. It’s complete as I type, but it’s on my PC not the server.
Then the next task is providing integration between Rob’s mapping site and the DB. Currently we do this by Rob fetching a modified version of summitslist.csv every day. He fetches it, twiddles with it, fixed bits of it and then imports and uses that data for the map site. The mapping of activations was tested by me exporting a dump of 3 months activations and Rob importing that data statically. These methods have worked and demonstrated that it’s possible for the mapping site to be a viable and valuable resource for SOTA users using snapshots of the data.
The next stage is to expose a real DB that Rob’s mapping site will use. For security reasons, nobody gets any access to the DB directly, access will be via a snapshot DB. i.e. every so many hours, data from the real DB will be exported to the snapshot DB. Instead of Rob importing huge data sets into his site, he’ll be able to run queries directly. For performance reasons, it will be worthwhile maintaining some of the data on Rob’s site, i.e. summit lists as they change at most monthly. Chase/Activation/S2S data can fetched when requested and cached if it makes sense.
When we’ve had that running for a while, we’ll look at the impact that the internal export and re-import and the extra querying and data transfer has on the load figures. It may be we need to spend a bit more money and move up the hosting ladder for greater guaranteed performance rather than running a shared server. Or maybe the effect is lost in the noise. I don’t know. What we wont do is anything that jeopardises the security and integrity of the data or the performance we have now.
Only when we have hard figures will we look into further requests for direct access. It’s never going to be a free for all with unlimited access but requests for access that we feel benefits SOTA users will be considered. We’re not there yet.
Andy, MM0FMF
Database manager (should be mangler based on my history!)
