Database updates

In reply to G4OIG:

Tomek, Gerald and Pete: I stand corrected! Couldn’t ever find how to do that… So the login is only to guarantee that UPLOADS of data are provided by the correct user? And therefore, a login should not be necessary just to VIEW data? Not sure on this point. Pete: hands off my beloved captcha - it took me ages to program the thing 8-P!

Andy: good news on the organisational front there. Just drop me a line when you think the snapshot DB is ready to start testing.


In reply to DM1CM:

The login is used to prevent people without an account from browsing the data. There was a time when you could only see your own data but that was instigated as something to do with RSGB contesting, before my time.

Having a log in stops Yandex, Bing, Yahoo, Google and a bazillion other webcrawlers indexing everything at great expense in wasted CPU time and our transfer bandwith.

Even most of the Chinese webcrawlers now make some attempt to honour robots.txt! For a while when the database app used to see the IP block of Baidu webcrawlers it did a redirect to the PRC Army website. I had to do something, about 40% of the daily bandwidth use was Baidu spidering everything over and over and over and over and over. They can waste their own government’s web bandwith not ours!


In reply to MM0FMF:

OK on the login. Quite agree about the webcrawlers, pests all of them! - it got so bad even on the SMP, that I decided to block them all via my robots.txt which simply instructs them not to index the site, at all. And luckily, they do seem to honour that…


In reply to G4ISJ:

" Rob, as the database access issue isn’t going to be resolved anytime soon (I guess), how about you making it so that activators can upload their own activation CSVs to SMP in a similar way to GPX files. "

Pete, as Andy points out, things are moving along and a snapshot of the QSOs DB will soon be made available to the SMP, so there won’t be a need to upload QSOs data there.

What will be needed, however, is some input from those with an interest in the SMP regarding what kind of reports, or data export options they would like to see in the activations page. As some may be aware, there’s already a stub window in the page (which doesn’t yet do anything) waiting for some inspirational input…

Best regards, Rob

In reply to DM1CM:

Dear Rob,

For export it would be nice to have:

  • A bitmap of the map plot without any of the normal buttons (so better than a screen grab)

  • A kml

I sometimes use GeLog to achieve the latter, using hamqth location information. But it’s a bit fiddly. Can you pull chaser coordinates out of the database? A kml based on hamqth locations and known summit coordinates for S2S QSOs would already be a lot better than can readily be achieved with GeLog.

For pdf output you should start with the map graphic. A summary of QSOs would be great - just a time-ordered list like the database activator’s log page, I’d suggest, with S2S QSOs highlighted. Maybe the export dialogue should have a text field for the activator to add a description to add after the graphic. Something like this perhaps, put together from a screen grab and the database activator’s log:

A means of linking directly to SMP showing the activation would, I suppose, be a mixed blessing…

73, Simon

In reply to G4TJC:


" A bitmap of the map plot without any of the normal buttons (so better than a screen grab) "

  • this is one of the things I was working on during idle moments last month for the tracks page. Such a thing will not be easy to implement, since to date (February 2014) there has existed no native functionality, neither in the browser, nor in browser-side JS, which offers automated screen-grab (of course, you can do this manually using tools in your OS: Win/*nix/Mac). I’m currently investigating software packages which may make automated screen-grab possible, but it’s early days yet, not only in my understanding of the technologies, but also in the development of those technologies themselves. And as things stand right now, it may only be available in Chrome, Firefox and Opera browsers.

An interim (and browser-independent) solution might be to use Google StaticMaps API to generate a clean (controls-free) map area as image (already tested), and then to generate the QSO paths via a spherical- to cylindrical-coordinates conversion, map those point-to-point on the image, and finally to overlay the chaser/S2S icons on the image, which is then passed to a PDF-generator as a PNG. Simples! - and all automated. However, without having yet tried that route, I would say it can’t be guaranteed to look precisely like the browser-based SMP interface. But, based on experience with server-side image generation, I’m guessing it should be pretty good - the one big problem might be in generating those pretty dots and dashes in the QSO lines which, in the browser, are so effortlessly managed by Google Maps API polylines and SVG. Ho hum…

QTH/chaser locations should not present any real obstacles - as you can see, that already works ~90% of the time in the activations test page. Texts, tables, links, PDF files on the fly, etc., are no problem.

The SMP is a work in progress!

HTH, Rob