This announcement is to let database users know of a proposed update to the database. With advanced warning nobody will get caught out by the changes!
The change will only affect people who upload logs using the CSV upload facility. If you enter your logs through the webpage then you will not see any changes.
I intend to tighten the requirements for CSV files being uploaded as a number of users are uploading malformed data. The effect is that the filters on the log viewing pages are unable to work correctly. Over the past months I have run a number of cleanup scripts on the database to fix common errors but I’m getting fed up fixing the same problems again and again. It’s not really a user problem per se, it can’t really be a user error if the database accepts the submission. It’s more a design decision that now needs reviewing in the light of accumulated observations.
The biggest source of malformed data is the Band field. The webpage entry forces the user to pick from a list of descriptors such as 7MHz, 144MHz, 21MHz etc. I do see users entering 20m, 40m or 10m etc. The problem is that that trying to view all contacts on 14MHz will not include contacts logged as 20m etc. as the software looks for “14MHz” in the data. Worse is when 40m is entered as 40MHz, or 15m as 15MHz.
The new parser will reject obvious errors like 40MHz etc. but I haven’t decided whether it will allow, for example, 40m in the input and convert it to 7MHz or if it will only allow the same fields as the webpage entry. I feel that the software should not try to second guess what the user really means and should only allow valid data and should reject anything else. But I’m happy to listen to other user’s views before I change anything especially as I have yet to write any parser updates.
Yes, there are other updates/bug fixes/features the database needs, some of which may also roll out with this update so you don’t need to ask for the confirmation * to be turned on again!
I would favour strict parsing - provided the error messages are informative. The “microwaves” band option is redundant and could do with being removed IMHO.
To my mind strict parsing of the band is ok, as long as the exact frequency is accepted. E.g. 144.325 MHz or 145.575 MHz are accepted as 144 MHz like in the present state. I can easily export the Band (2m) and the exact frequency in my log program HamOffice, but not the start frequency of the band.
The “microwaves” band option is redundant and could do with being removed IMHO.
I totally agree Richard. It would also be good to see on which bands the VLF contacts have been made by splitting that section into frequency based records. A personal request from me would be for 1240MHz to be redesignated 1296MHz as this is more appropriate for the majority of contacts made on the band - it does throw me a bit each time I see it. Logically 433MHz should be 432MHz, but I’m not quite at the level of OCD where I need to ask for that to be changed!
I believe that 432MHz (not 433MHz) is the band accepted by the other parts of the system.
1240MHz also confuses me when ever I put in a 23cm entry.
My own SOTA CSV Editor silently makes the following changes when it opens a CSV file:
3MHz becomes 3.5MHz
145MHz becomes 144MHz
430MHz and 433MHz become 432MHz
1296MHz and 1297MHz become 1240MHz
It does NOT (yet) handle bands entered as ‘40m’, ‘20m’ etc.
Using Frequency bands to replace ‘VLF’ or ‘MICROWAVE’ may be useful for new entries made into the database, however those contacts already in the database using the old format need to either be allowed to stay as they are or be updated by the user.
Andy,
Please let us know in advance what changes are decided upon, so those of us that write or use our own software can update that ready for the change of the database system.
Without getting involved in the technicalities of parsing (which I know nothing about) I’m sure Alain F6ENO will have read Andy’s message. Many of us rely on Alain’s excellent ADI2SOTA program to easily convert ADIF files from our logging programs into csv format which I believe should still work with the new arrangements when they are embodied into the database software.
While I am writing Andy, and this has always been the case, is there any way that the database could be made to accept multiple contacts with the same activator/summit on different bands as part of a multiple csv upload. For instance if I work GX0OOO/P on the same summit on say 160/80/40m the csv upload will only import the first band contact into the database. The others then have to be inputted manually.
73 and thanks for all your work in keeping the system running and also improving it,
Thanks for the comments so far. This is exactly the kind of information I want. I’ll be requesting logs from people so I can test the changes with real world data when I’ve got something to test.
Thanks for the comments so far. This is exactly the kind of
information I want.
Ooo does that mean we can remind you about same day activations showing in the correct order? They should show in order of time of 1st contact, but they actually show in order of entry to the Database, within the same day.
This affects the lists of all-time activations for each summit on the Database, and the person credited with the 1st activation on SOTAwatch (presumably read from that list).
Would be nice - as it would stop my ear getting bent by the REAL 1st activator of Ailsa Craig GM/SS-246!
Ooo does that mean we can remind you about same day activations showing in the correct order? They should show in order of time of 1st contact, but they actually show in order of entry to the Database, within the same day.
This also applies to the last activation. On one occcasion I was the last to activate a summit, cronologically, but a previous actiavtor entered his data after me and he was shown as last.
No! Well yes you can remind me but I wont be playing with the code that does that this time. Anyway those features are not dependant on the text in the Band column of a CSV file being uploaded.
Of course that other issue is still an issue - enter several activations carried out on a single day into the database and they get jumbled up when you look at your activator record. Thankfully I keep my own database which includes summit order.
I’m still wondering who uses 1240MHz for SOTA and as to why it came to be the frequency representing 23cms, especially as SOTA was originally a UK based scheme.
I’m still wondering who uses 1240MHz for SOTA and as to why it came to
be the frequency representing 23cms, especially as SOTA was originally
a UK based scheme.
Thank you in advance for your work. I think you should just specify the way you want frequency entered but because I am still keeping up my excel log that John G3WGV gave me in 2002, if possible I would like the option of entering ‘1.832’ in the frequency column and it being converted to an acceptable form (1.8MHz) automatically.
Sometimes I get one letter of one callsign wrong in a list of say 50 QSO’s. I would like the facility to edit that one entry. Currently, I pull off that summit’s activator log into excel, cancel it from the database, rebuild it by adding all the ‘absent’ columns and reload it as a TSV. I have a template to help me but it’s still a 30 minute job. Unfortunately somewhere along the line a ‘z’ gets added to every time in the time column but the database will not re-accept it. I then spend time removing it from each time box using cursor, backspace & return wiping out a few times along the way.
The last problem is as mentioned. The database takes no account of the time you were on a summit in a multi-summit day and regularly scrambles the order making me look like I’m in training for 2012! HI.
Finally, it may just be me who is ignorant of the nuances of the database - that is there may be better ways of using it that I don’t know about. Bottom line is: If none of these issues are fixed I am grateful for the database as it is. It’s run by volunteers after all and generally it’s an excellent service which is much appreciated.
rebuild it by adding all the ‘absent’ columns and reload it
as a TSV. I have a template to help me but it’s still a 30 minute
job.
What ‘absent’ Columns ?? I have done this a couple of times - originally using a text editor, but later using my own SOTA CSV Editor) and not had any ‘absent’ columns.
From a usability perspective the database dropdown menu system could do with some rationalisation. The Association Lists are inconsistent and have now grown very long. They could perhaps be rationalised by sorting into Continents to make access a little easier? Other sub divisions could be USA, Canada and UK, each of which consist of several Associations.
That sounds like a good idea to me, Richard. I would add that since radio hams think in terms of prefixes the drop down Association menus should be in prefix order at the Association level.
The lists could do with a quick polish. One is ordered numerically by AssociationID (an internal database concept), others are ordered alphabetically by Association name and others ordered alphabetically by Association code. Changing them all to use the same ordering would be easy and a start to something improved. I think by Association code, i.e. callsign prefix is the best choice.
In reply to G4YSS:
Editing on line is fraught with errors and I’d have to write a load of code to handle it. Downloading the log, deleting an activation and reuploading shifts the burden onto other programmers. It also means the code runs on your machine rather than on the server where I would have to ensure multiple copies can run in parallel. The problem is that you either download your entire log or nothing and then have to remove many entries from the downloaded log. Probably the fix is to be able to download the log for just 1 activation in a form the uploader will accept. Then editing becomes easy, just add extra entries or correct the typos.
In reply to ALL:
OK, I know that Stuart G0LGS has a log editor and Alain F6ENO has ADIF conversion tools. Are there any other database tools / tool authors I have forgotten about or don’t know about who need to be appraised of any changes?
EDIT:
And I suppose I need to put some nonsense on the login page telling people that logging in will put a cookie on their computer so as to comply with the new EU law.
