Sota news may 2012

In reply to M1MAJ:

The only people who have problems are those people are some kind of programmer. All non-programmers who come up against the requirements as they exist accept them and move on. There is a lesson in this.

At no point can I recall anyone asking why you have to enter a CSV in QSO date/time order but can enter QSO via the web in any order.

There’s the rub as a certain Hamlet was reported to say. :wink:

Andy
MM0FMF

In reply to MM0FMF:

The only people who have problems are those people are some kind of
programmer. All non-programmers who come up against the requirements
as they exist accept them and move on. There is a lesson in this.

I don’t have a problem with “requirements”. I do have a problem when the actual requirements differ from the documented requirements, which it seems to me is the case here.

In reply to MM0FMF:

In reply to G8TMV:

The real problem is that the system doesn’t have any way of
defining an
activation in terms of which QSOs are part of it.

Except it does.

At the moment it just uses the date and counts QSOs.

Except it doesn’t.

Please do continue as I’m enjoying this.

That is exactly the attitude that is likely to make me walk away from SOTA.

Any system which requires falsification of the data to make it work is broken.

Now, how about a reasoned explanation please.

Colin G8TMV

The reasoned explanation is thus:

The system is the best we have come up with so far.

The system needs to be able to:

  • Have both manual and CSV upload facilities for logs
  • Be able to deal with different activations of the same summit on different dates
  • Be able to deal with single activations of a summit that crosses midnight UTC

The system does not need to be able to provide a 100% accurate confirmation * system for cross-checking contacts. This is an extra service that gives users a head start in checking the accuracy of their entered data, but does not guarantee to be definitive or complete.

This is the only thing that ‘fails’ on our current system. If anyone can come up with the solution to this issue, and the time, experience and tenacity to put it in place, including applying it to the 1.3 million QSOs already in the Database, then please step forward.

No-one on the current MT has yet found a “solution” to this “problem” - a problem that only exists if you wish to take an exceptionally purist view of things. Well, not a solution that would be actually worth spending the time implementing it for the disproportionately minute improvement it would cause anyway.

b) Two activations of a summit on the same day (and yes I know you wouldn’t get points for the 2nd one), but I expect Tom would like the count of his Cloud activations to be correct ;).

The rules clearly state that you can only activate a summit once per day. Many a time I have made some QSOs, descended, then gone back up the same summit the same day and made some more. But all those QSOs get entered together as one activation, and rightly so.

Any system which requires falsification of the data to make it work is broken.

The date entered, whether it be via the manual web entry, or the CSV upload, is the date of the activation, which is defined as the UTC date of the first QSO. Therefore, no data is falsified. What is falsified about entering a log for 0002z as part of an activation that started ten minutes earlier on the date you declared?

Tom M1EYP

In reply to G8TMV:

Any system which requires falsification of the data to make it work is broken.

Except it doesn’t and isn’t.

The CSV upload allows multiple activations to be uploaded in one go. In order to determine where one activation ends and another starts requires some key events in the data to act as triggers. These events are currently when the date changes or when the summit changes plus the obvious termination case of no more data. (A good programmer can make any source code look like FORTRAN IV and any IO look like a deck of cards if he tries hard enough.) So the upload will chew through lines from the CSV file adding them to the current activation until 1 of those 3 events is spotted. Assuming it’s not the terminal EOF case then a new activation entry commences when the date changes or summit changes. This process continues till EOF. You could have as many activations in the CSV file until you hit Windows Server 2008 file size limit or till the database activation count hits (4 billion or so). The triggers could have been special lines (much like special cards in a JCL/360 deck) but that would be harder to use with some log extracter. Not impossible but probably more error prone as the line could be inserted in the wrong place.

Now you can argue whether those conditions are valid if you want… should a CSV file contain just a single activation, should date change be end of activation, should there be some extra info marking the start and end of set of QSOs making up an activation. They’re all good questions and we can use hindsight to see if the decisions made in 2002 were the right ones. I don’t think anyone seriously expected anyone to do a multi-day activation in 2002. But they happen without UTC date issues. Judging by the continual load, the GB/month of data and the fact there are ~1.4 million activator QSOs logged they were probably not that bad decisions though. Allowing multiple activations in the file is probably the right thing to do as it allows logging programes to dump out multiple QSOs from your real log for import.

So the reason you can’t easily enter an activation when it crosses the UTC date change is that is a trigger to mark the end of an activation in a CSV file. I could change that. It would make things easier for some people who extract their logs to SOTA format from ADIF or whatever. But changing how a CSV file works is going to bug as many people as it pleases if the change is not selectable. It’s not just changing how the CSV file works but has additional requirements either of file format or UI changes.

Working on things like this delays introduction of things which others may want more than you want. Just looking at how the uploader works and writing a reply for you, someone whom is unaffected by the subject at hand, has stopped me working on stuff that Barry wants. The fix for people exporting logs where their activation does cross the date change is no more than a few moments work with a spreadsheet. Clickety-click and it’s done and they can upload. There’s nothing terminal about the issue. Yes, annoying but not as important as some of the other things that need fixing.

So there you go, broken? I don’t think so.

Andy
MM0FMF

In reply to MM0FMF:

So there you go, broken? I don’t think so.

What is broken is that the requirements contradict the documentation.

When this question comes up (and it has come up before), there’s this slightly reluctant revelation that the date in QSO records is expected to be the date that the activation started. That is slightly surprising, but we live in an imperfect world and if that is the spec so be it.

But what if somebody does read the spec? It’s in:

http://www.sotadata.org.uk/ActivatorCSVInfo.htm

This states, quite clearly, no ambiguity, that the “date” column of CSV or TSV records should contain “The date of the QSO.” Don’t take my word for it - go read it. There’s no mention of having to fudge it if an activation crosses a UTC date boundary.

There’s been a big change since 2002. Back then, SOTA was UK-centric. Activations crossing midnight were rare, and when they did happen, would be made by “serious” activators likely to get many QSOs before and after midnight. This “edge case” wasn’t really a problem.

Ten years on, SOTA spans the planet. Activations crossing the UTC day boundary happen in daylight hours and may be expected to become commonplace. They may also be activations that get relatively few contacts. The “two before and two after” scenario which triggered this latest discussion is not at all unlikely. If such an activation is submitted using a CSV file in accordance with the published specification, it will not score. I’d say that was broken.

I venture to suggest that circumstances in which you would actually want a sequence of QSOs crossing a date boundary to count as two separate activations are much rarer. It only affects the score if it crosses a YEAR boundary. Only a small handful of people care about the number of activations of a summit recorded against their names.

My view, for what it’s worth, is that the code should favour the common case of ordinary people doing ordinary activations, at the possible cost of requiring people with special requirements to do something odd. It seems to me that simply changing the CSV upload code so that it does not start a new activation if the date increments by 1 would achieve precisely that. It would have the added advantage that the written specification would be correct.

I can’t see that it would break anything except the obscure case of people wanting to record separate activations of the same summit on consecutive days. These people could simply be advised to upload the two activations separately.

You’re right that this doesn’t affect me personally. And yes, I’m looking at it from a programmer’s perspective, a world in which edge cases are what it’s all about. And no, I can’t put the effort into providing a suggested fix because you’re running a “closed source” system. But your helpful description of the CSV upload process suggests to me that this issue is really rather simple to fix.

In reply to MM0FMF:
I reckon it should be left the way that it is. Sure, if an activation goes over midnight, there will be a need to edit a CSV file to wind back the post midnight dates so they stay in the same activation. However, at least at the moment we can clearly delineate between two activations.

It could happen on the turn of the year to get an activation in early Jan 1 local time here, which is still before midnight UTC, and activate a summit for two rounds of points because the two activations fall in different UTC years.

Regards,
Wayne VK3WAM