Database changes

In reply to MM0FMF:

And the winner so far with the longest S2S I’ve seen is PA0SKP

09/Mar/2013,08:12,DL/PA0SKP/P,VK5CZ/P,VK5/SE-017,DM/RP-064,14MHz,CW,15676

15676km!

Interesting.
Is it possible to put in a S2S Distance Roll of Honour to see what we need to chase :slight_smile:

Pete

In reply to G4ISJ:

Not easily Pete.

The database stores stuff that you can’t “magic” out of other data.

So the chaser data stores the fact some user had a QSO with VK5CZ/P when he was on some summit on 09/Mar/2013,08:12 using 14MHz,CW and the callsign DL/PA0SKP/P

We JOIN up the summit with the summits data and find out all the info on the summit VK5CZ/P was on including the lat & long.

We JOIN the chaser data key with the s2s key to find out the summit id for DL/PA0SKP/P.

We JOIN that summit id with the summits to find all the info on the that summit including its lat & long.

We do that for every record in the S2S table for a user and then that nice ASP.NET framework pushes it into a datatable view.

Then for every row in the datatable we run down pulling out lat&long for each summit pair and calculate the Great Circle distance and insert that back into the datatable with this.

public static int GCDistance(Double slat1, Double slng1, Double slat2, Double slng2)
{
// we get bad data in some summits so catch any crap values
try
{
Double R = 6371.0;
Double lat1 = DegreeToRadian( slat1 );
Double lon1 = DegreeToRadian( slng1 );
Double lat2 = DegreeToRadian( slat2 );
Double lon2 = DegreeToRadian( slng2 );

Double dLat = lat2-lat1;
Double dLon = lon2-lon1;

Double a = Math.Sin(dLat/2) * Math.Sin(dLat/2) +
Math.Sin(dLon/2) * Math.Sin(dLon/2) * Math.Cos(lat1) * Math.Cos(lat2);

Double c = c = 2 * Math.Atan2(Math.Sqrt(a), Math.Sqrt(1-a));

return Convert.ToInt32( R * c);
}
// catch crap data with System.DBNull for lat, long etc.
catch (Exception ex)
{
return 0;
}
}

Then render to HTML and push it down the interweb tubes to your PC!

I was surprised how quick it runs on a shared machine. Damn fast these big multi-core Xeons :wink:

So the distances only exists in a transient fashion. The values don’t get saved. It’s not worth the effort as we can evaluate them so quickly.

But that makes “who has the biggest” harder to find.

Andy
MM0FMF

1 Like

It was indeed a special S2S QSO. Ian send me his QSL-card direct. He was also surpise of the S2S DL-VK!!! He was working with 10 watts. I worked also with only 10 watts.
Tks for your nice database.

73 Sake, PA0SKP

1 Like

In reply to MM0FMF:

Andy,

Not sure if I am missing anything re uploading the csv files in the new format but having tried to upload an S2S csv file for the period 2009 to 2012 I always get an error in the date column. All dates are shown red.

here is a part of the file:

V2,OK1CZ/P,OK/KR-068,02/08/10,0832,7MHz,CW,OK1IF/P,OK/PL-023
V2,OK1CZ/P,OK/KR-068,02/08/10,0833,7MHz,CW,OK1FRT/P,OK/KR-010
V2,OK1CZ/P,OK/KR-010,02/08/10,0935,7MHz,CW,UT4FJ/P,UT/CA-156
V2,OK1CZ/P,OK/KR-077,02/08/10,1314,7MHz,CW,Z30A/P,Z3/WM-052

I don’t see any problem in the csv file. Can you pls advise what I am doing wrong?

Thanks
Petr

In reply to OK1CZ:

V2,OK1CZ/P,OK/KR-077,02/08/10,1314,7MHz,CW,Z30A/P,Z3/WM-052

Looks to me like you’re missing the last field?

V2,“mycall”,“mysota-ref”,“date”,“time”,“band”,“mode”,“call”,“sota-ref”,“comment”

73, Rick M0LEP

Edit: Or it might be a bit more complicated than that…
Check this thread too: Hiking in the mountains: tips for beginner hikers - Mountain Day
…and the relevant format info at Sotadata3

In reply to OK1CZ:
I THINK this is just that dates have to have the century in them
DD/MM/YYYY

73
Gerald
MW0WML

In reply to MW0WML:

I tried with both DD/MM/YYYY and DD/MM/YY format and the problem is still there.

Also, last field “comments” is optional.

The only problem is the date field which the software does not like and says error for all 130 lines of the file.
I hope Andy can look into this and help.

73

In reply to M0LEP:

No, it understands a comment can be missing off the end of a record. Any of the following are valid.

V2,“mycall”,“mysota-ref”,“date”,“time”,“band”,“mode”,“call”,“sota-ref”,"comment"
V2,“mycall”,“mysota-ref”,“date”,“time”,“band”,“mode”,“call”,“sota-ref”,
V2,“mycall”,“mysota-ref”,“date”,“time”,“band”,“mode”,“call”,“sota-ref”

n reply to OK1CZ:

Petr, it’s failing because the S2S award starts on 1-Feb-2013 and so the parser rejects any QSO that is an S2S if the date is earlier. If you delete your summit ref then the data uploads.

I took the decision that I could either apply a date limitation to every query made on the database or ensure the S2S data does not exist for dates before the start of the award. I decided to limit the creation of S2S data to after the award started because the number of times I would have to valid and check dates would only be when an S2S record was entered. If I did the check on every S2S query made to the database, I would end up executing a lot more code as people view tables etc. As it stands, you can select all associations and display the honour roll. This requires the database consider 1.35million chaser QSOs.

At today’s usage, the extra execution cost is not much but is observable; I was trying to think ahead and ensure I wasn’t building in limitations to scaling when we have more users and many more QSOs.

I have noticed that the “Facts & Figures” page does not list the number of S2S QSOs, only the total chaser QSOs. I’ll get that fixed with the SWL stuff.

Andy
MM0FMF

1 Like

In reply to MM0FMF:

In reply to G4ISJ:

Not easily Pete.

So the distances only exists in a transient fashion. The values don’t
get saved. It’s not worth the effort as we can evaluate them so
quickly.

Thanks Andy. I expected it might be something like that.

On a slightly related database issue.
I like to plot my S2S and chased summits on a map like this:

To do this I use the summit maidenhead locator.
Whilst this is available directly from Sotawatch for all UK summits, it doesn’t appear in the database for DX summits.
eg

Summit Information for G/SP-013
Gun - 385m, 1 point
Association: England Region: Southern Pennines
Latitude: 53 9 2 N, Longitude: 2 2 46 W
Grid Reference: SJ 970615, QTH Locator: IO83XD

as opposed to

Summit Information for OK/JC-105
Pakostov - 528m, 2 points
Association: Czechia Region: Jihoceský
Latitude: 49 8 23 N, Longitude: 14 22 33 E

However if I browse the summit info from the database (as opposed to Sotawatch) and select “History” the Locator is available, as it is from the mapping project too.

Is it possible that the DX Locator info could be populated on the normal summit detail page just like the UK ones?

It would save me a lot of time searching for them :slight_smile:

(you could also show the winter bonus details which is also only available from the history!)

Pete

In reply to G4ISJ:

Remember database is not SOTAwatch. 2 seperate systems in different parts of the UK. (Long and complex story to explain why.) There’s a long and tedious story as why some data is and isn’t on SOTAwatch as well.

The Oracle is sotadata.org.uk. It’s where SOTAwatch gets it’s data as does the Mapping Project. (Rob, expect some answers tonight!)

I don’t store them Pete, I evaluate the QTH locator on the fly too. It’s these multi core Xeons, Pete, they’re much too fast :slight_smile:

You have the lat & long and QTH loc is only those items expressed in shorthand. I’d post my lat & long to Maidenhead routine the database uses but I’ve just seen a bug in there that even the most inexperienced programmer would be embarrased to make. I’ll fix that tonight when nobody is watching :wink:

The info you need is stored in summitslist.csv and there is a link to that at the bottom of SOTA Database

That CSV is updated every day and contains summit info and activation info for each summit. You only need to download that at the start of each month to have the latest summit data. Note however, it includes every summit including the ones that have been and gone.

Your log plus that should give you all the data you need to plot map lines. p.s. I’m sure there is an option for Google Maps that plots a flat map but great circle curve lines. It then looks like the route maps you get in the inflight magazines on aeroplanes.

Andy
MM0FMF

1 Like

In reply to MM0FMF:

Andy,

Thanks for the info. I didn’t realize the older QSOs were not valid for the S2S table. Had to delete 100+ contacts :frowning:

Now the upload worked OK.

73
Petr OK1CZ

1 Like

Apologies in advance if this question is a silly one!

I use the SOTA Results and Summits Database for various views and in chasing I change the view to sort order of Dec - Jan.

When I refresh the page, to see if a fresh chase has been added, the order changes and I have to select Dec - Jan again.

Is there a way to default the view to remain in the order selected, even on a page refresh?

Thanks.

1 Like

Nope not a silly question at all. At present no, that setting isn’t sticky, you have to set it each time.

1 Like

Cheers Andy, many thanks, not a problem,

Ian.

Holy thread necromancy Batman! :smiley:

I’ll add it to the list of things to do

5 Likes

Cheers, that would be great :grinning:

Is it possible to add how far away from your next award you are to the ‘my awards’ page?

That’s part of the fun, reviewing your own status and figuring out what is needed next should you be an award chaser.

2 Likes