Important Database Change

Online Resources Database
21

It’s excellent experience being able to do things like this. My SMS spotter does lots of low level socket activity to /GET and /POST on SOTAwatch ( and the WOTA and HEMS sites). Assorted logging in and playing with cookies and slicing data here and there. I did it that way as I was learning Python at the same time. A year or so later I learned of all the ready to run HTTP and URL handling libraries Python already has built in and saw how trivial it was to do things that took me a lot of head scratching and trial and error to make work. I could have cried. But it was all useful stuff to learn. When I update the spotters for the new SOTAwatch, then all the hand-rolled code will be ripped out and library routines used. That’s simply because there’s lots and lots of use of that code and it’ll be a damn sight more reliable than my code when the corner cases come in!

Last night I was working on how to push CSV data and found there are many open source CSV handling class libraries for C# but the database has all sorts of hand-rolled code to do that. I thought I should actually start using a tried and tested class library and remove all the code that works (sort of) we have. Well that’s for the future.

In 2010 I did lots of date handling code for the database. I wrote it all myself. On 29th July 2011 someone asked the database what was happening 6months back. My date code said “ah, 29th February 2011 please” and the database engine said “you what, have an invalid datetime exception”. I changed the code to use the C# function to take 6months of 29-july-2011 and it said “you want 28-feb-2011”, it knows about leap years which I had blissfully ignored. I always use the C# and Python date libraries now.

So you get a :+1: for writing your own and making it work. Now you have the satisfaction of knowing you can do it if you have to, throw it all away and use an open-source library. It’s what I do now.

1 Like
22

Hi Andy,

I am not sure, but does the file only change when a database update runs? If that only a few times a month? If so, the file could be regenerated at that time and just sit there until the next update, as a static file that could be regarded as always up to date. No dynamic generation required when external software wants the file.

I am stabbing in the dark here because I don’t have any experience of working behind the scenes with IIS.

73
Andrew vk1da

23

No, the file updates once every 24hrs at 0200Z I think. I need to change it so it’s not on the hour but at something like 0217Z so everyone who does some housekeeping at 0200Z doesn’t slow our housekeeping down!

24

I’ve tweaked the code so it should (yeah should) cope with locks. There should be no change to the file itself or where and how you find it. I was trying to do the “git diff…” between the new and old versions whilst talking to Mrs. FMF on the phone. We’ll see if it’s any better.

Whilst there, I did another quick tweak which I’ve been meaning to do for a while and that is the ability to sort the summits displayed in order of summit ref, or newest or oldest activation. If you can sort a region of summits by when they were last activated you can make sure you don’t go up a summit that was on a day or two before. If you select “By Oldest Activation” you get unactivated summits first. The order the unactivated summits appear is dependent on the order they were loaded.

As usual, if anything seems bugg^W broken, then let me know.