Database Update(s) April 2017

I intend to make a number of database updates over the Easter break.

I’ve made the first update just. There was a huge degree of variation in the ordering of the years in the date drop down selector, some were ascending, some descending. Now they should all be the same, starting with the current year and working backwards to the start date. I changed the order as most people want to view recent history more than ancient history and having 2002 come up meant a lot of scrolling if you used a lower vertical resolution display such as a 1366x768 laptop or my 1280x800 tablet. Now the current year comes first. The list stops when the feature you are looking at stops. i.e. Summit to summit honour rolls stop in 2013.

EA9 which was added at the start of the month has been added to the ALL EA associations.

As usual, if you spot some new errors then let me know and I’ll fix them.

3 Likes

Pound pastrami, can kraut, six bagels…

[quote=“DM1CM, post:2, topic:15028, full:true”]
Pound pastrami, can kraut, six bagels…
[/quote]— bring home for Emma.

:wink:

Well not all the code I wanted to change was changed but between me and the summits team we’ve bashed out plenty of fixes to the data in the DB.

These fixes include feet and metres value being swapped, bad height data, missing values, corrupted summit codes.

We have a somewhere near 2000 summits where the wrong conversion factor between metres and feet was used that are still to be fixed.

I think there are a whole chunk of summit names that come out like “Knob, Charles” and “Whopper, Mount” or even ““Whopper, Mount””.

These are being hoovered up as we revise the association data and add new summits.

“Knob…”? “Whopper…”??

In feet or metres, size isn’t everything…

Why do we need feet in the database at all?

If redundant data isn’t present it can’t get inconsistent.

Martyn

1 Like

In an ideal world it would all be in metres, but unfortunately many of our friends across the pond still cling to pre-metric units.

…and so the presentation code simply calculates feet values from the meters altitude values on the fly as the GUI (graphical user interface = web page) is being built before being sent to the user via the magick of the web. Thus making, as Martyn rightly points out, feet altitude values in the database redundant.

This is what I like, everybody is an expert even when they don’t actually know why things are done the way they are or the implications of changing how data is stored or presented.

For all the experts who know better than the people doing the job, here is your starter for 10 (as B. Gascoigne used to say) what would be the effect if the the feet values were calculated on the fly?

Some of the values would come out slightly different, no doubt, because they were originally presented in feet and converted to metres. Obviously there will be rounding issues. That had occurred to me.

The question really is whether, given the tiny number of countries habitually using imperial units, whether this matters enough to fuss with maintaining two different columns. If you really care, then arguably there should be an additional column which says which is definitive.

Depends how many decimal places were stored in the meters field: if there were, say three decimals places in the meters field, one could convert to feet, round to nearest whole number, job done.

But as a lowly metal machinist and data noob, I’m just guessing, of course - what would I know about such abstract and complicated things as (looks up in dikshunary) errrm, “databases”?

Tiny number of countries, yes, but not tiny number of users…

I get it, I wish we were on metric as well. I’m a scientist for pete’s sake, and I have trouble visualizing litres, kgs, meters, etc. I have to use them all the time, and it’s still not easy for an American like me. Imagine the rest of the Americans on here that don’t have to use them often.

Even many in the UK aren’t fully on metric from what I’ve gathered.

Thank you MT for conveniencing those of us in the US.

Yes you’re right there. Certain areas of everyday life have proven exceptionally resistant to metrication, but everybody educated to a reasonable standard in the last half century or so is at least familiar with metric units, even if some pretend not to be. Most trades and professions are totally metricated.

But that’s not really the point. The suggestion - and it was only that - is that it might be simpler and easier to store the height in one place in a consistent way rather than laboriously trying to keep two different representations in step - something that has obviously failed if there are still over 2000 to correct. “One fact one place” is the mantra of database design, albeit dressed up as the jargon term “normalisation”. Other representations can be computed for convenience.

I was certainly aware that rounding error could be an issue, but my understanding is that many USGS surveyed heights have a measurement tolerance of ± 10ft anyway.

I would in principle say exactly the same thing about dropping the UK grid references from the main database, as they can be determined from the lat/long values, though it’s quite a lot more work than a simple multiplication.

Martyn

In the US, we get our data from the USGS which provides horizontal (lat/longs) and vertical DEM data in meters. See Getting USGS Data for more discussion. The satellite surveys over the past few years has become increasingly more accurate with publicly available data with LIDAR data down to 1 m or now dm (decimeters) accuracies near major population centers.

In the US, we have 41 Associations comprising nearly 52,000 SOTA summits. The feet datapoint is calculated from the metric datapoint. One problem is that this data is rounded to whole integers so we lose accuracy. The USGS is not too quick to point out that a given measurement is actually a plus/minus figure, e.g. 340 m +/- 1 m.

So for SOTA purposes, are we accurate enough? Probably. But it makes no sense, as Andy mentions, to introduce errors by converting from a rounded number, like feet, to meters then back to feet.

Guy/n7un

Speaking for the non-scientist metric challenged populations of the world, I can easily calculate whatever I need. And I actually do know how to make use of 1g = 1 ml = 1cc. Can I actually visualize those in my mind? No, I can’t.

Why do I always make sure I know the height in Feet of any target Summit? The Topo map I will have with me indicates contour lines in FEET. Go figure. If every reference to Feet was removed from the Database, I think American SOTA activators could deal with the conversions. And I suspect there might not be much fuss about it. Go to the Find a Summit feature of the Database. Meters and Feet are both listed. Then click to view extra data - where did the Feet go? No big deal.

Now lets discuss parts of the world who hate the way the US presents calendar dates. There is a very popular US west coast recreation focused company with a world wide mix of customers. What is one of the points many of the non US customers hammered them on? Calendar dates. And that isn’t even a conversion. It only requires a simple rearrangement of the order. You would have thought they had been asked to raise the Titanic instead.

If Summit elevation in Feet is not there, I will calculate it. If elevation in feet is there, I will check it. I am just glad I can look up whatever is there. If it is listed there, accuracy is appreciated. Where there are known issues - they eventually get fixed. Thanks.

Glenn

That is not the proposal.

The proposal is to eliminate the possiblity of storing metric and non-metric values which are not consistent with each other.

This is being fixed Glenn. W2SE, a while ago, pointed out this inconsistency. Jon, G4ZFZ, is undertaking a fix to include either both datapointson the various web pages, or allowing you to select what measure you want, imperial or metric. Now dates are another can of worms…

Guy

What we’re actually doing is fixing where the wrong conversion constant was used.

There will always be some error between the real height in feet and real height in metres if you only have the height to the nearest metre and calculate feet from that. I do recall some early data used that was in feet not metres but I think that data was originally measured in metres and then converted to feet for publishing. We may have taken those values and converted back to metres in the past. It’s these multiple conversions that need fixing. We could do just use the metre value and convert that on the fly and not save the feet values but we only save to the nearest metre so there will be possibly a bigger error than saving a more accurate nearest foot value. Either way there will be plenty of people who will tell me I’ve done it wrong and I should save them internally in attoparsecs and convert them back when needed!

We’ll fix the ones with the wrong factor first. Then where we have more accurate primary data (be that feet or metres) we’ll update values that may have been calculated from rounded values.

There’s loads and loads of people who think in feet and so we (the MT) feel we should provide the best values for those people that we can with the tolerances and resolution we are using.

We’re slowly updating US associations from P500ft to P150m as that lowers the bar on summits by around 8ft and adds a few hundred extra summits. Fixing the heights at the same time doesn’t make this job that much bigger.

I don’t know, how many feet does a fly have? Surely it should be a fairly consistent number amongst flies?:grin:

Matt
VK1MA

1 Like

Is this to be interpreted as meaning those who are charged with providing that data, over time improve their data collection methods? At that point there is more accurate primary data available. I would certainly assume you always used the best data available when initially entering a summit into the Database. Verifying large portions of it all over again could be a very big job. I would think anything off far enough to be noticed will be reported by the activators themselves. At that point just check that summit against the improved data.