Other SOTA sites: SOTAwatch | SOTA Home | Database | Video | Photos | Shop | Mapping | FAQs | Facebook | Contact SOTA

New version of SOTAData enters extended Beta stage


#68

Hi Patrick,
as always if you are able to provide programming hours and have the skills to change the interface to add ADIF import to the database, I suggest you contact the management team and offer your services. In SOTA everything is done by volunteers so while we can all have good ideas, getting them implemented is usually the problem. Sometimes the advantages don’t outweigh the work required.

73 Ed.


#69

Hi,

That’s what i already did using the contact form.
I’m not a professionnal programmer, but do have some skills and can probably help…

73, Patrick


#70

Speaking partly for Andrew here, the reasoning for there being a new database app are as follows.

  • The old app works fine but is built on a now very old framework that is still supported but nobody knows for how much longer.
  • The way the app works makes it very difficult to support single sign on.
  • The menu system doesn’t work well on Android and is difficult to change.

With those thoughts myself and Andrew decided a brand new database app was worth writing. This has been designed to support SSO from the start. As someone said the menus now work on Android and the technologies used are more future-proof. Andrew took the existing code and SQL routines and wrote the new version from them. Along the way a few warts were removed (the need to upload CSV files twice) but the code will behave essentially the same as the old code except it looks different. A more fundamental internal change is the new version supports an API for extracting data. The old application offered the screen displays you saw and the ability to download certain data in CSV files. The new one will allow data to be extracted directly by applications, e.g. “all of MM0FMF’s 2017 chases” or “all of G4OBK’s 2019 activations”, “all the summits in association PY1”.

So this is where we are now. We have a written application that copies all the users features of the old application. It’s been beta tested, now it’s time to given it more testing. There will be bugs and there will be parts that could be improved. Bugs need fixing ASAP. There are places were supporting keys and mouse action are not quite the same on different pages, sometimes filter selections get lost when moving pages. None of these are bugs, they are trivial in the grand scheme, you can use it but sometimes you may need to select CW on a filter on a page and on the next page.

Once we can see how it stands up to more significant use we can work on fixing the bugs and these less important features. Then when that is done, we can look at enhancing the core with things such as ADIF. In the meantime there are plenty of 3rd party solutions available to convert between CSV-v2 and ADIF so that you can move logs between the database and your own logging software. There are logging programs that can output CSV format files directly.

Christophe maintains a webpage that lists many SOTA tools, he also has a number of web based tools at his site: http://www.on6zq.be/w/index.php As Ed pointed out there is an ADIF converter on the Sotamaps page: https://www.sotamaps.org/extras and I know that logging programs Rumlog, Log4OM can directly produce CSV files (there are more but I don’t remember them.)

So this new database application is not complete but is just the start.

If anyone has written any SOTA programs that they make available and they are not listed on Christophe’s page then contact him (details on his pages) so he can add them to the definitive list.


#71

Hi Andy,

as you might aware of, I’m a app developer.
Is there any chance to make use of the https://api-db.sota.org.uk/ API?

Is there any description available? I’m mainly interested on uploading the QSOs.

vy 73, Thomas


#72

I’ll volunteer to help out. I’ve done a decent amount of XML programming over the last twenty years. ADIF 2.x appears to be an ad hoc SGML-like monstrosity, but 3.x is supposed to be legal XML.

What platform is used for the site? I’m probably most fluent in Python, but I’ve programmed in a lot of languages, even Concurrent Euclid.

A few years ago I wrote a program to compare old and new summit lists and create a CSV showing location, elevation, and name changes. That was used for updates to W3 and W6. I’m glad to contribute that. I should probably port it to Python 3, though.

If someone would send me a link to the beta site, I’d appreciate that.

wunder


#73

Eventually. Not right now while it’s in Beta stage


#74

Try solving the puzzle :slight_smile:

Honestly, the easiest approach is simply to take the existing ADIF2SOTA code and adding it in to the Beta code. Everyone seems to want to reimplement it, but it’s been implemented already about three times. It’s on the list to do, it’s just not a high priority given the abundance of tools that already do just that.


#75

I started that, then decided it meant you didn’t really want beta testers.

wunder


#76

Yep, that’s me, old grumpy-bum sitting around telling people to get off my lawn while I write software and never publish it. It’s lonely here in my hermit cave on my own with nothing but a laptop. Since there’s nothing to do now the kids are off my lawn, I spent all of 5 minutes throwing together a quick one-liner to solve the puzzle:

for i in `wget -q -O- 'http://reflector.sota.org.uk/t/new-version-of-sotadata-enters-extended-beta-stage/19124/1' | grep -A32 "<ol>" | sed 's/<.*>\(.*\)<.*>/\1/'`; do wget -q -O- "https://www.sota.org.uk/Summit/$i" | grep "<title>" | sed  's/.*, \(.*\), .*/\1/'; done

But no, I don’t want beta testers. I just have just spent the last few months ignoring every bit of feedback I’ve gotten. It’s easier that way.


#77

Hi Andrew,

New sites are looking good. As noted earlier, I found my SSO credentials were surviving an overnight break whereas previously they would time out, presumably being session cookies.

I notice some of the reports do offer mode-selections for results but have the existing defect that they do not really offer a mode based score, rather they select the activaations with at least one contact of the nominated mode. Then the analysis option no longer offers mode-based stats. This may be intentional, if so I need to put my log into a spreadsheet for analysis, rather than the log summary produced by the analysis report. But if not intentional, one to note in the list of queries.

73 Andrew VK1DA/VK2UH


#78

the one liner works a treat, thanks Andrew. It does fail on clues 23 and 25, but those are easy to fill in.
The new sotadata looks great, I’ll do more testing tomorrow.
73 and HNY and thanks for all your work,
Malcolm VE2DDZ


#79

Today I have made available an updated version of VK port-a-log that has only SOTA V2 CSV files for upload to http://sotadata.org.uk/

73
Peter VK3ZPF
http://www.vk3zpf.com


#80

New sotadata page on an ipad 6 showing a duplicated menu options list, different font and rather unusual menu topics. This error was not showing a few days back.


#81

Thanks for the new version, especially for the possibility of uploading just one file for activations and S2S contacts !

I have recognised by accident that on summit reports the 60 m band is missing. The total of QSO s is correct but the 60 m band is not showing.

73 s Bruno HB9CBR


#82

A few days after I saw this thread on the Reflector, I solved the puzzle and got into the Beta database portal. I was motivated by the fact that I use the SOTA database a lot. I do many activator logs with numerous contacts, as well as plenty of S2S contacts, and I spend considerable time doing logs.

I type most of my logs into the database manually. Therefore I care about the features provided, more than those who upload their logs as CSV files.

I’ve submitted two large, real activator logs via the new portal, and they’ve been accepted without any major issues. These were both hand-typed into the fields provided and then submitted. I’m delighted that this interface works so well at this stage!

Most of you are far above me in your understanding of how this database works, but I do have a clear perspective as a frequent user. I realize that we have a Beta version, that many of the points I see may be temporary, and that changes are perhaps intended. Some of what I see may not be what others see, because of differences between our platforms. Here’s what I’ve noted about the Beta Version since I logged on:

  1. The summit cannot usually be selected by simply typing in the parts of the summit reference. Sometimes the Association will come up if I type a letter or letter, but not always. The last three digits always require scrolling down and selecting the number.

It would be nice to just type in the summit information, instead of going through three awkward selections, since we usually know this.

  1. When I’m using the current Operational interface, as well as other software, my computer does auto-fill for many fields. This is mostly a time-saver. So far I’ve seen no auto-fill for the fields in the Beta version.

  2. Each QSO that is typed requires mouse-clicking on the button “Add QSO” for it to be added to the log. Hitting the “ENTER” key does nothing. In the Operational version, hitting “ENTER” completes the entry and is much faster than using the mouse. This is relatively important.

  3. If I skip a contact entry, then discover my omission before submitting my log, I can still type it in at the end of the list with either interface.

In the Operational interface, the added entry is automatically inserted into the list in QSO time order, before submission of the log.

In the Beta version, the added entry remains at the bottom of the list.

However, once submitted, the added QSO ends up in correct time order in the database list, regardless of which interface is used.

  1. The spacing between lines in the Beta version is very wide, so that only 10-12 QSO’s are visible. This is about half what I see in the operational version. This means I have to scroll among the lines much more as I check my entrees. On a big activator log, like the 57 QSO log I did today, this is a bit cumbersome. It would be nice to be able to adjust some of these parameters. The look and feel is very different from an Excel spreadsheet, which most of us are familiar with.

  2. There is very little color in what we see. It’s very gray. A little color is used to great advantage at some of the key transactions, such as entering QSO data, and at log submission. Color is a powerful tool in spreadsheet-type data display, and more color could be used for the main features. Our brains key in on color, we process it very fast, and good websites and pages use color to advantage.

  3. The operation of the Beta version is incredibly smooth. I’ve not seen a single glitch or hang-up since I first logged on. I’ve used most of the menu functions, accessed various logs, downloaded logs, entered S2S contacts, and looked at logs of others, and it all flows perfectly. It seems as fast as the Operational version, even when downloading huge piles of logs, which I have done.

In contrast, the Operational database often kicks me out, for no apparent reason, and sometimes requires me to do multiple log-ins to complete even a single activator log. It also kicks me out when I do S2S logs, which are fairly short, but then I must re-type all the information - it isn’t in the buffer like the activator logs. Some days are worse than others, but the Operational interface either times-out or otherwise glitches, and I see the screen of junk that tells me I have to log in again.

Up to now I’ve assumed it was because of my older system, my router, my WiFi, my typing errors, hitting two keys, etc., and who knows what that caused me to be un-logged.

The new interface - so far - is smooth as silk, absolutely delightful by comparison! I haven’t tried a long delay yet, but my comparison experience is like night and day!

  1. The pages are simple and uncluttered, and they are very clean.

  2. However, the font I see on the Beta version is not very easy to read - it’s not jet-black, the letters don’t have sharp edges on my large, sharp monitor, it looks more gray, and it needs improvement.

The Operational Database is much easier to read, even with the denser fields and closer lines, and it’s all about the font and the type. I wish I could pinpoint the problem better, but legibility is VERY important.

  1. The menus on the new version work fine - but I prefer drop-down menus, or some combination - perhaps we could use redundant choices. The menus are really small, and they would be better with color.

  2. The complete logs are easier to read in the Operational version. They use boxes, black fonts, etc.

  3. The formats we use for the dates D/M/Y when we enter our log data are reversed from the format Y/M/D used to display our stored logs.

  4. The complete stored logs don’t show the time period or dates covered by the log, at the top of the log, where it matters. Every log displayed should tell what period it covers. We should not have to scroll down through our huge collections of logs to see what period each one covers.

  5. The Beta version of the manual S2S entry page has major improvements over the Operational Version.

While the fields are essentially the same, when entering more than one S2S contact, if I click “Add Another”, the Operational Version wipes all the fields clean and I have to retype ALL the information for EVERY field. The Beta Version keeps ALL the information in the fields, so all I need to do is update the key details of the QSO.

In particular, the information for MY summit ref is still there, so I don’t have to go through the cumbersome summit-ref selection process for each S2S log entry!!

This is a HUGE improvement, very helpful! Of course not clearing the fields can cause errors - if a field that needs to be be changed is not edited, the log may contain an error. I still like not having to re-type every field!

  1. The headings on the log spreadsheets appear in a very tiny font - and they stay at the top of all the columns, so we can’t see them while looking down the logs - at least the headings should be larger and bold, maybe even displayed in color. If all the columns were colored, we would soon learn what each column contained, never mind the headings! More color - more legibility - less fatigue.

  2. The Analysis features for the activator and chaser logs are very cool! I was surprised to see these options. They work smoothly and reasonably quickly. While the displayed results are HUGE, it all runs just fine through my old hardware!

  3. When submitting most logs, or downloading, a large blue rotating circular display appears to show that something is happening. This is much better than an hourglass or whatever.

However, when I SUBMIT an S2S log, I see nothing to indicate progress. There is no rotating circle, nothing. The log is uploaded, but there’s no progress indicator visible.

  1. I checked to see if my S2S logs processed through the Beta version are correctly updating my chaser logs, scores, etc., and so far I see no problems. The interface seems to be doing just what it should, going into the SOTA Database.

I may seem critical of a few points here, but overall I’m just amazed and delighted that this new product is running so efficiently. I know these great results represent untold labor and mental challenge over countless hours.

Please accept my comments in the positive spirit of suggesting a few useful improvements that will have a huge positive effect for the thousands of users who will soon use these tools to enter and display their SOTA logs.

This is a long post, but I really believe that the reason you gave us the riddle to solve, was to select a few motivated early users, and not to have too many people posting useless comments.

I’m not sure if you want feedback - but since you posted a fun riddle I could solve, at over 70 years old, you got it. This Beta version is an amazing piece of work - and with a few significant tweaks it could be so cool!

I’m running this on a Dell Pentium desktop machine made in 2003, with 2G of RAM, using Windows XP, Google Chrome, over a WiFi router and a cable modem. It runs SO MUCH BETTER than the Operational version that kicks me out! I think the data transactions to the database are faster with the Beta version, but I have no way to be sure.

When I solved the riddle, I wasn’t sure if I really wanted to see where it would lead. So far, so good!

Thanks and 73,

George
KX0R


#83

Took me a little while to track this down: 60m is being reported by the API, and the 60m column was present on the page too, but I’d left the 60m column out of some CSS, and the browser decided the best way to display that would be to collapse the column from view.


#84

Your iPad has failed to download the icon font that’s specified.


#85

Perhaps so, but where did the additional text come from, overlaying the standard menus? Ie. the words “account circle, library books, terrain, cloud upload”

The normal menu titles are displayed in a different font underneath those texts. I’m theorising this has come out of a javascript library function that is not being suppressed. But just a guess.

Andrew VK1DA/VK2UH


#86

That’s the actual character names within the font library. Without the font, it’s displaying that in whatever your default font is.


#87

Wow, I worked on a SAP enterprise portal still running in IE6 compatibility mode and I thought I’d seen every weird message possible!

Reported as it seemed slightly odd. Ipad users are used to odd things happening though!

73 Andrew VK1DA/VK2UH