ADIF Processor - Google Earth, ADIF Enhancement & Markdown Tables

M0NOM · 15 July 2021 16:20

Introduction

I’d like to announce the availability of my ADIF Processor tool. It originally started out as a way to process the output of Fast Log Entry which has limited support for ADIF fields by using key value pairs in the comment section to populate other fields in the ADIF file.

However, as it grew I added the facility to output a Google Earth KML file and it does very basic propagation ‘predictions’ based on a very simple atmospheric model. So it an easy way to turn an activation (or any other radio activity) into an interactive QSO map. If you have read any of my recent activation reports you’ll be familiar with what screenshots look like, but there is something special about being able to explore an activation’s QSOs interactively.

It also has a module to generate fixed width column output, which I used to print a ‘log book’ out on a 132 column line printer, but much more useful is that I have enhanced that facility to produce a Markdown table - which can then be pasted directly into activation reports.

There is still work to do, but if you want to try it now there is a beta-version on AWS: ADIF Processor

Specifying your Location

A SOTA activation log will have a MYSOTA record and this is enough to allow it to work out your location. If you were just operating portable, the MYGRID record with a 4/6 character Maidenhead locator will put you somewhere approximate. If you are operating from your home QTH, and your location is set in QRZ.COM, this is enough for it to work out your location. Otherwise you need to provide an override.

There are two options:

Latitude, Longitude: these can be obtained from maps.google.com by searching for a location, right-clicking, and selecting the latitude, longitude. This can be pasted into the form directly.

Finding a location in Google Maps

I also routinely use the HamGPS application to record my 10 character Maidenhead locator which can be specified as the GRID and will put you within a few metres of your location.

Activity Logging

Alternatively, if you have been doing one of the following:

Summits on the Air
Wainwrights on the Air
HEMA
Parks on the Air

then you can enter the summit/location reference in the Activity panel. Please report any bugs, the ADI file that applications generate vary wildly and there will be issues.

If processing is successful you’ll see the results page. If you didn’t specify your location you’ll see an error and won’t be offered a Google Earth KML file. If the application was unable to determine the location of a contacted station their callsign will appear in the box below. You can always go back and add a location in the ADI file. When using Fast Log Entry a Maidenhead location specified in the record starting with a hash #IO84NI is good enough. Sometimes folk don’t set their location right in QRZ.COM - you’ll often see them in central London if they are UK based. It’s up to you how far you go to fix these issues.

The three output files are:

An ADI file containing locations where they could be determined in the Latitude & Longitude ADI fields.
A KML file for loading into Google Earth.
A Markdown file. This will contain a table of contacts that can be pasted directly into reflector/discourse:

DATE	TIME	CALLSIGN	FREQ	MODE	RSR	RST	SOTA-TX	SOTA-RX	QTH	MHL
2021.07.13	10:17	G8CPZ/P	145.475	FM	59	59	G/LD-040			IO84KI
2021.07.13	10:20	G0WPO/P	145.450	FM	59	59	G/LD-040	G/LD-051		IO84LJ
2021.07.13	10:27	M0VPM/P	145.550	FM	58	55	G/LD-040	G/LD-004		IO84KP
2021.07.13	10:33	G7CDA	145.350	FM	58	59	G/LD-040
2021.07.13	10:36	G1OHH	145.350	FM	59	59	G/LD-040
2021.07.13	10:37	M1BUU/M	145.350	FM	55	55	G/LD-040			IO84rc
2021.07.13	10:40	G6LKB	145.350	FM	59	59	G/LD-040
2021.07.13	10:41	M7MCG/P	145.350	FM	59	57	G/LD-040		WEETS HILL
2021.07.13	10:42	G7SXR	145.350	FM	53	55	G/LD-040		LEEDS

KML Output for Google Earth

If using the Android version of Google Earth you can just click on the downloaded KML file. If using the browser version of Google Earth then select Projects and load from file. I don’t recommend using the ‘Professional’ application version.

You can click on both stations and the contact traces (light blue) in Google Earth for a summary of the station/contact details, and callsigns hyperlink through to QRZ.COM

Depending on the activity and state of the station (/P, /M, /MM) the application chooses an appropriate icon.

Have Fun!

There are lots of enhancements planned (one of the first to provide more ‘enrichment’ of the ADI record from the QRZ.COM XML data stream) but I think it is stable enough to release to the world. It would be great to see some of your results as screen grabs.

I’ll also try and expand this post to make it into more of a useful ‘manual’.

Please message/email me with any problems, including the ADIF you used.

Kind regards, Mark.

M0WIV · 15 July 2021 17:49

Mark,

Thank you, I’ll give that a go tomorrow as I’m enjoying a beer at the moment after a hard afternoon chasing on CW - only 2 QSOs but 10 points earned from them.

The facility to enter the SOTA reference looks good.

M0LEP · 23 August 2021 11:00

I have some fairly basic ADIF files that log my activity, both SOTA chasing and otherwise, so I tried a quick log upload to see what happened, but there’s obviously something my ADIFs are missing. All the QSOs have CALL, BAND, FREQ, MODE, RST_SENT and RST_RCVD, and I try to make sure they all have GRIDSQUARE as well. All the activity’s from my home, so I figured providing an override location should cover the lack of any MY_* fields. What else is essential?

MM0FMF · 23 August 2021 11:05

How to make an ADIF that works.

Start with a CSV that works and go here:

https://www.on6zq.be/w/index.php/SOTA/SOTA2ADIF

G4TJC · 23 August 2021 11:21

You will need a SOTAREF for at least one end of each QSO as appropriate for chaser/activator uploads.

M0LEP · 23 August 2021 11:24

Thanks. That’s fine for the SOTA, and may give me a few hints. My log’s not all SOTA, though, and obviously I’m not chasing from a SOTA summit, either.

MM0FMF · 23 August 2021 11:26

That was the purpose… to show you a valid ADIF format for SOTA.

M0LEP · 23 August 2021 11:59

My old general-purpose logger doesn’t produce any SOTA-specific ADIF fields. I just put SOTA references into the QTH field, and the perl I hacked together turns the logger’s ADIF into csv that works with the SOTA database (once I’ve removed all the non-SOTA lines).

I got something presentable out, though it took a few steps (export ADIF, convert to csv, remove non-SOTA entries, convert to SOTA ADIF, upload to ADIF Processor, download KML, open in Google Earth) and it’s only got my SOTA contacts in it…

M0NOM · 23 August 2021 13:40

OK so you found a bug

Can you believe I have never tried this with a log from my home QTH? The logic to determine your location from QRZ.com in that situation (ie not portable) was missing.

So I’ve uploaded a bug fix, if your QRZ.com location (preferably set as Lat/Lon, but at least Grid) is set you are good to go.

The idea always is for the software to determine as much as it can on its’ own.

Regards, Mark.

M0NOM · 23 August 2021 13:43

If you add an entry in the ADIF comment field for any SOTA activators you make contact with of the form ‘SOTA: G/LD-010’ the ADIF transformer will recognise that and set their location and the MY_SOTA value in the output ADIF file. It does the same for WOTA, POTA and WWFF references but in those cases it sets the SIG ADIF field to be WOTA/POTA/WWFF and the SIG_REF ADIF field to be the entered reference.

Mark.

M0NOM · 23 August 2021 14:04

Also worth mentioning there is a new feature now on the Options panel, click on the Options link to show the panel.

If you tick the Show Local Activations Sites checkbox then transformer will add markers on any SOTA summits within the specified radius of any activator you have contacted, and if you are on a SOTA summit your own summit. This applies to any of the other activities too, where there is a specified location. So if you click on the activators callsign in the list, or manually zoom in, you can explore their local summits.

The markers can be clicked on for a link to the relevant SOTA Summits page.

M0LEP · 23 August 2021 17:58

I think QRZ has my home QTH set accurately enough, but I’ll check next time I’m looking. I tried setting an override location but that didn’t seem to fix the issue with the original ADIF I’d exported. I’ll see what I can do about ensuring key and reference pairs get into the exported ADIF where appropriate. Is it intended to work when neither end of a QSO is one of the SIGs?

M0NOM · 23 August 2021 18:02

Absolutely. The most common issue I find is that folk don’t set their home QTH in QRZ.com. In the UK this defaults to a London Grid square. I don’t think I’m able to lookup a location based on an address, that would be the last resort.

If you want me to check your ADIF file please email me it at mark@wickensonline.co.uk.

Regards, Mark.

M0LEP · 23 August 2021 18:10

QRZ has my position to four decimal places, which I would expect to be enough. I’ll send you the ADIF I’ve been trying…

M0LEP · 23 August 2021 18:53

…and thanks. For anyone interested, it turns out one QSO had garbage where it should have had a Maidenhead locator… oops.

(There was at least one other incorrect locator in the ADIF which the KML allowed me to spot.)

M0NOM · 28 September 2021 22:37

I’ve been working on the ADIF Processor in my spare time and have now added support for SOTA V2 CSV files. It is early days, so no doubt there will be some issues, but CSV files downloaded from the SOTA Database should work OK.

You don’t have the flexibility to fix location issues like you do with an ADIF input file, but it does as good a job as it can.

Location determination has improved greatly since I last posted. Stations that have a ‘default’ location set in QRZ.com are now detected. Typically in the UK stations can end up mapping to Central London, or for example the middle of the Atlantic.

In these cases the application now ignores the faulty location data in QRZ.com and instead performs a geocoding based location lookup on any address data provided in QRZ.com and tries hard to get as accurate a location as the data will allow.

If you experience an error please email or PM me the CSV file so I can correct the issue.

Given that you get an ADIF file out this turns the application into a somewhat fancy SOTA CSV to ADIF file converter.

Regards, Mark.
M0NOM

M0NOM · 28 September 2021 23:04

What this means of course is that you can now use Google Earth to be a virtual activator as the entire SOTA database is available for visualization.

Take for example the recent 09/09/2021 activation of Matterhorn HB/VS-006 by Martin @DK3IT. The downloaded CSV converted to Google Earth KML provides a wonderful SOTA fuelled world to explore.

The Google Earth KML file that was used to generate this image can be downloaded from here: 1632869690919-DK3IT_566759_activator_20210928.kml.

Regards Mark.

HB9EAJ · 18 December 2021 11:30

Hi Mark,

Thanks a lot for providing your great ADIF Processor!

I experimented with the resulting KML file and created an animation by using some features of Google Earth.
You can see the result while I filmed my activation two weeks ago at 1:30.

73 Stephan

M0NOM · 19 December 2021 21:32

Hi Stephan.

Really like the video, and you have very intelligent used Google Earth to demonstrate a DX success!

Thanks for using the software.

Mark.

HB9EAJ · 20 December 2021 08:56

Hi Mark,

Thanks for your feedback!

It was my first try with Google Earth. It doesn’t always do exactly the way I had in mind, but I think one gets the basic message.

First I created some paths and then manually followed them. The resulting video was recorded with the built-in movie maker.

Your ADIF Processor is so cool!

73 Stephan