Don’t worry Brian, that’s what beta versions are for.
I look forward testing the next iteration!
73 Stephan
BETA build-65 posted to Apple Testflight and Google Play Beta. Two crashes fixed:
- Crash on launch when upgrading from a prior installation. The database schema change between older versions and the current Beta changed in such a way that crashing could occur. Fixed.
- Crash when doing a Summit or Park database search and you either select “cancel” or start out with a blank search string. The code was not properly detecting a null condition when the string is null vs. empty. Fixed.
The maps are still not loading on Android, but work on iOS. Hopefully with the two crashes fixed you can get further into the app.
Full release notes in the usual spot: Changelog – SOTAmāt
73 de Brian AB6D
1 Like
Downloaded the new update (older phone, Android 10) and now the app does not crash on opening. Also downloaded the SOTA and POTA databases.
No maps as yet, but I note in any case that the “Point me” feature does just that - I have a SOTA summit about three kilometers away from me, and the pointing seems to be accurate, and very responsive - turning the phone around here in the apartment, the compass pointer and red summit pointer follow my movements very smoothly - great job!
Cheers, Rob
1 Like
Hello,
Is it not possible to change the frequency local? Do I need to change them every time Online before?
It would be a huge game changer if I can change it directly on the activation to whatever frequency I need. Not only the predefined. Okay I maybe can predefine every frequency but it would be a lot of work doing that for every association.
73
Julian
One can define a start frequency and a number of increments.
So for example
14.270 Start with 10 steps of 1 kHz will give you
14.271
14.272
14.273
14.274
14.275
14.276
14.277
14.278
14.279
So not the full freedom but usually enough to find a free frequency
4 Likes
Hi Julian,
The problem is that EVERYTHING has to be encoded in a small number of characters sent over FT8, so there is only a limited number of combinations available. When that code gets to the SOTAMÄT server it looks at your configuration on the server to change back the combination of characters into the spot that you want to send.
73 Ed.
3 Likes
ah ok that’s good. Didn’t notfied this function thx Joe!
73
Julian
Hello Ed,
yes I understand the function behind SOTAMÄT. But I think with some decoding skills or even an other decoding table ( I know its a lot of work) it would be possible? Or am I wrong?
I love the tool. It is very useful in the nord of Italy, because there is a lack of phone connection in the mountains.
73
Julian
1 Like
Brian would be the one to reply on that point Julian - I am just a user, like yourself however I suspect Brian has squeezed out all of the possible usable space while keeping working within the limitations of FT8.
Now if FT8 could be changed to allow more data to be transferred in the “special message” then more flexibility could be possible but I suspect many, many other requests are further up the queue for the WSJTx programmers that such a request would be dropped.
73 Ed.
2 Likes
For sure! Thx Ed! Yes I know we where limited…
Thx a lot for your replies!
73
Julian
1 Like
Hi Julian,
First of all, many thanks for the S2S we had, also from your recent stay in Tenerife.
Brian has done a very good job of encoding the maximum amount of information that can be transmitted by FT8 and decoded by some skimmers.
After removing conflicting or special meaning suffix combinations, 1 million combinations remain.
Therefore, the formula that can be seen on the Sotamat preparation page applies:
Combinations = (((TotalSummits + TotalParks) * TotalFreqSteps) + Commands)
On this page you can also see your current used combinations, green means you’re within limits.
I gave two talks about SOTAmat this year, at the HB9SOTA general assembly (in German) and at the Ham Radio in Friedrichshafen (in English). Both links point to the corresponding PDF files. Perhaps some of you experimenting with SOTAmat will find this useful.
@AB6D Everything seems to be working as expected, thanks for the update!
The “Point me” feature could also be used to quickly point a Yagi in the right direction.
73 Stephan
2 Likes
Hello Stephan,
thank you too for the nice S2S!
Okay thanks for the information, now I’m seeing the lack of knowledge thats missing on my side.
Thanks for the presentation I will study it today!
73
Julian
2 Likes
@IN3JIB to your post:
Okay I maybe can pre-define every frequency but it would be a lot of work doing that for every association.
Short answer:
You don’t define frequencies per association. You define a single frequency/mode table. That table is shared for use by all SOTA Associations, SOTA Regions, Summits, POTA Locations, and Parks.
Long answer:
I did a presentation on this that was recorded and is on Youtube, but here it is in written form.
When trying to use FT8 to send commands we have several issues to deal with:
- What do monitoring skimmers listen for and report to PSK Reporter,
- What information does FT8 allow us to send to the skimmers,
- What information do government (FCC) regulations allow us to send to the skimmers,
- What does the PSK Reporter database store and allow us to retrieve.
- How many bits do we need to encode every possible SOTA Summit and POTA Park?
- How many bits do we need to encode every mode? Every band? Every frequency step within each band?
Chapter 1: What do PSK Reporter Skimmers do, and what does PSK Reporter store?
Skimmers that report reception reports to PSK Reporter (such as WSJT-X) don’t forward every message that they hear. They filter both in format and in time (removing certain duplicates). The de-facto standard is to only capture and report non-redundant “CQ” messages. Skimmers don’t send the entire string of the message received, they parse out various data fields (frequency, callsign, mode, etc.) and send a schema of data to PSK Reporter.
Any skimmer that supports the PSK Reporter system must batch 5 minutes worth of reception reports before forwarding, otherwise it risks being blocked by the server. PSK Reporter sometimes gets 400 reception reports per second. SOTAmat has a direct pipe from the database and is looking at every report to find matches.
Chapter 2: What information does FT8 allow us to store:
Joe Taylor’s document on the FT8 protocol says:
Standard amateur call signs can be conveyed in 28 bits, but
compound calls such as PJ4/K1ABC and special-event calls
like YW18FIFA may require more than twice that number.
Every FT8 message is 77 bits long (over the air it becomes 237 bits of data encoded as 79 three-bit FSK symbols). The first 3-of-77 bits tell you which of 7 possible message types you are dealing with, or if the first three bits are 0 then the next three bits tells you one of another 8 possible message types. In other words there are only 15 possible message types, and today 10 of them are defined as follows:
Type Purpose Example message Bit-field tags
0.0 Free Text TNX BOB 73 GL f71
0.1 DXpedition K1ABC RR73; W9XYZ < KH1 / KH7Z > -08 c28 c28 h10 r5
0.3 Field Day K1ABC W9XYZ 6A WI c28 c28 R1 n4 k3 S7
0.4 Field Day W9XYZ K1ABC R 17B EMA c28 c28 R1 n4 k3 S7
0.5 Telemetry 123456789ABCDEF012 t71
1. Std Msg K1ABC/ R W9XYZ / R R EN37 c28 r1 c28 r1 R1 g15
2. EU VHF G4ABC/ P PA9XYZ JO22 c28 p1 c28 p1 R1 g15
3. RTTY RU K1ABC W9XYZ 579 WI t1 c28 c28 R1 r3 s13
4. NonStd Call <W9XYZ> PJ4/ K1ABC RRR h12 c58 h1 r2 c1
5. EU VHF <G4ABC> < PA9XYZ > R 570007 JO22DB h12 h22 R1 r3 s11 g25
But remember I said that Skimmers only report on CQ messages? Well CQ messages are of type “1. Standard Message” and there is only enough room for a 28-bit callsign. In other words, while theoretically FT8 allows us to send up to 71 bits of any data we want (type 0.5 Telemetry data), one big issue is that nobody (no standard skimmer) is listening for it! If we could use those 71 bits then we would be able to encode a lot of information.
One ‘miracle’ (or happy accident, or bug with an unintended wonderful side effect) is that there was a skimmer on the market that didn’t decode callsigns based on the FT8 schema (the bit fields as shown above) and instead decoded the all the bit fields to a human readable string and only then did it parse out callsigns for sending to PSK Reporter. That software is SparkSDR. A second ‘miracle’ is that I accidentally discovered the feature (bug?) when sending an FT8 type 0.0 Free Text message that had a callsign with a suffix inside and saw it show up in PSK Reporter.
Free Text messages allow up to 13 alpha-numeric characters (with a few other symbols like ‘/’) to be encoded into the 71 bits (plus the 6-bits for the 0.0 type ID = 77 bits total like all FT8 messages). While SparkSDR was reporting these messages to PSK Reporter, it does not report the entire 13-character message, and instead has to follow the PSK Reporter schema which only includes the sender’s callsign. Miracle #3 is that SparkSDR will report the callsign with the suffix to PSK Reporter, and PSK Reporter will store that in the database.
Chapter 3: How much data can we encode into the suffix?
Well first we have to know how many characters we can get from SparkSDR to the PSK Reporter. Via testing I discovered that the SparkSDR parser will only ‘find’ a callsign + suffix if it is preceded by some other string followed by a space.
S KA2VYS/1234
The smallest string + space allowed would be a single character (the letter S in the example above) plus a space = 2 characters. This leaves 11 characters out of 13. Next we need to encode a callsign which in most parts of the world is in the form of PP#SSS (or less) or 6 characters. For a suffix we need a / character and that means we have 4 characters left for the suffix (2 + 6 + 1 + 4 = 13).
If your callsign is shorter than 6 characters, then SOTAmat will increase the preamble like so:
STM AB6D/1234
…where STM is short for “SOTAmat”.
Given that we can encode this much data (26 letters + 10 numbers = 36, plus ‘space’):
/1234 = 36 * 36 * 36 * 36 +
/123 = 36 * 36 * 36 +
/12 = 36 * 36 +
/1 = 36
= grand total 1,727,604 combinations
Converted to bits that is 20.7 bits of data.
We can get additional bits of data by:
- Using the time of when the FT8 message is sent. Every minute is broken into four 15-second time slots and that encodes an additional 2 bits of data (which I don’t currently use in SOTAmat)
- Using the audio frequency offset from the base “standard” frequencies (such as 14.074MHz). If you consider the 2,300 Hz audio passband and ‘bin’ it into 23 Hz chunks you can get about 6.64 additional bits. (SOTAmat no longer uses this trick for a number of reasons.)
So in summary we can theoretically (not not legally nor practically) get 20.7 bits + 2 bits + 6.64 bits = 29.34 bits of data.
Chapter 4: What do governments (ex. FCC) allow us to send?
I don’t have space here, but this is a complex question. I studied it for 3 months. In summary you can use a “self assigned Indicator” so long as:
No self-assigned indicator may conflict with any other indicator specified by the FCC Rules (such as “AA”, “AG”, “AE” or “KT”) or with any prefix assigned to another country (such as “DL”, “F”, “G” or “VE”).
Well figuring out what conflicts with every other county’s prefixes sounds simple until you start digging into this more deeply.
- I wrote a program that generates a “safe list” table and compresses it with run-length encoding. This is inside the SOTAmat mobile app and on the server and is a big array of numbers.
- Next we want to avoid suffixes that happen to have human meaning (including swear words and profanities). I took Google’s open source database of “bad words” and removed those suffixes.
- Next I decided to remove every English word of 4 characters or less so that we don’t transmit “HELP” or any other “meaning”. I used an open source dictionary for this.
What remains is about 1 million usable suffixes, or about 19.9 bits of data available.
Chapter 5: How many bits do we need?
- There are about 163,000 SOTA Summits and 47,000 POTA Parks = 210,994 in total = 17.7 bits required to encode each of them
- There are 7 “modes” supported by SOTA (AM, FM, CW, Data, SSB, DV, and Other). This requires 3 bits to encode each of them.
- There are about 16 amateur radio “bands” in most countries. This requires 4 bits to encode.
- With some bands we have about 4 MHz of spectrum (ex. 2 meters, 6 meters, etc.) and on other bands we have far less (like 300 KHz on 40 meters). The question is what precision do we allow people to specify a frequency at? 1 Hz intervals would be more than the 1M allowed combinations for our suffix. If we choose 1,000 Hz intervals CW operators aren’t happy. The point is that there are a lot of possible frequencies, and just the 10 meter band alone at 1 KHz quantization would require 21 bits by itself!!
So we have 19.9 bits of data available to us, and we need much more than that just to encode the 10 meter band in a way that isn’t friendly to CW operators (who like 100 Hz step size = 10x the combinations).
Chapter 6: The Solution
Since some operators want to be able to select from many possible summits around their homes, and other operators want to be able to choose from a wide variety of frequencies, the solution is to give every user their own 1 million suffixes optimized to their own desires. I could have a configuration that includes a single SOTA Region that contains a single summit and have 1,000,000 possible frequencies to choose from. Someone else might only want to work about 120 frequency steps on 20m SSB and 120 frequency steps on 40m SSB (total of 240 steps) and that would allow them to have up to 4,166 Summits in their configuration to choose from.
Chapter 7: It is a little more complicated
This picture shows how the encoder and decoder work:
You can see the process to encode:
- The Summit/Park index is multiplied by the frequency/mode index to get a unique “raw-ID” for a spot.
- The raw-ID is pushed through an expansion table that gets rid of the swear-words and illegal suffixes to get a clean-ID
- The clean-ID is modulo encoded into 1 to 4 suffix characters with a space shift algorithm
- The user’s callsign + suffix is turned into a 13 character FT8 Free Text message by expanding a preamble (ex. “S”, “SM”, “STM”, etc.) to fill the space.
The decode process is more complex:
- Each user has their own private configuration. The meaning of your suffix “/1234” is not the same as my “/1234”.
- The callsign is looked up in the database to find that specific user’s configuration
- The suffix is inverted through the “bad words” table, and then turned into a Raw-ID
- The raw-ID is turned into a frequency/mode and a Park/Peak ID
- Here is where things get really complicated:
- The server remembers when your mobile app accessed your configuration (call it Day 1). On Day 1 the SOTA database might have had 120 summits in region W6/SC.
- On day 30 the SOTA organization may have added a new summit to the W6/SC region so there are now 121 summits in that region. That means that all summit indexes for all regions after will shift their ID’s. Look at the picture above to see what inserting a new summit does to the matrix. Everything shifts.
- On day 35 you do a SOTA activation. Your mobile phone still thinks region W6/SC has 120 summits while the server knows there are 121. You send a self spot with a suffix based on this data. If you spot yourself on a summit that comes after W6/SC in your table your index will be based on the old definition.
- The server receives your self spot (still day 35). Rather than use the current 121 summits it will look at the timestamp of when you last pulled your configuration from the web site into your mobile phone. That timestamp will read “Day 1”. The SOTAmat server will look at your configuration and the timestamps on all the edits to the regions you’ve referenced. It will see that W6/SC summit 121 has a newer timestamp of “Day 30” compared to your configuration of “Day 1”. The server will decode your suffix based on the 120 summit region in agreement with your phone. These timestamps are critical to the operation. This is why the “Experts Only” blob stuff in the UI is really for experts! If you use the Experts Only features, you are updating server side timestamps when you pull a configuration manually rather than using the app to do it.
Anyway, I hope that answers your questions about why you can’t just pick any frequency you want. If you are willing to only configure for a single region you plan to activate, then you can reserve most of your 1 Million combinations to be for frequency choices.
73 de AB6D - Brian
P.S. Appendix
- Since SOTAmat shipped 2 years ago there are more skimmers that do what SparkSDR does. The author of CWSL_DIGI updated his software to support SOTAmat
- The SOTAmat Skimmer is a plugin that adds SOTAmat capabilities to WSJT-X.
- There are about 90 SparkSDR skimmers worldwide, 30 CWSL_DIGI skimmers (often with very good antennas), and about 10 SOTAmat skimmer plug-ins = about 130 active skimmers. You can be heard from almost anywhere in the Americas and Europe, and often in Austrailia and New Zealand. Asia (and Japan in particular) is spotty due to a different band plan and lack of skimmers.
You can volunteer to help out by running your own skimmer: instructions here.
4 Likes
Okay thx a lot Brian for this detailed explanation!
I got it now 
Maybe during Christmas I’m able to put set an skimmer station at my parentes QTH.
THX a lot for your explanation and also for your APP!
73
Julian
3 Likes