The process fails on average 4-5 times / month. I’m not sure of the failure cause, I reckon it’s a file locking / multiple access problem in that there is a process using the file (somebody downloading it) when the daily update schedule starts. The result is the process trying to create the new file cannot because the filename/handle is locked and in use. I have tried many “this solution fixed my problems at once” solutions I found on the net and none have worked. It could be me, or a misconfiguration in the hosting company.
It’s got worse since SOTA became more global and more a 24hr operation. That’s why I think it’s a multiple access problem. The update runs at what was a graveyard time and now is just a quieter time and the chances of someone asking for the file at the same time as the update is greater.
When it fails, there will typically be no public copy of the updated file available. I get sent an email telling me the process failed so I know to kick off the update again. I’ve seen that fail and have had to sftp into the server and remove stale files. But 99.999999999% a second run works. I normally see the fail email around 8.30am and the file is in place by 8.32am. This morning I was up late. (Boo last day of the huge UK Christmas shutdown today, work tomorrow).
My advice to anyone who uses the file is not to overwrite your existing copy with the new one. But to fetch that to a temporary location and when it succeeds, delete the old copy and move the just downloaded copy. If you don’t have anyway to roll back a failed download (no matter why the download failed) then you will bork yourself completely at some point.