I have an ft817 and I am familiar with its quirks, and Guru @EA2IF is even more an expert on it (indeed a Guru!).
I agree with his advice re the SWR indicator, even one blob showing on that is a sign that something needs improvement. When your antenna is very low (4m is very low for the 40m wavelength, only 10% of a wavelength) that affects the impedance and the efficiency. So not only is the antenna presenting the wrong impedance making it hard for your transmitter to deliver power to the antenna, what power reaches it is soaked up by the nearby earth, rocks, operator, kitchen sink etc. This is why the majority of activators are using poles higher than 6m. 7m is a good compromise on pole strength and longevity while giving good antenna performance. When erecting your wire antenna, try to get the ends as far above ground as possible. They must not be lying on the ground. The ends of the antenna have a large effect on the antenna resonance so if they are very close to ground the antenna frequency goes lower than it would be if it was 1-2m agl. If the ends are on the ground, the antenna is effectively shorted out.
Re the metering on the 817, do not expect to see more than 50% of the nominal power output when using ssb mode. The metering circuit is not fast enough to indicate the peak power of the voice signal, so it only indicates the average power. This could be between 30% and 50% of the actual peak. It may worry you to see the indicator at half scale, just mentally double it and that will reassure you.
Voice technique on ssb is very important. Talk like a newsreader, never let your tone drop below “HERE IS THE NEWS”. Every word has to be spoken at a high tone and level. This takes practice and nobody was born with this skill, they all had to learn it and practice.
I have made world wide contacts with the ft817 at 5w on ssb, but that was in better propagation conditions than we have now. Just keep faith that provided your antenna is up in the air, well matched to your radio, the meter indicates about 30-50% of max when you speak, and you keep your voice up, you are doing the best you can without fancy compressors or external amplifiers.
You did the right thing by self-spotting on suitable frequencies. That should work well once your antenna is radiating a good signal. But once you’ve done that, stay on that frequency if it is still clear. Not all chasers are at their radio when they see your spot. Sometimes I am a long way from the shack when I see a new spot and it takes me 5 mins to get to the radio. Staying on the frequency for at least 10 minutes after my spot is my usual practice.
I suspect with the low antenna and the high reflected power you were effectively operating at less than 1 watt, which is long way down on 5w. Answering other CQ calls is a good tactic, but don’t assume that just announcing your callsign while many others are doing it, will get you heard. If they are louder than you, how will you be heard? You would need to use extra techniques, by making your call a bit longer such as adding “Summit to Summit” if that is the case, - if other callers have given their callsign already, maybe the only part of it the other activator hears is your “summit to summit” phrase or part of it, and most activators like S2S points and will then stop the pileup and request the S2S station to repeat his call. So you do it with technique, rather than a bigger transmitter power or antenna. Try it.
73 Andrew VK1DA/VK2UH