OK, so I didn't want to disable Smart mode altogether, so I've swapped the positions of Smart/Angle modes. My settings are now:
A01 - all active
Pos 0 (Angle mode) = 0%
Pos 1 (Smart mode) = 100%
Pos 2 (RTH) - 100%
This does work, just got back from a test flight in the backyard. Purple/White in Pos 0 (Up/away), Green/White in Pos 1(Middle), Red in Pos 2 (Down/RTH). At least this way, if I accidentally hit the Mode switch while I'm trying to get to the Landing gear switch, it should switch to Angle mode (if not already there).
Also, thanks to Forrestt and BobW55 for your info/testing. Great work!