Just to be clear - this was my fault, I just wanted to let others know of this behavior.
So, I decided to test the RTH function, went from angle mode to RTH and everything worked as expected.
After confirming RTH worked, I took control and continued flying as if I was in angle mode.
BUT, I...