Make sure all your compass calibrations are done.
Instead if using the "magic" button.
In smart mode.
First short press both left and right arrows, try the watch me mode first, then turn on your altitude function.
But you have to do it twice.
Maybe because it is on by default. Then go for a...