When reading my altimeter I believe the altitude reading is the reverse of the barometric pressure: the higher the altitude, the lower the barometric pressure & the lower the altitude the higher the barometric pressure? (the actual dial numbers are reversed to reflect the altitude-not the pressure)

During a storm when the barometric pressure drops then my altimeter should read higher than it actually is? I have always noticed that at sea level I barely am conscious of breathing whereas at 5200 ft. elevation where I live one is usually conscious of breathing except during a storm/lower barometric pressure when I never 'feel good'/tired & have others say the same. Any meteorologist out there that can explain this better or more accurately (or a more logical form to post this!)?