Yes, bypassing the water softener might be a good idea for the long term health of the fish.
As for the high PH, that's not really that big of a deal, and not that high. Like has already been said, a majority of fish will adjust just fine. My bigger worry with using well water would be any metals or sulfer in the water.