Water doesn’t compress (for all intents and purposes), so no. I.E. the water pressure of the ocean would indeed push inward against the balloon much more strongly, but the water inside the balloon would push back by the same amount.

Nothing special would happen if you pop the balloon, the rubber would just rapidly contract, basically the same as it would above the surface.