You can remap the button or disable it entirely, and it's the least prominent button on the controller, so I think your level of irritation would be low to none.