You may not like it, but the easy answer to this question is Crocs. You buy them in bulk, leave them by the door, and slip them on to do your thing outside. Or I guess sabots if you are oldschool and French (they pair very well with some type of slippers we call “charentaises” that were originally made as inserts for sabots. So basically if you are using this ancient combo you don’t even have to take your slippers off to go outside).

For the bike ride you would obviously get some proper “outside” shoes. Although if you have dedicated gardening shoes you could choose to keep them on if you don’t care about looks (and if my grandparents were of any indication, I’d say that if you have grandkids, you don’t).