Sounds to me like you just haven’t had enough practice, given that you aren’t interested in driving and haven’t don it much. If you can find people to drive you around or don’t mind taking buses and paying for Uber etc, not driving will probably be no more than a minor inconvenience - depending on where you live and how easy it is to get around without a car. Could influence your decisions about where to live and where to work, and having appointments could occasionally suck, but again it depends on where you live.