The barrels smallest diametral dimension has to smaller than the bullet, otherwise the rifling wouldn’t engage.

Kind of, but not really. The bullet actually slightly expands from heating. At the same time though, if you measured the diameter of a cold barrel and a hot barrel you would get two different measurements. If the cold barrel is too tight of a fit the bullet has an increased chance of lodging when it heats up and expands.

This is why bullets moved towards copper jackets with some sort of other metal center chosen for various justifications. The copper jacket is soft, east to expand into rifling grooves, not hard enough to damage and wear out rifling, and can stretch a little if it gets too tight.

Really, the BIGGEST difference is going to be the casing, chambering, and pressure. It’s like how people talk about 5.56 and .223 being the same. They are the same only at face value of size and clambering, but very different on rated pressures. A rifle rated to shoot .223 should only shoot .223. A rifle rated to shoot 5.56 is rated for a higher pressure and can shoot both.

It’s also kinda funny how the most common rifle rounds are smaller calibre than 9mm, so if you ignore all the other metrics, most handguns shoot “bigger” bullets than most rifles.

True, but this is also why “stopping power” is then measured in ft-lb (foot-pounds) or joules for metric. Pistols may shoot bigger rounds, but at significantly slower speeds. Less powder for recoil control and less barrel length for pressure build up. The bigger size bullet is a compensation for the smaller gun. A rile with a smaller round has far more energy behind it’s round.