Screws maybe

But you have to machine a hole in the metal for the screw to function

You know how hard it would be to drill a hole for a screw in a cave with scrap parts and no power drill?

All you need to crudely weld, probably as strong as a screw, is heat. Fire is easy in a cave.

I’d wager they could melt some metal together before devising a cave drill press.

We have no idea what metal or alloy it is. They could have an alloy that melts easily, but once hard, it adheres to other metals and practically unbreakable. Just find some scraps of that alloy to easily weld.