I skipped the PS2 gen entirely when I was a kid (went from PS1 to X360), so I don’t have fond memories either, but I have a few titles in my bucket list that I’ve played/been wanting to play for a while now.

• Since you liked Shadow of the Colossus, you should try Ueda’s other PS2 game, Ico. It’s very sweet.

• As a huge Ace Combat nerd, I also recommend the PS2 trilogy (Ace Combat 04, 5 and Zero). My favourite of the three is 04, which has a more grounded and somber story and consistently good gameplay. 5 is like a Tom Cruise action movie, but I find the gameplay subpar compared to the others. Zero is like the MGR: Revengeance of Ace Combat lol

• Zone of the Enders 2 is my favourite mecha game ever, with exhilarating battles, an awesome soundtrack and great set pieces (the Crevasse battle is absolutely humongous and I still can’t believe that nothing similar was ever replicated on consoles that are thousands of times more powerful than a PS2).

• If you like mascotte platformers, I’ve heard great things about Ratchet and Clank, Sly Cooper, Jax. I’ve only played a few levels of the former and it gave me huge motion sickness, though :/
  There’s also Klonoa, but i think that one’s got a remastered version on modern platforms nowadays.

• People mostly remember the Final Fantasy titles, but there are a LOT of JRPG games on PS2, although I haven’t played them myself and I can’t attest to their quality (sometimes people’s nostalgia is just… Off the charts lol).
  Star Ocean: Till the end of times is consistently praised among both the Star Ocean community and the larger JRPG crowd. I can’t attest to that as I don’t have nearly enough time to play a JRPG title nowadays, but I liked the PS1 title (2nd story), and I’ve always heard people putting it above or slightly below that one, so it looks like I’m missing out.
  I’ve also heard good things about other titles such as Rogue Galaxy, Ar Tonelico, Grandia 3, Legaia 2 (I’m very fond of the first one, although the story was nothing to write home about), Wild Arms, a few Tales Of titles, and probably many more I’m forgetting about. Maybe look out for some of those if you’re into the genre.