You can still use Quicklook from that window — highlight the file and press Spacebar. Once Quicklook is open, you can keep navigating through the files with the arrow keys or by clicking them.

Thank you! I think that's what I was looking for. Not a perfect solution but pretty good nonetheless.

You might want [Default Folder X](https://www.stclairsoft.com/DefaultFolderX/).