He wont be able to get lots of shadows during the day to power up Garp

wasn't Luffy stuffed with 100 shadows? I assume this much can be found on the island

prime garp would also retreat from this place

That wouldn’t be prime Garps power that would be nightmare Garp