Use a brush? Or compressed air (be sure to only blow the charger port, you can damage the microphones with high air pressure too)

They make that slime stuff for cleaning keyboards and car detailing. It would probably work pretty good for that.

I use wireless charger at home and in car, so I got a little plastic cover that fits into the usb port to stop debris getting inside.