Might want to blow out any pocket lint, nose-hairs or other schmutz from the port first before spraying something wet like electrical contact cleaner in there. You can buy canned compressed air at any office supply or computer-oriented store. It's the stuff they sell to clean all the crumbs, sesame seeds and paper clips from under the buttons on a keyboard.
Problem is though, this can drive the offending foreign substances/objects/dust-bunnies further into the device up around the circuit board and scroll wheel area. Not really good either. So be careful or take it apart gently for a more thorough cleaning.