Radio Shack sells " contact cleaner" some come in bottles with a bristle brush attached to help scrub away gunk. the contact cleaner wont harm the plastic, but makes quick work of grime.
you may wnat to use a tooth pick or such to get down in there also and scrape away.
also the connectors can sort of self clean by inserting and removing an opposite (male) connector many times, this will wipe away the dirt. do this in conjunction with the radio shack contact cleaner,
get it wet and work the male end in and out until your worn out.