Sounds like mildew. Wipe with a hot, bleachy cloth or use an old toothbrush dipped in bleach for the hard-to-reach bits, then wipe over to remove residue and allow to dry. To stop it building up in future, wipe condensation off windows and frames each morning and allow air to circulate (keep a window open a crack and don't put stuff up against the window). If it's for odd angles or high up, Farmers has a gel that comes with a brush attached that's pretty good.