Try solvents (white spirit, brake cleaner, iso propanol, aerosol oil, hand cleaner) before going for aggressive chemicals.
Alkaline chemicals and aluminium should never be mixed if you want to use the aluminium bits again.
There are paint pens available as well (eg Edding 8750 or 751) - very useful covering scratches etc and for colouring in components (like the center bolt of hope headset top caps). Also very useful for black spoke nipples that lose their colour very readily during wheel repairs/adjustments.
The Edding paint pens are soluble in brake cleaner if you don't like the finish or it ends up going somewhere it shouldn't.