"Downtown" is the Charlotte Amalie area. Men can wear any shorts that they like, but must have on a top (cannot be without a shirt). Women cannot wear their swimsuit, unless covered up by shorts/skirt and a T-shirt or tanktop. Basically, think of it this way: Beachwear is for the beach. Don't you think you'd feel uncomfortable walking around in downtown New York with a bikini on? Not acceptable here, either. Other parts of the island are fine, just not in town.