Yes, I've lived here, for many years and for sure, it's not for everybody. But, listen people! Your visit here doesn't have to have a thing to do with land-based anything..For years, I have been in the boat charter business here. There are few destinations at such a tropical latitude and all the joys that can bring, which are more easy to get to. Why not use that access, actually never leaving US soil except for the altitude to fly, get on a boat and see the REAL St. thomas and St. John. Surly taxi drivers and store clerks (granted, more than in recent years--maybe due to unusually demanding tourists?) have nothing to do with your time in and on the sparkling waters surrounding our islands. Why throw the proverbial baby out with the bathwater? You will not meet any cruise ship "sheeple" this way...in fact, you might not meet ANYBODY if that is your preference. You can enter the BVI and do the same thing....nothing at all! if you wish.
Use it for what it is, a bustling port, with workable airport, lest we forget the quick stop to purchase your duty free liquor and diamond at internationally reasonable prices. Or better yet, just transfer straight back to the airport with your fond memories of some of the most beautiful waters and beaches in the world and big plans to hightail it back here next year.