The property, which was briefly on the market for $180 million in 2020, has belonged to the Whitney family for more than a ...