Some of the disgraced financier’s victims have said they were trafficked at the property south of Santa Fe, where the nearest neighbors are miles away.