Isaac Cory built the inn, originally a general store, in the late 1820’s in order to move the expanding business from his home up the road. It was central to the commercial life of Westport Point. The store was on the first floor, a milliner’s and tailor shop were on the second, and the third floor was a sail loft and a rigging storeroom. There was also a post office on the premises from 1840 to 1887. In 1950 the building was converted to an inn. The property is listed on the National Registry of Historic Places.