Franklin Ellis mentions the "Fountain" or "Fountains" twice in his History of Columbia County, published in 1878. He describes the place itself.
The "Fountain," east of the city limits, is the place of the springs alluded to by ancient writers two centuries ago or more. Here is located the old Hollenbeck hotel, a very ancient tavern-stand. (page 364)He also describes the hotel that was located near the "Fountain."
The present Hollenbeck tavern, near the "Fountains" on the Columbia turnpike, is a very old place. The tavern was kept nearly or quite a hundred years ago by James Bedell. (page 362)It appears that the Hollenbeck tavern was also known, or at sometime became known, as the Fountain Head Hotel. In the decades around the turn of the last century, the Fountain Head Hotel seems to have been the center of civic life in Greenport. The Greenport Town Board met there. Judges heard court cases there. Political parties held their caucuses there. In 1910, forty horses were auctioned at the Fountain Head Hotel, and in 1913, a mammoth hog named Belle was publicly slaughtered there.
The water that came forth out of the ground at Fountain Head seems to have had something of the same appeal to tourists as the waters at Saratoga Springs. On August 14, 1911, the Hudson Evening Register reported that James Adams of New York City, a guest at the hotel, being fond of the water that flows at Fountain Head and daily indulging himself in the cool, clear fluid, "slipped upon the rocks where the water runs and took a bad fall, sustaining a number of bruises."