Fairhaven was first settled in 1659 as "Cushnea", the easternmost part of the town of Dartmouth. It was founded on land purchased by English settlers at the Plymouth Colony from the Wampanoag sachem Massasoit, and his son, Wamsutta.

Prior to the second half of the nineteenth century, whale oil was the primary source of fuel for lighting in the United States. The whaling industry was an economic mainstay for many New England coastal communities for over two hundred years. The famous whaling port of New Bedford is located across the Acushnet River from Fairhaven. Fairhaven was also a whaling port; in fact, in the year 1838, Fairhaven was the second-largest whaling port in the United States, with 24 vessels sailing for the whaling grounds. The author of Moby-Dick, Herman Melville, departed from the port of Fairhaven aboard the whaleship Acushnet in 1841.

However, once New Bedford's predominance in the whaling industry became apparent, Fairhaven's economy evolved into one that supplemented the New Bedford economy rather than competing directly with it. Fairhaven became a town of shipwrights, ship chandlers, ropemakers, coopers, and sailmakers. It also became a popular location for ship-owners and ship-captains to build their homes and raise their children.

Among Fairhaven's natives was Henry Huttleston Rogers (1840–1909), who was a businessman and philanthropist. Rogers was one of the key men in John D. Rockefeller's Standard Oil trust. He later developed the Virginian Railway. Rogers and his wife, Abbie Gifford Rogers, another Fairhaven native (who was the daughter of the whaling captain Peleg Gifford), donated many community improvements in the late-nineteenth- and early-twentieth-century, including a grammar school, an extraordinarily luxurious high school, the Town Hall, the George H. Taber Masonic Building, the Unitarian Memorial Church, the Tabitha Inn, the Millicent Library, and a modern water-and-sewer system. These structures were erected to top-quality construction standards, a trademark philosophy of Henry H. Rogers; some are still in regular use more than one hundred years later. The town didn’t maintain the grammar schools and they are left derelict and boarded up, a sad end to his legacy. His grandson was The 1st Baron Fairhaven (1896–1966).