I have no documentation that Mark Twain spoke here, but ...

Diefendorf Hall stands among a row of buildings along Main Street in the Village of Fort Plain. The building is historically significant for hosting Women’s Rights Movement leaders Susan B. Anthony and Elizabeth Cady Stanton in 1867. Stanton addressed the assembled audience on “Universal Suffrage.”

Diefendorf Hall was built circa the early 1860s. An 1868 Atlas of Montgomery and Fulton Counties, New York shows that Diefendorf Hall was aptly owned by John I. Diefendorf. The 2012 National Register of Historic Places nomination form for Fort Plain Historic District describes the building as a “two-story, five-bay, brick, flat-roofed, commercial building with two storefronts: one two bays wide and the other three bays wide.” It is believed to have been utilized for many purposes including a distillery, music hall and general shops (Mohawk Valley Villages website, 2013).

https://www.wgpfoundation.org/historic-markers/diefendorf-hall/