The church records of West Newbury include the comment "Mr. Joshua, being chosen to the office of deacon among us, June 26, 1770, God was pleased to incline his own heart to the office, March 2, 1771." Joshua and Elizabeth Chase Bailey were the parents of eight children born between 1742 and 1766. A daughter Anna, born 17 August 1753, married a Samuel Currier and moved to Berlin, Vermont.

Ebenezer was the last child born to Joshua and Elizabeth Chase Bailey.88