After 1804, New Castle Presbyterian stopped allowing burials in the small churchyard next to the church on Second Street and purchased a plot of land to the west of town.