Interesting as I haven't heard of Swift before. This old Georgian cottage dates from around 1760 and was built on land adjoining Flat Croft which became part of the graveyard. By 1848 the land for the graveyard was in the possession of the Whitehouse family who owned and farmed much of the local land. In particular, it was owned and occupied by John Whitehouse. He would go on to build a terrace of hovels for the poor known as Whitehouse Cottages on the same land. Subsequently demolished, I could still find bricks from the demolition in the bottom corner of the cemetery in my younger days.
However, the thatched cottage is scheduled in the 1948 Tithe Map as under the ownership of Rev John Holden Harrison, whom I assume to be the replacement for Rev Hyla Holden Rose (deceased); and the occupier is listed as William Wilkins. Moving back to the 1941 census reveals Thomas Wilkins & family, whilst the 1891 census finally reveals Joseph Twist (gardener aged 62) & sister. Maybe 'Twist' has been remembered as 'Swift'.
Finally, the local historian N C Meacham recalls the thatch being removed from the cottage as a WWII precaution. However, I'm convinced that it was reinstated after the war, and that is how I remember the cottage. As a child I had a saying: "Moor End Lane has two old cottages - one is thatched but not half-timbered (this one), and one is half-timbered but not thatched (Moat Cottage)". I subsequently remember the cottage with its tiled roof as pictured, bu I can't recall when the transition took place.