Probably the nickname for the house "the old palace" refers to its connection with Bishop Blythe, as in a Bishop's Palace. From Armitage's 1910 "Chantry Land":
There is a reproduction of a watercolor on page 42 which is the same house that Botham painted, information about the picture reads:
House, demolished probably in 1810, which stood a little to the south of Norton House. From a water-colour drawing, copied, about 1877, by the late Miss S.E. Addy, from an original in the possession of the Rev. H.H. Pearson. The copy belongs to Mr Sidney Oldall Addy, M.A., by whom permission to reproduce was kindly granted.
And from the text (page 41 following reference to the Blythes of Bishops' House):
William Pearson, once curate at Norton, writing to his brother Henry, the vicar, says : "The Blythes lived once in an old house just in front of Mr.Holy's [Norton House] near where the yew tree now stands. Mr.Read used to have a picture of the remains of it. It was pulled down some forty or fifty years ago." Again, the Rev. Henry Pearson has a note that the Blythes lived in an old house pulled down by Mr.Read in front of Norton House near an old yew tree.
Thomas Asline Ward mentions the Norton habitation of the Blythes, for in an allusion in the autumn of 1809 to Norton House, where he used to visit John Read, he says : "The prospects from it will be greatly improved by taking down the old houses in front, which will be done in spring. One of these will be much regretted by Chantrey ; and you will, I dare say, join in the regret. He has frequently admired and sketched it. A Bishop Blythe, who is said to have been born at Norton, is reported to have erected it for the residence of his honest but homely parents." Thus apparently the house was removed in 1810.
Mr.Addy has a water-colour drawing of an old timbered house which, before it was destroyed, stood opposite the south front of Norton House. Probably this is the house mentioned by Asline Ward, and it may be one of those enumerated by Mrs.Sterndale in the quotation from her book which appears in the previous chapter.