Andrew Berkeley Drummond was born on 13 September 1755, the eldest son of Robert Drummond and his wife Winifred Thomson. He came from a Scottish banking family, but was brought up in London and educated at Winchester School. In 1781 he married his cousin, Lady Mary Perceval. Her younger brother, Spencer Perceval, was Prime Minister of the United Kingdom from October 1809 until his assassination in the lobby of the House of Commons in May 1812. 

It is unclear why Andrew Berkeley Drummond chose to take out fairly short leases on properties in Hayes, but his first lease was in the Pickhurst area which he left in 1784. He then moved to Hayes Court, which was empty in early 1785, and in 1790 Revd John Till recorded living at the property “Alexander Drummond Esq, Lady Mary, 4 men and 3 maids when the family is here, at other times only one maid”.

By then, Andrew Drummond had become a partner in the family banking firm, Messrs Drummond. The Bank traded at the sign of the Golden Eagle on the east side of Charing Cross, a residential area favoured by the Scottish gentry in London. Messrs Drummond were highly successful during the last quarter of the eighteenth century, their banking clients included King George III, English and Scottish aristocrats, important architects, artists, composers and craftsmen of the day. King George III dealt directly with Andrew Drummond concerning his finances.

Andrew Drummond does not seem to have been involved with taking part in the Vestry or other parish activities and by 1794 the leaseholder at Hayes Court was Revd Edward Lockwood, who may have been related to him. Andrew Drummond moved away from Hayes and, on his father’s death in 1804, he became head partner of Messrs Dummond and inherited his father’s country estate at Cadland, near Fawley, in Hampshire.