Vicary Gibbs was called to the bar in 1780 and joined the Western Circuit in 1783. The following year he married Frances Cerjat Kenneth Mackenzie whose father was Major William Mackenzie.  Their only daughter Maria was born in November 1785. He steadily rose in the legal profession and Hayes became a rural retreat away from his London property.  He made his name as a barrister with the defence of a number of high political cases. There is no doubt about his great skill as a lawyer but his sharp tone earned him the nickname of Vinegar Gibbs, although some of the family’s friends referred to him as ‘Gibberish Vicks’.
Arrival in Hayes
In 1790 he lodged with his wife, daughter, 3 men and 3 maid servants in the large farmhouse of Lady Farm near Pickhurst Green, Hayes.  The farmer at the time was Richard Hutson who also  lived there with his daughter, three of his sons and one man servant. Richard Hutson had died by 1793 and Vicary took over all the farmhouse until 1797 when he purchased the house on the Common, later to be known as Hayes Court. He also bought the Nest which he rented out to Mrs Margetson and John Nisbet’s house which he demolished. It had been a boarding school for at least 16 years and in 1798 there were 41 boarders.
He incorporated the land into his garden which had already been altered by the agreement to a diversion of a road which he felt was too close to his house. He extended his property in 1798 by exchanging a piece of land he had purchased in the south east of the Parish near Round Oak for 2 acres of the Common. The income from the land which the Parish received was used to support the Parish poor. 

Vicary Gibbs also helped the poor on a number of occasions.  For example, in 1798 he gave £1 10s. 0d.(£1.50) for James Coomber who had lost his eye and was disabled from work and a similar sum in 1803 to provide clothing for Christopher Smith and his nine children. He was one of the five trustees of the Charity School set up in 1791 to whom the Hon. George Legge, commonly called Lord Viscount Lewisham, conveyed the charity school and its grounds. He continued as trustee until his death in 1820 making an annual contribution of £3.3.0 (£3.15).

In 1798 it was recorded that there were 15 people in the house including 4 men and 4 maids and Henry Blake a labourer, his wife and one child.  Nearby in another house were the gardener John Mackenzie his wife and 5 children.

As his career progressed the house was extended several times, particularly after 1804 when the family took responsibility for the children of his sister-in-law Helen who died in 1802 at the age of 34.  Her husband Major Alexander Mackenzie Fraser was fighting in the Napoleonic Wars but sadly died in 1809.
Vicary bought other properties including the Grove in 1816 which after his death was given for her lifetime to Marianne Fraser, the eldest daughter of Alexander and Helen Mackenzie Fraser. 

William Pitt, the Prime Minister, arranged for Gibbs to become solicitor-general in 1805 and he was knighted.  In 1807 he was made attorney general, a post he held for just over five years.  It was this period of his life that the family at Hayes found most trying and there was great rejoicing when he resigned his post to be a judge in the court of common pleas rising to chief justice of the Common Pleas in 1814. Life proceeded more smoothly for the household although he suffered some ill-health and retired in 1818. He had planned some further changes at Hayes Court to its outbuildings when he died in 1820.

 He was buried in Hayes Churchyard and a very fulsome tribute was made to him by Lord Lyttleton which was recorded on his large memorial on the Chancel Wall. It is now on the North Wall.

In every situation he displayed an understanding clear, vigorous, and acute … Charitable without ostentation, kind and affectionate in his domestic relations he lived and died, a firm friend to the Constitution of this Country both in Church and State.