The third Duke married twice.

Caroline Harvey

• Born: 11 December 1823
• Daughter of Robert Harvey of Langley Park, Buckinghamshire
• She married the future 3rd Duke on 2 October 1851 in St Mary's Church, Eton. The service was conducted by Edwin Hotham. The witnesses were: Robert Harvey N. Gore Langton; Anne Harvey Robert Bateson Harvey; G. Neville Grenville Thomas Baldwin; William Hotham; and Seymour Neville Thomas Bateson. (BRO D10. AR105/82/1/1). See: Buckinghamshire Advertiser commentary on the marriage. 
• Children:
  • Mary (30 September 1852 - 17 October 1944)
  • Anne (25 October 1853 - 20 March 1890)
  • Caroline Jemima Elizabeth (11 April 1858 - 25 May 1946)
• Died 28 February 1874.

Portraits

• Portrait of Caroline Harvey at Stowe

Alice Anne Montgomery

• Born 29 September 1847
• Eldest daughter of Graham Montgomery and Alice Hope-Johnstone
• Author and painter
• Marriage to third Duke 17 February 1885 
• No children
• Married 8 August 1894: Wilbraham Egerton, 1st Earl Egerton of Tatton (17 January 1832 to 16 March 1909)
• Died 15 September 1931

Portraits

• Portrait of Alice Anne Montgomery at Stowe
• Portrait by Dicksee