Colonel Thomas Marshall (1730-1803) received this property as part of a land grant signed by Virginia Governor Patrick Henry. Marshall was a personal friend of George Washington whom he served with as a Lieutenant in the French and Indian War. Marshall later served in the Revolutionary War, was at Valley Forge and rose to the rank of Colonel. After the war he was the first surveyor general for most of Eastern Kentucky which was then called Fayette Country Virginia. He moved to moved to what is now Woodford County Kentucky to fulfill that position.  He was at the Danville Convention of 1788 which established Kentucky as a separate state from Virginia and the first state West of the Appalachian Mountain Chain.

Colonel Marshall was married to Mary Randolph Keith (1737-1809) and together they had fifteen children reach their adulthood. The best know of these children is their oldest, John Marshall, who became the most important and longest serving chief justice of the US Supreme Court. The second son, Captain Thomas Marshall, also served in the Revolutionary War before coming to Kentucky and settled in what is now Mason County. His father is thought have helped him finance the house he built known as Federal Hill. In 1800 Colonel Marshall and his wife moved to Federal Hill to be with Captain Marshall and his family. Colonel Marshall died in 1802 and he and his wife Mary are buried in the Marshall Family Graveyard behind the house. At his death Colonel Marshall owned over 60,000 acres of land of which he willed at least 14,000 acres to his son Captain Marshall.

Captain Thomas Marshall (1761-1817) served during the Revolution and married Frances Maitland Kennan. He moved to Kentucky in 1789 and he and Frances lived at Federal Hill when it was completed. He had seven children and served as Clerk of Mason Ct. in addition to being a large farmer and land developer. In addition to the main house, he had a brick building built next to Federal Hill which was used at one point to sell land and at another as the County Clerk’s Office. One of his sons, General Thomas Marshall (1793-1853) lived in Lewis County near Tollesboro KY.  He was a general in the Mexican War and is also buried in the Federal Hill Cemetery.  Captain Marshall willed Federal Hill to his daughter Elizabeth or Eliza Coleston Marshall.

Eliza Marshall (1801-1874) married her cousin Martin Picket Marshall (1798-1883). They started living in the house soon after their marriage and spent the rest of their lives there. Martin Marshall was a Unionist, friend of Henry Clay and favored gradual emancipation of the slaves. He was originally a lawyer but then became a gentleman farmer. In his will he left Federal Hill to his daughter Mary (1829-1908) along with around 150 acres of land. For his daughter Phoebe (1842-1915) a second house was built close to Federal Hill and she was also given a modest land holding. (That second house was allowed to deteriorate badly and has now been knocked down for safety reasons.) Neither Mary nor Phoebe had surviving children.

On Mary’s death Federal Hill was purchased by Louis Marshall M.D (1873-1910)., a cousin and practicing physician. He married Pearl McGhee (1877-1925) and they had one daughter Elizabeth. Louis Marshall. Dr. Marshall died at age 37 from a disease he had contracted from one of his patients. Elizabeth Louis Marshall (1911-75) was born after her father’s death and was left an orphan. at age 14, after her mother’s death She inherited Federal Hill where she lived almost all of her life. She married Virgil Thomas Fryman, a school teacher and farmer, and they had one son, Virgil Thomas Fryman Jr who inherited the house on his mother’s death. She and Virgil are buried in the Marshall Family Cemetery.

Virgil Thomas Fryman Jr (1940-2014)., known as Tommy, was a Harvard Law Graduate and Senior Partner in a Lexington Law Firm. He married Lisa Fiedler and they had three daughters, Elizabeth, Anne and Julia. Tommy passed in 2014 after a long illness and is buried in the family grave yard. All of the daughters are now grown. live out of state and are not interested in living in Federal Hill. Therefore, Lisa Fiedler Fryman donated the house to the Wallingford Foundation in October 2023. The Wallingford Foundation has launched a project to repair the house which was in bad condition since it had been vacant most of the time since the death of Elizabeth Louis Marshall Fryman in 1975.