Chipping Ongar was one of the first Conservation Areas to be designated by the County Council and contains over 100 listed buildings and other structures. The town was the administrative centre of the Saxon Hundred and the fortress established during this period was enlarged after the Norman Conquest. The town became known as Castle Ongar in the 12th century because of the "motte and bailey" castle, the remains of which still survive. The line of the former medieval town enclosure can also be traced to the north and south of the inner bailey, around the Pleasance car park and along Castle Street.
The oldest surviving building in the town is St. Martin's Church, the chancel and nave of which date from the 11th century. The White House and Castle House are the largest houses in the town which date from the 16th century. The only other building from this period which still stands within the town enclosure is the Old Market House - No. 171 High Street - which housed the market in the 1840's.
The form and layout of the historic core of the town have changed little since the medieval period. A weekly market started in the 12th century and the market place is still apparent as the widest part of the High Street between Wren House and Manor House.
The town was an important staging place for travellers from London to East Anglia and by 1848 coaches departed on a daily basis from the King's Head Inn. The railway service to London, which opened in 1865, brought prosperity and change to the town with a wealth of Victorian buildings, including a police station (now demolished), Budworth Hall and the local water works.