There was an Augustinian friary on the site from the thirteenth century, then an almshouse for housing retired freemen, then a soup kitchen was built next to Almshouse in the nineteenth century, before the site acquired its current function as a working office.

The building also serves as the basis of the Inner City Project of the National Trust. This project takes people of ages 12–25 and over 50 out to the countryside in order to increase appreciation of the city's natural surroundings.

It was nearly demolished in the 1960's when massive redevelopment took place. The central motorway comes within inches of its walls.