Tucked out of sight behind Highgate High Street, you could pass right by this evocative gothic brick folly and not even know it was there.

Set between two modern office buildings lies an innocuous Victorian stone building no bigger than a large garden shed.

For years, it sat untroubled while workers, Highgate residents of many decades, and even the landowner remained unaware of its existence.

It was not until recently that 40-year resident and Highgate Society member Michael Hammerson took note of the one-up, one-down structure hiding between Broadbent Close and Townsend Yard, just off High Street.

He said: “When I first noticed it, I said ‘what the blazes is that!’ It’s not characteristic of Highgate or anywhere in Middlesex, it’s extraordinary.”

Mystery surrounds the little building, which would not look out of place in a countryside village – and Mr Hammerson is mystified as to what the structure was and why it was built.

“It was built with all sorts of materials,” he said. “My theory, and it is only a theory, is that it was built by the Townsend family.”

They were a local aristocratic family whom Townsend Yard was named after. Mr Hammerson added: “When the Jacobean Ashurst House in Highgate West Hill was torn down, they may have taken materials to use.

“When they ran out of stone, they then started on brick and then, when they ran out of that, they started using clinker [cement] material.”

Christopher Cansick, who owns the land the building sits on in Townsend Yard, said he never knew it was there as it is only visible from Broadbent Close, despite belonging to the land in Townsend Yard.

He said: “It sounds extraordinary but I never looked at it at all. It’s quite interesting. I haven’t the faintest idea what it is.”

Attached to the stone structure is a mast, which pirate radio stations have allegedly been using to illegally transmit from for about 10 years.

Mr Hammerson wants to see the strange little building protected from any potential future developments of the land it sits on.

“It could make a nice cafe or an art gallery, or anything really, so long as it’s exposed and restored,” he suggested.

Mr Cansick, director of export merchanting company Mark Cansick & Co in Townsend Yard, disagreed.

“To talk about using it as a cafe is looking through rose-tinted spectacles,” he said.

“It’s no bigger than a garden shed. A large garden shed but a garden shed no less.

“If the car park is developed in the future then maybe, but I have no intention of changing it. It can stay there as a memorial to the past.”