A document in the national archive, held on the island, suggests that it was built around 1865. This is a grade II listed building; details of the listing can be found on the English Heritage site, here. According to that page this is an unaltered tollhouse, unlike many others which have been extended and modified - my own tollhouse was more than doubled in size when the extension was built in the 1980s.
There are several mentions of this tollhouse on the Brook Village website. This site suggests the building is older than the document in the archive, but may be referring to an older building maybe? It seems that the house was not a safe place to pass if you were up to no good!
The pictures below were taken in May 2012.