Almost half of Brits have never visited a Scottish Castle