You could do a compressed model in 1:48 and it would still be impressive as the dickens. The cathedral on my layout is closer to 1:64 scale than 1:48, and compressed at that, yet is is a BIG building, and impressive, because in the real world, it was an even bigger building.
Hogwarts would be a lot of fun to build.