I use paper buildings, from "susaeta"
http://www.editorialsusaeta.com/resultados.phpThey are very cheap, easy to assemble, and it is funny doing it

and a lot of variety, and I am sure that there are a lot of paper manufacters out there
Some buildings like village set are quite big, like 1:72 scale, but just cutting the first floor you get excellent 10mm buildings. (figure 1)
For castle walls I do not assemble and get walls sections to be adaptable for differents games. It is true that if the walls sections are not glue together they do not support the weight of WM units, so something has to be done. (figure 2)
If you do not like paperwork I have the " medievla Fortress cod 72004" from Miniart and it is really cool, hard plastic. It is to 1:72 scale but for fantasy it fit really good. Maybe a little tall for historic. (If you are interested in some pics just tell me and I take some for you, I did not assemble the fortress yet

)
http://www.modelhobbies.co.uk/shop/miniart-medieval-fortress-72004-p-1725.htmlAnother posibility is zvedza (1:72) hard plastic and really nice stuff
http://www.hobbiesguinea.com/default.php?cPath=152_133_66_190_709&osCsid=c02275fdca0bd7c5025c6cfa3752fe61Also Zvedza and miniart do their fortress like modules to assemble and upgrade the castles