Castles could include a warren of houses and shops, as well as its tradional parts shown here. The houses and shop could be built against the castle walls, or if there was enough space in the center, there.
This is the basis for the Woodhall complex. At this stage we have the watch-towers and the mediaeval hall, but laid out differently. There is no castle-like tower, but a much more modern building. I do like that Gatehouse, though!