So for example.
Here is a Brigade small dwelling.
Looking at the actual sizes and assuming that 1mm = 1 foot, then we can see in scale terms the Brigade building is fine, it makes sense and is plenty big enough for people to live in:

The door is 8` tall by 4` wide, this is much bigger than any regular door so it should look too big when next to a figure.
So they have 'heroe'd' up the door to make it look right even though a true scale door would be smaller.
Here is their perfectly sized small building next to mine:

My small MHU is about twice the size, eeeek too big for a small building no?
But once a figure is based and put next to the excellent Brigade one, suddenly it seems a bit small for a house, and that oversized door even seems too small.

So you can base your buildings to make them 'look' right or you can make big ass oversized ones as I do.
It comes down to taste.
I prefer my buildings and vehicles to 'feel right', (by that I mean to look like they can house the infantry) rather than be true to scale.