Sounds like you got it, more or less. Current views of how object recognition works in the brain are a lot like current deep net models of object recognition (e.g. Alexnet and beyond): a heirarchical series of processing steps in which units at successive processing stages get more selective for specific things, and more invariant to image variation (size, position, lighting, etc). One view of holistic face perception is that it is just the natural consequence of having units tuned to whole faces (or to large portions of the face). But why this should be implemented in humans as a specific category-selective patch of the brain is an open and fascinating question, that I am now hopeful network modeling may inform.