In this article I would present evidence to show that timing of the flow of neural signals within the ventral visual stream is a crucial part of the neural code for categorization of faces. We recorded the activity of 554 inferotemporal neurons from two macaque monkeys performing a fixation task. More than 1000 object images including human and non-primate animal faces were presented up to 10 times, each for 105 ms without intervals. About one-third of the cells had selective responses to both human and non-primate animal face categories. The onset latencies were significantly shorter for responses to human faces than those to animal faces (P<0.0001, paired t-test, mean difference = 22 ms). In our results a relatively large number of face cells could not differentiate category of faces by rate code alone. Spike timing improves the neural information for discrimination of different stimuli that evoke similar firing rates. Response latency could play a role in organizing information about complex visual scene by perceptual segregation of object categories within the scene and by gating information flow in neural assemblies involved in different aspects of face recognition.