Neural adaptation paradigms have been used in the electrophysiological and neuroimaging literature to characterise neural populations underlying face and object perception. It was recently reported by Nemrodov and Itier (2012) that adaptation of the N170 event-related potential (ERP) component is not stimulus category-specific over rapid adapting stimulus durations (S1 durations) and interstimulus intervals (ISIs). We therefore tested the category-specificity of adaptation over a range of S1 durations and ISIs. Faces and chairs were presented at S1 (for 200, 500 or 1000. ms) and S2 (for 200. ms), over a variable ISI (200 or 500. ms). Mean amplitudes of the P1, N170 and P2 visual ERP components were measured following S1 and S2 stimuli. Faces at S1 led to the smallest (i.e., most adapted) N170 amplitudes to both faces and chairs at S2, more than chairs at S1. N170s at S2 were smallest after a 500. ms S1 duration; but N170 amplitude did not vary over ISI. Effects were also seen for the two surrounding positive components, the P1 and P2. Presenting faces at S1 led to enhanced P1 amplitudes evoked by S2 chair stimuli. The P2 showed the smallest amplitudes following the shorter 200. ms ISI. These results indicate that adaptation of the N170 is not actually category-specific but instead dependent on the S1 category (regardless of S2 category), and may also be influenced by earlier effects at the P1 (i.e., not specific to the N170). This challenges the assumption that N170 category adaptation indexes effects on distinct neural populations that differ between faces and non-face objects.