One of the greatest impediments to detecting changes in species distributions in response to ocean warming is the lack of baseline data. In a recent article, we compared old (1940-1959) and new (1990-2009) herbarium records of Australian seaweeds and found a net southward shift in the latitude of northernmost collections of temperate species, implying a flora-wide poleward retreat over the past five decades. Huisman & Millar (2013) criticised our methods, contending that a comparison of herbarium records from different time periods cannot be used to infer changes in species distributions without field-based validation. However, our analysis compared the median position of extreme records of random species from random locations rather than focusing on particular species and their possible loss from specific sites. Hence, ground-truthing 'extinctions' are of limited value to the interpretation of our analysis. Moreover, subtidal ground-truthing over biogeographic scales is not logistically possible and even runs counter to entire disciplines (e.g. palaeontology, extinction biology and biogeography) that assess hypotheses of extinction and shifting distributions. Huisman & Millar also questioned the direction of biases in the data set. We show here that patterns of collection effort should have produced an apparent shift northward in the absence of a true shift southward. Even if herbaria were not designed for the purpose of detecting species' range changes, we contend that such collections can contain useful information on the distribution of species across space and time.