Estimating relative velocity in the natural environment is challenging because natural scenes vary greatly in contrast and spatial structure. Widely accepted correlation-based models for elementary motion detectors (EMDs) are sensitive to contrast and spatial structure and consequently generate ambiguous estimates of velocity . Identified neurons in the third optic lobe of the hoverfly can reliably encode the velocity of natural images largely independent of contrast , despite receiving inputs directly from arrays of such EMDs [3, 4]. This contrast invariance suggests an important role for additional neural processes in robust encoding of image motion [2, 5, 6]. However, it remains unclear which neural processes are contributing to contrast invariance. By recording from horizontal system neurons in the hoverfly lobula, we show two activity-dependent adaptation mechanisms acting as near-ideal normalizers for images of different contrasts that would otherwise produce highly variable response magnitudes. Responses to images that are initially weak neural drivers are boosted over several hundred milliseconds. Responses to images that are initially strong neural drivers are reduced over longer time scales. These adaptation mechanisms appear to be matched to higher-order natural image statistics reconciling the neurons' accurate encoding of image velocity with the inherent ambiguity of correlation-based motion detectors.