Purpose: To determine whether there is an association with a congenital visual or hearing impairment (VI or HI) and Autism Spectrum Disorder (ASD) in children. Methods: A systematic literature review was performed using nine relevant databases limited to peer reviewed articles in English between 1994 and January 2016. The search identified 1248 articles after duplicates were removed with subsequent screening of the abstracts excluding a further 1199, resulting in 49 full-text articles that were then independently assessed by five of the authors with a final 15 articles meeting the inclusion criteria. Bias assessment was determined through consensus of the first five authors. A meta-analysis of the included studies was performed to estimate the relative risk of ASD in the VI and HI groups compared to the general population based on reported prevalence rates in similar geographical regions. Overall prevalence rates for ASD were calculated from the combined studies in the VI and HI populations. Results: The overall prevalence of ASD in VI and HI populations was 19% (95% CI 13–25%) and 9% (95% CI 6–12%) respectively. The overall risk-ratio of ASD was greater in the VI 31.0 times (95% CI 18.62–51.56); z = 13.21, p < 0.001) and HI groups 14.1 times (95% CI 3.41–58.62; z = 3.65, p < 0.001) compared to reported ASD prevalence in the general population. Conclusion: There is a high association of ASD in VI or HI children and therefore these populations should be assessed for ASD in the presence of a visual or hearing disability.