Background: Endoscopic surveillance of Barrett’s esophagus (BE) is probably not cost-effective. A sub-population with BE at increased risk of high-grade dysplasia (HGD) or esophageal adenocarcinoma (EAC) who could be targeted for cost-effective surveillance was sought. Methods: The outcome for BE surveillance from 2003 to 2012 in a structured program was reviewed. Incidence rates and incidence rate ratios for developing HGD or EAC were calculated. Risk stratification identified individuals who could be considered for exclusion from surveillance. A health-state transition Markov cohort model evaluated the cost-effectiveness of focusing on higher-risk individuals. Results: During 2067 person-years of follow-up of 640 patients, 17 individuals progressed to HGD or EAC (annual IR 0.8%). Individuals with columnar-lined esophagus (CLE) ≥2 cm had an annual IR of 1.2% and >8-fold increased relative risk of HGD or EAC, compared to CLE <2 cm [IR—0.14% (IRR 8.6, 95% CIs 4.5–12.8)]. Limiting the surveillance cohort after the first endoscopy to individuals with CLE ≥2 cm, or dysplasia, followed by a further restriction after the second endoscopy—exclusion of patients without intestinal metaplasia—removed 296 (46%) patients, and 767 (37%) person-years from surveillance. Limiting surveillance to the remaining individuals reduced the incremental cost-effectiveness ratio from US$60,858 to US$33,807 per quality-adjusted life year (QALY). Further restrictions were tested but failed to improve cost-effectiveness. Conclusions: Based on stratification of risk, the number of patients requiring surveillance can be reduced by at least a third. At a willingness-to-pay threshold of US$50,000 per QALY, surveillance of higher-risk individuals becomes cost-effective.