Building evaluation capacity in community organisations is promoted as a key means of creating an evaluation culture that will reorient organisations towards identifying and measuring outcomes, demonstrating achievements and enhancing accountability. Building evaluation capacity presents a number of challenges for micro-organisations, including limited resources for evaluation, a low existing evaluation skills base, and, an organisational culture and practice that does not align well with evaluation tools and techniques typically used to demonstrate program outcomes. The implementation of the Thriving Communities Healthy Families pilot project in Tasmania, a place-based health promotion intervention delivered through Neighbourhood Houses (NH), included an evaluation capacity building (ECB) component. The pilot project is used in this article as a case study to illustrate the tensions that arose for community development workers in the NH as they implemented a new program, and contended with the demands of developing their evaluation capacity. We found that while community development workers value evaluation and the importance of demonstrating program outcomes, the model of building evaluation capacity of the workers themselves was burdensome. Further, some workers perceived that evaluation activities impacted negatively on their primary role of supporting vulnerable community members. We conclude that, in very small community sector organisations, ECB requires an ongoing mentoring/relationship-building approach, rather than a traditional training approach. This has implications for how to fund ECB and raises questions on where best to focus ECB in these micro-organisations.