In this paper, we explore and describe what is needed to allow connected and automated vehicles (CAVs) to break traffic rules in order to minimise road safety risk and to operate with appropriate transparency (according to recommendation 4 in Bonnefon et al., European Commission, 2020). Reviewing current traffic rules with particular reference to two driving situations (speeding and mounting the pavement), we illustrate why current traffic rules are not suitable for CAVs and why making new traffic rules specifically for CAVs would be inappropriate. In defining an alternative approach to achieving safe CAV driving behaviours, we describe the use of ethical goal functions as part of hybrid AI systems, suggesting that functions should be defined by governmental bodies with input from citizens and stakeholders. Ethical goal functions for CAVs would enable developers to optimise driving behaviours for safety under conditions of uncertainty whilst allowing for differentiation of products according to brand values. Such functions can differ between regions according to preferences for safety behaviours within that region and can be updated over time, responding to continual socio-technological feedback loops. We conclude that defining ethical goal functions is an urgent and necessary step from governmental bodies to enable the safe and transparent operation of CAVs and accelerate the reduction in road casualties they promise to achieve.