The risk that flexible forms of employment are harmful to the health of workers is a major public health issue for the many countries, including Australia, where such forms of employment are common or have been growing. Casual, contract and part-time employment in Australia rose rapidly in the decade to 1998 and remains high at 40% of employees in 2011. We investigate the impacts on mental health of employment on these terms and of unemployment. We use nine waves of panel survey data and dynamic random-effects panel data regression models to estimate the impact on self-rated mental health of unemployment, and of employment on a part-time, casual or contract basis, compared with permanent full-time employment. We control for demographic and socio-economic characteristics, occupation, disabilities status, negative life events and the level of social support. We find almost no evidence that flexible employment harms mental health. Unemployed men (but not women) have significantly and substantially lower mental health. But among the employed, only men who are on fixed-term contracts, most especially graduates, have lower mental health than those who are employed on full-time permanent terms. Women have significantly higher mental health if they are employed full time on casual terms.