Despite its widespread clinical use, there is little data available from population-based studies on the determinants of serum sex hormone binding globulin (SHBG). We aimed to examine multifactorial determinants of circulating SHBG levels in community-dwelling men. Study participants comprised randomly selected 35–80 y.o. men (n = 2563) prospectively-followed for 5 years (n = 2038) in the Men Androgen Inflammation Lifestyle Environment and Stress (MAILES) study. After excluding men with illness or medications known to affect SHBG (n = 172), data from 1786 men were available at baseline, and 1476 at follow-up. The relationship between baseline body composition (DXA), serum glucose, insulin, triglycerides, thyroxine (fT4), sex steroids (total testosterone (TT), oestradiol (E2)), and pro-inflammatory cytokines and serum SHBG level at both baseline & follow-up was determined by linear and penalized logistic regression models adjusting for age, lifestyle & demographic, body composition, metabolic, and hormonal factors. Restricted cubic spline analyses was also conducted to capture possible non-linear relationships. At baseline there were positive cross-sectional associations between age (β = 0.409, p<0.001), TT (β = 0.560, p<0.001), fT4 (β = 0.067, p = 0.019) and SHBG, and negative associations between triglycerides (β = -0.112, p<0.001), abdominal fat mass (β = -0.068, p = 0.032) and E2 (β = -0.058, p = 0.050) and SHBG. In longitudinal analysis the positive determinants of SHBG at 4.9 years were age (β = 0.406, p = <0.001), TT (β = 0.461, p = <0.001), and fT4 (β = 0.040, p = 0.034) and negative determinants were triglycerides (β = -0.065, p = 0.027) and abdominal fat mass (β = -0.078, p = 0.032). Taken together these data suggest low SHBG is a marker of abdominal obesity and increased serum triglycerides, conditions which are known to have been associated with low testosterone and low T4.