Builders in Ballarat say the lure of more work and money in Melbourne is taking away the vital workforce needed to meet the government's ambitious regional 2051 housing targets.