Even in the hottest U.S. housing market in more than a decade, new home construction has turned into a frustratingly uncertain and costly proposition for many homebuilders.