Thanks @enchantedspirit, I actually showed up to their meeting they were having to try to stop me from building. Mostly they wanted to keep the houses small in the neighborhood, under a 1,000sf,maybe that bigger houses didn't fit in, they didn't want the taxes to go up, and I think they wanted to keep out families with kids. They tried to say they were historic houses, but they weren't historic, they were 50 years old track homes and cheaply made without architectural significance. My dad thought it was just jealousy. After I finished the house other people started doing the same thing, or building new houses on the old lots. The neighborhood is very nice now with a mix of old and new houses, different sizes and styles.