Saginaw is an incorporated city with over 20,000 people living there. There is an office of zoning and planning. I have a number of friends who live there and in all of their cases they are subject not only to the city zoning, but also the HOA zoning of their respective sub divisions. I don't know that the neighborhood in question has an HOA, but city zoning would be in place.
It is true that unincorporated Texas tends to have more lax zoning requirements than incorporated areas. The cities here don't seem any different really than the cities that I lived in when I was in other states. Rules, permits, inspections, code enforcement. I would say the most striking difference to me is the love of fences and the ease in putting them up in most areas of Texas. Privacy fences are very standard here. When I have lived in other states, fence coding was very different. There are some HOAs here that don't allow privacy fences, but for most areas big privacy fences are the norm.
I really doubt the house can be "taken" from the owner. But I think it will be a hard sell.