We don't have to "permit" to build anything where I live. If you live in the county seat, you have to get a "building permit" to build. But that's basically you paying off the law enforcement to leave you alone while you build.
Or give them a reason to stop and ask if you have a building permit....