There are two problems with this. First, you need someone to oversee the whole thing, pay attention to what's happening, and be competent enough to make the right decisions. Moderators have to be "managed" too in a sense, they need a little feedback on what they're doing, especially in the beginning.
Second and more difficult, there isn't a lot of people who would be suitable to replace them. In my time it was a struggle to replace the mods that were quitting, there's a lot of people out there but not many with the right attitude.
Moderating done right is basically a thankless job, because you don't get anything out of it personally, all you get is satisfaction that the community is working well, which isn't the egotistical type of satisfaction that most people want. If you love community it will make you happy, if you just care about your own affairs it won't do anything for you.