I don't see a way (as admin) to remove someone from my community without deactivating/blacklisting them. This screenshot is from the "All Users" area. I'm not sure the use case for needing this but for testing right now, it would be handy for removing my test accounts without having to log into them.