I got annoyed by the underground attacks so went to turn them off, and then about an hour later got another one. tried turning them off again, but the setting just keep turning itself back on, no matter if I turn it off in game or in the start menu. I am on the latest mod version, not playing with other mods that should affect it.

same issue. Hope this will be fixed fixed because the underground attacks are just a annoying the more you play.

I did more research and turns out it isn't caused by this mod, it is because of a game setting. in Settings -> Other, there is one called "Use different mod settings per save", if you disable that, mod settings wont randomly reset themselves between loads.