Hello,
You should be able to turn the setting on or off at any time, in a running game, with no issues. I've tested this myself and it's working correctly for me - so it may be another mod interfering. Also, please be sure you're on the latest experimental, as there have been some bugs related to the API for this feature in earlier versions of 0.18.
If you can upload the save somewhere I can also take a look and see if I can find what might be wrong.