Now that I know a bit more about Lua, I went looking for why things weren't being set. In maps\caves\surfaces\overworld.lua and cave.lua, the surface.map_gen_settings table is being defined, not changed. That messes up any existing changes (as far as I can tell that's what's happening, or something like it). I was playing with overworld.lua, and by defining the resources in a table, and using pairs() over it with the autoplace settings (surface.map_gen_settings.autoplace_controls[key] = value), I specifically changed each resource. I completely removed the other settings for my tastes, as well as the enemy nests from the resource table. This time the map respected my enemy frequency/size settings (commenting out the enemy nest line in the table seemed like it reset it to default/normal game settings i.e. quite sparse and small).
I also have another two mods which adds ores, but they spawn up top. I might try adding a pairs() over data.resource, and see if I can move them from the overworld to the underground. It's more sensible than exactly specifying what resources you modify (and I see there's code for Bob's ores and stuff - that's not necessary if you loop over data.resource)