Hi, I've got myself into a situation where the game gets almost unplayable. When I load my (heavily modded) saved game, UPS drops significantly after a while -- 0.3 was the lowest value I've seen so far.

I managed to hit F4 before the drop and watch the load on the game. Turns out the number for this mod skyrocketed from 0.x up to 45.x. At the same time, the load of some other mods (Cannon Turret,
Armor Plating , and Big Bertha Artillery Sounds) increased significantly as well.

After looking at the other mods, I found that Cannon Turret and Armor Plating listen to on_entity_damaged (spores flying around at night will damage lots of walls and turrets), while Big Bertha Artillery Sounds listens to on_trigger_created_entity. I've uncommented the debugging code in all these mods, and Big Bertha produced a lot of logging output (writing to log probably made the response time even worse). The log contains lots of lines like these:

1309.568 Script @__vtk-ta-big-bertha-artillery-sounds__/control.lua:19: "wisp-red"
1309.570 Script @__vtk-ta-big-bertha-artillery-sounds__/control.lua:15: VTK-BERTHA-DEBUG
1309.570 Script @__vtk-ta-big-bertha-artillery-sounds__/control.lua:17: {
  __self = "userdata"
}

Grepping the log file for "on_entity_died for entity", I found three entries:

 222.427 Script @__GCKI__/common.lua:160: Entered event script on_entity_died for entity wisp-red
1419.307 Script @__GCKI__/common.lua:160: Entered event script on_entity_died for entity wisp-red
2335.422 Script @__GCKI__/common.lua:160: Entered event script on_entity_died for entity wisp-red

Meanwhile (night isn't over yet), more than 200K red wisps have been created!

$ grep VTK-BERTHA-DEBUG factorio-current.log | wc -l
207665

Could you please include a check that prevents such excessive multiplication of red wisps?

Two hours later: 4 red wisps died, 589461 have been created. Factorio is using more than half of my RAM (i.e. >8GB) and fully occupies one of the CPU cores. No activity has been logged for a long time:

12860.263 Script @__vtk-ta-big-bertha-artillery-sounds__/control.lua:19: "wisp-red"
14070.986 Script @__vtk-ta-big-bertha-artillery-sounds__/control.lua:15: VTK-BERTHA-DEBUG

but I haven't been able to switch back to the game again. I'm going to give up now and kill the process (I really wanted to see if it gets better after the night, but it takes too long.)

Note that one easy fix is to just remove this mod - this will remove all the entities defined in it, so should get rid of the issue immediately.
You can re-add it to the game later if you want too, all without any real consequences afaik, well, except I guess it'll need to rescan for forest tiles, so new wisps might only start appearing in a day or two.

Ok, so replication happens through this thing in prototypes/wisp-red.lua:

  attack_reaction = {
    { range = 40,
      damage_type = 'physical',
      reaction_modifier = 0.1,
      action = {
        type = 'direct',
        action_delivery = {
          type = 'instant',
          source_effects = {
            type = 'create-entity',
            entity_name = 'wisp-red',
            trigger_created_entity = 'true' },
          target_effects = {
            type = 'create-entity',
            entity_name = 'wisp-flash-attack' } } } },
  },

I.e. when thing is attacked, it runs create-entity effect with wisp-red being created there.

That'd obviously spiral out of control fast without any kind of limit, so there's this in control.lua:

local function on_trigger_created(event)
    if not (InitState and InitState.configured) then return end
    -- Limit red wisps' replication via trigger_created_entity to specific percentage
    if event.entity.name ~= 'wisp-red' then return end
    if utils.pick_chance(conf.wisp_red_damage_replication_chance)
    then wisp_init(event.entity)
    else event.entity.destroy() end
end

...
script.on_event(defines.events.on_trigger_created_entity, on_trigger_created)

Which checks conf.wisp_red_damage_replication_chance = 0.2 from config.lua and either adds that new red wisp entity to tracking with the mod (20% chance) or kills it outright (80% chance), resulting in that 20% replication chance on them taking damage.

If some other mod creates an effect that constantly applies damage accross the map, this is what unfortunately what should happen - red wisps will get created in geometric progression by taking damage.

But there's another place where these get created:

local function on_mined_entity(event)
    if not (InitState and InitState.configured) then return end
    if entity_is_tree(event.entity) then wisp_create_at_random('wisp-yellow', event.entity) end
    if entity_is_rock(event.entity) then wisp_create_at_random('wisp-red', event.entity) end
end
...
local function on_death(event)
    if not (InitState and InitState.configured) then return end
    local e = event.entity
    if entity_is_tree(e) then wisp_create_at_random('wisp-yellow', e) end
    if entity_is_rock(e) then wisp_create_at_random('wisp-red', e) end
...

Where entity_is_rock check is implemented like this:

function utils.match_word(s, word)
    -- Returns non-nil if s ~ /\bword\b/ (pcre)
    return s:match('^'..word..'$')
        or s:match('^'..word..'[%W]')
        or s:match('[%W]'..word..'$')
        or s:match('[%W]'..word..'[%W]')
end

local function entity_is_rock(e) return utils.match_word(e.name, 'rock') end

So what if some mod constantly creates and destroys something with "rock" in the name on the map? Wisps get spawned too :)
E.g. space rocks or something.

Guess you can add debug print()'s to either of these places to find out in which of the two ways these things spawn, and printing entity name for "rocks" if it's the latter will probably tell which mod does rock-spam easily.

As for fixing it, I think it's a bug that neither of these spawn-ways respects conf.wisp_max_count value, which can be configured in mod settings, which would definitely prevent worst-case scenario of endless replication, as that's set to only ~1k by default.

For catch-all enforce-wisp-limit fix, I'd suggest adding this line as the first one to wisp_init function:

if Wisps.n > conf.wisp_max_count then return entity.destroy() end

That func runs for every single wisp that mod tracks (and it tracks all of them), and whatever the spawn mechanism, it should not allow count of these things to go over configured limit.

Ideally this limit can be enforced earlier, before entity is even created and such, but those should be relatively insignificant optimizations, as normally wisps should spawn relatively rarely.

Note that such fix would probably pad limit with red wisps, so ideally there should be some kind of "generation limit" for spawning those, but that's relatively complicated to implement (more than couple of lines).

Alternative fixes can also be:

• Disable red wisps' mechanics entirely - just drop that create-entity effect in wisp-red.lua file, or tweak entity_is_rock to always "return false".

• Find a mod that triggers these effects descibed earlier and figure out how to make them co-exist with this one - maybe exclude red wisps from taking damage, change damage type (they only replicate on "physical" iirc), fix entity_is_rock check to ignore rocks from the mod, etc.

• Set conf.wisp_red_damage_replication_chance = 0 in config.lua to have all newly-spawned red wisps to be always killed instantly.

Happy holidays regardless, and let me know if I can do anything else to help there.

I don't really maintain this mod anymore, and it's a hassle to reinstall factorio, remember how to test it, setup a save, etc to release proper new version with a fix, so given that you seem to be well-familiar with modding the game, guess description of the fix should suffice.
Besides, I'm not even the owner here anymore to do releases, so feel like it'd also not be very appropriate :)

Btw, adding more TA into factorio sounds like such a nice idea.

Thanks for the long reply -- and sorry for enticing you back to Factorio, of course! Forgot that you'd wanted to take time off …

local function on_trigger_created(event)
if not (InitState and InitState.configured) then return end
-- Limit red wisps' replication via trigger_created_entity to specific percentage
if event.entity.name ~= 'wisp-red' then return end
if utils.pick_chance(conf.wisp_red_damage_replication_chance)
then wisp_init(event.entity)
else event.entity.destroy() end
end

...
But there's another place where these get created:

local function on_mined_entity(event)
if not (InitState and InitState.configured) then return end
if entity_is_tree(event.entity) then wisp_create_at_random('wisp-yellow', event.entity) end
if entity_is_rock(event.entity) then wisp_create_at_random('wisp-red', event.entity) end
end
...
local function on_death(event)
if not (InitState and InitState.configured) then return end
local e = event.entity
if entity_is_tree(e) then wisp_create_at_random('wisp-yellow', e) end
if entity_is_rock(e) then wisp_create_at_random('wisp-red', e) end
...
```

Where entity_is_rock check is implemented like this:
local function entity_is_rock(e) return utils.match_word(e.name, 'rock') end

Guess you can add debug print()'s to either of these places to find out in which of the two ways these things spawn, and printing entity name for "rocks" if it's the latter will probably tell which mod does rock-spam easily.

I added logging directives to each of these functions, and also to the event script in Big Bertha Artillery Sounds. The strange thing is, entity_is_rock check seems to be left alone. But immediately after loading the game (first line), the log is spammed with lines like these:

 103.461 Script @__Will-o-the-Wisps_updated__/control.lua:704: Entered function on_trigger_created for event 22 (wisp-red, wisp-red)

 103.465 Script @__vtk-ta-big-bertha-artillery-sounds__/control.lua:7: Entered eventscript for on_trigger_created_entity (event 22)
 103.465 Script @__vtk-ta-big-bertha-artillery-sounds__/control.lua:10: Event entity: wisp-red  Event source: wisp-red

 103.465 Script @__Will-o-the-Wisps_updated__/control.lua:704: Entered function on_trigger_created for event 22 (wisp-red, wisp-red)

So, nothing seems to be spawned but red wisps.

For catch-all enforce-wisp-limit fix, I'd suggest adding this line as the first one to wisp_init function:
if Wisps.n > conf.wisp_max_count then return entity.destroy() end

Added that line, and turned on logging for that function go (including values of the arguments):

299.190 Script @__vtk-ta-big-bertha-artillery-sounds__/control.lua:7: Entered eventscript for on_trigger_created_entity (event 22)
 299.190 Script @__vtk-ta-big-bertha-artillery-sounds__/control.lua:10: Event entity: wisp-red  Event source: wisp-red
 299.191 Script @__Will-o-the-Wisps_updated__/control.lua:712: Entered function on_trigger_created for event 22 (wisp-red, wisp-red)
 299.191 Script @__Will-o-the-Wisps_updated__/control.lua:288: Entered function wisp_init
 299.191 Script @__Will-o-the-Wisps_updated__/control.lua:289: entity: wisp-red
 299.191 Script @__Will-o-the-Wisps_updated__/control.lua:290: ttl: nil
 299.191 Script @__Will-o-the-Wisps_updated__/control.lua:291: n: nil
 299.191 Script @__Will-o-the-Wisps_updated__/control.lua:294: More wisps than 750
 299.191 Script @__vtk-ta-big-bertha-artillery-sounds__/control.lua:7: Entered eventscript for on_trigger_created_entity (event 22)

Does it mean something that ttl and n are always nil?

I'll remove the logging for now (it still slows down everything immensely) and see if that makes the game playable again. If not, I'll try one of the alternative fixes -- but of course it would be far better if I could play with red wisps as I did before this.

Anyway, I really appreciate your help, and I wish you a happy new year! :-)

New development: Loaded back an autosave that started a minute earlier. FPS slumped down to about 15, so I could see that there was a big blob of red wisps next to a rock that got attacked by a continuous stream of water (from Water Turret). Close by, there was something looking like a fight between yellow and red wisp. After a while, the game crashed with this error:

Error while running event Will-o-the-Wisps_updated::on_entity_died (ID 4)
LuaEntity API call when LuaEntity was invalid.
stack traceback:
    __Will-o-the-Wisps_updated__/control.lua:324: in function 'wisp_create_at_random'
        __Will-o-the-Wisps_updated__/control.lua:671: in function <__Will-o-the-Wisps_updated__/control.lua:665>
stack traceback:
    [C]: in function '__index'
        __Will-o-the-Wisps_updated__/control.lua:324: in function 'wisp_create_at_random'
    __Will-o-the-Wisps_updated__/control.lua:671: in function <__Will-o-the-Wisps_updated__/control.lua:665>

Does it mean something that ttl and n are always nil?

They're initialized to some defaults there if not passed iirc, as in this specific case.

After a while, the game crashed with this error:

Sounds like game was already pretty much dead by then, I'd suspect something in lua breaking down :)

big blob of red wisps next to a rock that got attacked by a continuous stream of water (from Water Turret).

Guess that might be the cause - ton of minimal damage in some area of effect that causes endless replication, and yet they can't get close and destroy the turret, as they can do with any vanilla one.

I think best fix would be to change how this replication works, i.e. add replication cooldown which can be global or maybe passed from one red to all its clones, or maybe something more creative.
Boring fix can be to just remove this unique behavior.
Lazy fix can be to add switch to disable it, so that users having issues with it due to mods' interplay can fix it themselves :)

Not at home right now. But last thing before I left, was adding

 if Wisps.n > conf.wisp_max_count then return end

at the start of the functions wisp_create() and wisp_create_at_random(). Didn't have much time for testing, but I guess it would take some load off the game as these wisp wouldn't even get created. I'll see later if I can do anything about the replication cooldown (can't do much now without the game for testing).

at the start of the functions wisp_create() and wisp_create_at_random()

Yeah, that's why I suggested doing it in wisp_init() and NOT wisp_create ones, since as mentioned above - "replication happens through this thing in prototypes/wisp-red.lua: ..." - i.e. not wisp_create() or anything like that.

Issue seem to be replication, so code path there is wisp-red prototype triggering creation of new entity, which is then passed to on_trigger_created_entity(), where either wisp_init() is called or this new wisp-red entity is destroyed.

Note btw that prototype section responsible for this is pretty much original old stuff from 0.14 or earlier API, so it might be possible to do such chancy effect in a better way than "create + destroy-immediately" these days, i.e. add scripted condition to whether effect should happen, or have scripted effect entirely (which will decide what to create and how).

I have done some testing in my playthrough (heavily modded) and found that the limitless replication will spiral out of control. One factor that plays in the creation of red wisps is biters attacking rocks. This can be catastrophic at low level biters versus higher tiers. Tier 1 and Tier 2 biters do not do enough physical damage to red wisps to out right kill them, which then governs the replication aspect of the red wisp. In a game I purposely created a scenario of Tier 1 biters attacking nearby rocks, created nearly several hundred red wisps within a matter of minutes. This can be hurtful for the UPS drop.

As I am learning .lua still and am working to take over this mod, maybe I can propose having the setting in the menu to limit the red wisps and then further studying a bit, limit the total wisps (which already exists). mk-fg has more time with this mod than I do, years versus a few months, and may have some input as already stated in this thread.

Wonder if this changed recently - I think factorio versions that I've used didn't actually enable AI for chunks far from player structures, so any low-level attacking biter swarm chomping on rocks along the way will just be wiped-out by red wisps (if "wisps fight biters" option was enabled), and that will be it.

If factorio now simulates chunks much further away from players, guess it's possible that such wisps will just keep going beyond known edges of the map until sunrise :)
(also note - sunrise is one obvious limiting factor)

setting in the menu to limit the red wisps and then further studying a bit

I love configurable things, to the point of using linux and compiling many things in my OS from sources with my own patches, but pretty sure most users never look at these and/or actively hate them :)
Though then again, people who mod their game are somewhat different and self-selected for wanting at least some customization already (that's what mods are, after all).

Hence me calling "add a switch" approach a "Lazy fix" in one of the msgs above, as it basically shifts burden of knowing how things work and figuring out what is best solution from devs onto users.
Also, not that lazy fixes are bad either - no one has time to figure out what is objectively best, which is why I added all these crazy options that are there already :)

For catch-all enforce-wisp-limit fix, I'd suggest adding this line as the first one to wisp_init function:
if Wisps.n > conf.wisp_max_count then return entity.destroy() end

Reverted my changes so that this line is now only in wisp_init(). Also, there still was an autosave (about 10 minutes before the red wisps spawned) that I could load; together with the line added to wisp_init(), that helped to make my game responsive again.

About the options, I've just looked up the mod settings I used in my game:

Wisps can retaliate:        Yes
Wisp retaliation radius:    96
Automated defenses target all wisps:    No
Purple wisps corrode buildings:             Yes 
Wisp aggression factor:                           0
Wisps fight native enemies:                    Yes
Wisps aggro on players only:                  Yes
Maximum wisp count on map:                750
Forest pollution factor:                              100
Wisp chance (purple):                               0.5
Wisp chance (yellow):                               0.07
Wisp chance (red):                                    0.01
Wisp chance (green):                                0.02

About sunrise as limiting factor: I'm playing with Longer Days and Nights, with the day/night lengths multiplier at the default setting of 4.

Does that help in any way?

Didn't mean to ask for options, just wanted to note above why this shouldn't be expected to be a problem with vanilla game rules as I know these.
Day/night cycle extension factors-in a bit, but given how vanilla combat works, any kind of occasional reset like that should work, 3 minutes or 30 - doesn't really matter, unless I'm missing something.

Btw, not sure if factorio notifies about that, but I marked "Longer Days and Nights" as deprecated, since someone pointed out that there's much better way to implement it in modern factorio, and looking around for mods that do same thing right, found that someone indeed implemented it like that, and linked to that mod instead (don't remember the name, should be on first lines of the description).

Day/night cycle extension factors-in a bit, but given how vanilla combat works, any kind of occasional reset like that should work, 3 minutes or 30 - doesn't really matter, unless I'm missing something.

Longer cycles could become problematic because more red wisps could be spawned before they are removed by the sunlight-induced reset. :-)

Btw, not sure if factorio notifies about that, but I marked "Longer Days and Nights" as deprecated, since someone pointed out that there's much better way to implement it in modern factorio, and looking around for mods that do same thing right, found that someone indeed implemented it like that, and linked to that mod instead (don't remember the name, should be on first lines of the description).

Deprecating a mod means it won't be listed among the mods you can install any more -- but if it already is installed, the game won't tell users the mod has been deprecated unless the mod author takes care of it. I did that with a mod fixing a bug in Realistic Ores after the original author had been away for a long time. When he returned and updated his mod, I released a final version of my fix containing just info.json with a conflict against itself and a dependency on the original mod. Moreover, I also changed the mod description to urge people to abandon my fix for the original. Maybe that's not necessary for "Longer Days and Nights", though, as Factorio 0.18 should be released in the not-so-far future and may break it for good. :-)