seems to cause major massive lag spikes, as i was analysing the issues in jd play's ep100 save, it was both Concreep and Pickup Tower causing lag spikes - seems to be doing too many things in one tick. here is some relevant data:

 4.59%  factorio          [.] HeuristicEntityIteratorT<EntitySearchTypeInfo<Surface, Chunk> >::operator++
 4.24%  factorio          [.] Surface::listFriendlyEntities
 3.10%  factorio          [.] Chunk::updateEntities
 2.51%  factorio          [.] propagatemark
 1.86%  factorio          [.] luaH_get
 1.55%  factorio          [.] FluidBox::PipeConnection::calculateFlowSpeed
 1.49%  factorio          [.] singlestep
 1.40%  factorio          [.] TransportLine::move

When I develop this mod, I made some extensive tests (although not up to the point of numerical analysis). I have some conclusions in general.
1/ The total number of entities on the ground (and on map) is directly related to UPS/FPS. Each item on ground counts as one entity. Each UNTOUCHED chests containing thousands of items also counts as one entity. An idle assembling machine also counts as one entity. This is true even BEFORE pickup towers kick in. Each tiny item on ground is still consuming your processing power, even if they are untouched!
2/ My mod uses surface.find_entities_filtered to search for items to be handled. The processing time is related to both the area (range) and number of entities. Larger pickup range uses much more time (much more than linearly with area). More entities in the approximate vicinity (I do not know how the game the search optimized where far away entities affect much less.) uses much more time.

There is almost nothing to be improved in the code. This single command is the only one that I can use to look for all entities in a specific area.
Therefore, I can only give some general recommendations on the game-play side:
A/ Keep items on ground as few as possible at any time. That means clear the ground more frequent.
B/ For the same reason, lower loot richness in Schall Alien Loot. Do NOT be too greedy! I expect casual players to use medium richness, hardcore players use lower richness. High richness is only for newbies or no-mining scenario. This is why I provided options to adjust loot richness. The general principle is to let users to obtain enough ores in combat, while still not flooding the ground or storage system. Greed is natural behaviour, but too much greed IS problem.
C/ For the same reason, use less loot mods at the same time. Using multiple loot mods generate more types and amounts of loots, which I totally NOT expect that. Multiple loot mods are testing the limit of your computer. If you have too much excess still left on ground, you know you should either clear them ASAP or simply remove those loot mods.
D/ Split up big pickup searches into smaller searches will use less time overall, which means:
D1/ Use lower tier pickup tower when feasible. Just use those which are adequate to cover the kill area. For example, if your turrets are of range 48, then Pickup tower II (range 64) should be well enough. Even if having to cover a WIDE battlefront of width-128, using two PT II is going to be much faster than one PT IV! PT II also has much shorter interval to keep the number of entities low. Several small searches actually is better than a single big search.
D2/ Only deploy PT to where loots are generated. Idle PT (say, within the safe zone) still call for surface.find_entities_filtered. Even for nothing to pick up, it is still a call for a search. And a lot of them can still add up.
E/ I am not sure about interaction with other mods. But if there are a lot of tile ghosts (which ARE also entities), they are going to add a lot of burden to PT searches, AND also to normal game updates (such as assigning bots to build tiles). The second part is exactly why many MP server admins will really warn and even ban players on mass-laying concrete tiles by bots!

oh man, yeah. he had Concreep installed along with the pickup mod. I will forward your messages - I had already come to the same conclusions - he has too much loot being generated and too many high tier pickup towers all operating at the same time. he is already trying to clear as much off the ground as he can at once, which is why the lag spikes.

To supplement, mass-laying concrete tiles by bots (script assigning bots to lay is no different) consumes more processing power than you may expect.
I used to play a lot on public MP servers in 0.16 and 0.17, where NO mods are run. There could be like 40 players online at the same time, and we built megabases. Game are usually smooth. But when some random pop-in noobs had nothing to do, then they decided to lay concrete the whole base (not a rare case, sadly). The game instantly suffered to "Powerpoint slideshow", until some admins removed all those concrete tile ghosts. So it is no surprise this is considered "mild" griefing by many game admins.

I also played on MP servers that give some drops to the ground (mild mod-embedded scenarios, but still no external mods). Loots come either in form of coins or coal/ores. An alternate form would be dropping coal/ores when hand-mining a rock. If some players do not clear the items after "farming", lag came immediately. So I would say Factorio is (from 0.16 - currently) not quite tolerable with lots of items on ground, in general.

And yes, even on his earlier episodes, he already had much more loots than he needed and could manage. This could only go worse when the base grows larger. This is what I called "greedy" to have so much loots! ;-)

yes, i really think it's something for factorio devs to optimise and clearly they decided not worth the time considering they removed alien drops from the base game so long ago :-)

thank you for your time, hope to show you my 1 million SPM (no prod mods) base using your mods soon.

hi Schallfalke, I have resolved this issue in LootChest+ mod, I forked as LootChestUPS. it uses events when entities due to pick up item instead of scanning the ground.

Interesting work. It is an effective use of event, instead of having to rely on on_tick. It is a big improvement to the mod you are worked on.

Unfortunately, I cannot make use of same method you are using. I do not want to keep track of towers and their positions. I still want the distance being respected in my mod, but applying calculation of position and finding tower chest(s) within range in each on_entity_died event call does not sound too UPS friendly. I doubt how much better it could be than my current implementation.
I agree on_entity_died is effective for loots that are already defined in entity prototypes. Yet, it cannot catch non-loot items placed on ground. For example, my another mod Schall Remains & Pickup would leave (rather than lost) inventory contents of destroyed containers (chests, vehicles, machines, etc) onto the ground. on_entity_died event would not be appropriate to catch all these items.

So perhaps it is difficult to respect the distance and UPS at the same time. Neglecting the former means I can totally neglect the bot/belt/train transportation from the frontline to the factory. It is no different from transferring ores from the mining drills to the factory directly. I definitely consider this as cheating. Hence, the former factor is important to me, this is where I am willing to sacrifice some UPS.
Anyway, different people have different tastes. So good luck to your mod.

discovering the distance is an order of magnitude cheaper than searching the ground, in my testing.

also, on_entity_died can be used to initiate a search of an area - it could be an optional component of the mod. is there a technical reason you can't return loot with your other mod?

tbh you could reduce all of the abilities of the mod to options and enable them by default, then allow users to enable/disable portions of the logic to save UPS when they need.

hi Schallfalke, I have resolved this issue in LootChest+ mod, I forked as LootChestUPS. it uses events when entities die to pick up item instead of scanning the ground.

Yes, but it would be executed for every time an entity died, for the number of tower chests already on map.
Say, there are 12 PTs on map. Then there needs 12 distance comparison per entity death.
Multiplying this number by the number of biters killed, could turn out to be a large number.

ok, good point. the LCP mod only allows one for each player force. it's probably fine / good enough that there are two approaches to this problem, maybe once hanes and I figure out how LootChest+ is going to be, you can make a reference here and we can make one on our info page, and everyone can choose what fits their taste best.

also, on_entity_died can be used to initiate a search of an area - it could be an optional component of the mod. is there a technical reason you can't return loot with your other mod?

In fact, it is my first time seeing on_entity_died event.loot can be modified, discovered by reading your code, where you have used clear() on it.
If Schall Remains & Pickup has to make use of it, it has to do the opposite thing by inserting items to it. I am not sure it may even work, and whether has a limit or not. A modded warehouse could have hundred of slots worth of items within. So I do not want to rely on it.

tbh you could reduce all of the abilities of the mod to options and enable them by default, then allow users to enable/disable portions of the logic to save UPS when they need.

This sounds ideal if I have the time to invest into, test out, and maintain the extra code. Unfortunately, I do not have as much time for Factorio modding as ever.
There are at least 3 new mods that I have promised, but still no time to make. If I have the time for serious modding, that would certainly be allocated to them first. Only hotfixes and very simple mod features could go in front. All non-trivial optimizations have low priorities in my task list. So I should be honest to tell you that I do not see the time for that.

kind sir Ptx , thank u very much , i m back to 60 fps again thanks to you , i wanted to play against massive amounts of biters , walls and collecting remains and pragmaticly use of these materials , but i wasnt be able to play more than 50hrs, no metter which mod i played, finally i see your comment and awsome mod suggestion, a bit of a lame thing to use feels like cheat but , i gues i dotn have other choice. bunch of thx

factorio "lag spike" lag minifreeze freeze mini UPS FPS drop solved thx again