I tried your mod coz I was interested in having auto-crafting machines as needed, using crafting combinator.
I noticed a constant huge lag spike and, was not that surprised taking into account the size of my logistic network. I also noticed that I already have 1.1k cells and, I am not even on blue science yet! (Py suite...) Imagine the lag spikes later on....

So, I looked into you code to see if there was something that could be done and I notice that, for each cell, you take the boundaries and do the scan in that boundary. That is fine and makes sense on a "normal" network. But... if you have a lot of overlapping cells, it is a big waste if time. In my case, you are scanning 8 times the same zone (using a city block layout).....

It occurred to me that you could have an alternate scan mode (enabled/disabled through the options), call it basic mode, that works pretty well in city block style factories. Since in these layouts the factory is a big square instead of organic growing cells, would make more sense to first scan all cells to get the boundaries of the whole network and then, with the boundary of the network do ONE scan for each thing you need to find instead of thousands of scans per cells.

Yes, there is a chance of some random cell making the boundary bigger then needed and then you are scanning "empty" areas and reporting wrong data but the chance of this happening in city-block layouts is minimal.
Hey... call it a downside of basic scan mode!

I thinks this new scan mode its an easy implementation and would improve drastically the UPS, specially on big bases like Pyanodons bases!"

I decided to provide accurate numbers (until entity scan limit) rather than speed.

In networks that are perfect rectangles speeding up scanning by magnitudes is fairly straight forward. Just find top left and bottom right corner and scan the whole area once. But hardly any network looks that way, especially not while a network constructs itself.

For it to work in a generalized network it'd take an algorithm merging all roboport areas into one polygon then splitting it into as few non overlapping rectangles as possible.
Since we can't even cache the resulting rectangles, roboports running out of power or powering back on do not trigger any event, so it needs to check if cells are exactly like they where when it calculated the rectangles, or just recalculate every time.
Overall that process might not even improve performance.

Your suggestion is a variant of the former that doesn't care if there's some gaps in the resulting square. Yes it'd be fairly simple, but I don't see the point in knowing exactly how many ghosts there are if the ghosts cant be built by bots.
For a rough estimate a manually setting cc would be enough.
For huge base modules blueprint reading combinators will always be better.

I was fiddling with your mod, because he wasnt exactly doing what I wanted... I wanted to pair with crafting combinators to craft what was missing for construction but... I would have signals for any construction request, even if I had the items to build them.

So, I start poking around and found that the alert API is perfect for it since alerts can be filtered out by type and only look at the ones about missing construction materials.

Just a dirty adaptation of your code, (I need to clean the code and obtain the required data in a more proper way...) :

local alerts = logsiticNetwork.force.players[1].get_alerts{type=defines.alert_type.no_material_for_construction, surface=logsiticNetwork.force.players[1].surface};
for , a in pairs(alerts) do
for , c in pairs(a[defines.alert_type.no_material_for_construction]) do
local entities = {c.target}
local count_unique_entities = 0
for , e in pairs(entities) do
local uid = e.unit_number
if not found_entities[uid] then
found_entities[uid] = true
for , item_stack in pairs(
global.Lookup_items_to_place_this[e.ghost_name] or
get_items_to_place(e.ghost_prototype)
) do
add_signal(item_stack.name, item_stack.count)
count_unique_entities = count_unique_entities + item_stack.count
end
end
end
end
end

Just something I wanted to share with you, never knows when this might bring you another crazy and awesome idea :)

For building globally finding all ghosts everywhere would be fine.
However Ghost Scanner is meant as companion to LTN to automatically generate requests for self constructing bases.

For this the Alert API has 2 "features" making it useless:
It randomly? decides which 100 or so, missing entities to show, making LTN use way more Trains than required.
What makes it really useless though, it cant be traced to specific logistic networks.

I see your point.... But... Dont get so focused on LTN.

Its not like this mod requires LTN to work, so other uses for it exist outside LTN. Probably more application outside LTN then inside...
Huge city blocks usually share one single network and usually is a square. If its not, its because a new section outside the square is starting to be built but usually, more then 75% of the resulting square would be inside that square and pretty soon other blocks are build and the coverage is back at 100%.

Also, as I said, I am playing a city block Pyanodons pack, with only bots, no trains and its already huge with only green science (thanks Py Aliens). With the mod without changes, I was experiencing freezes constantly. Lua object statistics showed a 2-3 usage of ghost scanner (whatever that metric means).
With the square, it went down to 0.1-0.2 and now with the alerts is 0.004-0.006.

I dont say that the alerts are a good improvement for this mod. Nahhh, its a total different concept and has no place here. It was just me throwing some ideas, just in case you could find them interesting for some new mod.

But the square, as an alternative scanning mode that is enabled through the options.... seems interesting.

Sorry for the late reply.
From the description I'd guess you use one massive Logistic Network spanning all city blocks rather than one network per block.
I don't want to optimize for this case, since the devs said many small networks have a much better performance than one huge network in base.