Ghost Scanner

by Optera

Adds a combinator reading ghost requests from the logistic network it's placed in.

Content
1 year, 10 months ago
0.16 - 1.1
12.0K
Circuit network

g Performance of large logistics networks

5 years ago

One of my biggest problems with the Construction Signaler mod was how it performed in larger networks with lot of roboports. That mod would individually scan the logistic network area of each roboport, even though that meant scanning the same are potentially multiple times. I made a change to that mod to get the bounding box of the entire logistic network area first and then scan that area once, which made a massive difference in performance. It is possible that the bounding boxes of abnormally shaped logistic networks can overlap each other slightly, which means you might get one ghost signal reported in two different networks. This means I could end up with a couple extra items getting delivered to a location that didn't need them, but that was an insignificant price to pay in comparison with the performance improvement.

I am about to start a new game and was wondering if you have done anything similar to this mod.

For reference, here is the code change I made from the original mod. Likely it could be improved by someone who is actually proficient in LUA :)

    local function find_ghosts(network)
      if not network then
        return {}
      end

      local minx = 0
      local miny = 0
      local maxx = 0
      local maxy = 0

      local startpos = network.cells[1].owner.position
      minx = startpos.x
      miny = startpos.y
      maxx = minx
      maxy = miny
      for _,cell in pairs(network.cells) do
        local pos = cell.owner.position
        local r = cell.construction_radius
        if r > 0 then
          if pos.x - r < minx then
            minx = pos.x - r
          end
          if pos.y - r < miny then
            miny = pos.y - r
          end
          if pos.x + r > maxx then
            maxx = pos.x + r
          end
          if pos.y + r > maxy then
            maxy = pos.y + r
          end
        end
      end
      local bounds = { { minx, miny, }, { maxx, maxy } }
      local ghosts = network.cells[1].owner.surface.find_entities_filtered{area=bounds, type={"tile-ghost", "entity-ghost"}, force=network.force}
      return ghosts

end

5 years ago
(updated 5 years ago)

The naive approach of caching a rectangular bounding box will fail 99% of the time.

1) Caching networks is not possible as there are no events when logistic networks change due to roboports running out of power.
You'd to poll every roboport to check if it's still within the cached network before doing any scanning and if it isn't recalculate networks for all roboports, one failed roboports at worse turns 1 network into 4.
The lack of event handling means all scans for one sensor must happen in one tick, or we risk networks changing during scan.

2) Logistic networks must be polygon shapes and support sub network.
For example I built my smart furnaces as sub network with a 1 tile gap to the main logistic network completely surrounding it, neither of them are prefect boxes.

With all these problems I found the most sane, albeit not very efficient way, was to not cache anything and instead iterate over every logistics cell in a given network, scan in the bounding box of that cell. To prevent counting ghosts multiple times I use a dictionary indexed by unit_number.

5 years ago
(updated 5 years ago)

Hey Optera. Regarding your first point, nothing is being cached. The bounding box is calculated every time the signaler does the scan.

For your 2nd point, yes of course you are correct which is what I also pointed out in my message. But does that matter? If your main network overlaps your furnace network, what is the worst that happens? While your furnace network has ghost signals, it also means your main network will pick up those same ghost signals. What that actually means for individual players can vary, but usually it just means that one network will overproduce (or have delivered) a few excess items, but usually those items just gets put back into the logistic network for the next time they are needed.

So basically my improvement accepts a very minor inconvenience to gain a massive performance gain. For most megabases the tradeoff is a no-brainer.

5 years ago

Counting ghosts not placable by a network would be a bug.

I doubt the performance gain actually is big. scanning few large areas is equally slow as multiple small area scans. IF players build non rectangular shapes, worse case a large L, the scan area with your approach will become large enough to be slower than the current implementation.

5 years ago

Yeah, the behavior could be considered a bug, it isn't necessarily a good general approach for all players.

But to be clear, the performance gain was huge. The game would literally pause for a couple seconds on a ~1.4kspm base with maybe 15 construction signalers on the map. The biggest perf gain was in areas where there were large concentrations of roboports (needed for charging). The same coordinates were getting scanned dozens of times (once per roboport)

5 years ago
(updated 5 years ago)

The clean way to optimize scanning would be to get the complete network polygon, break it up into rectangles and scan each rectangle once.
I made an API request for networks to provide a way to fetch ghosts more easily: https://forums.factorio.com/viewtopic.php?f=28&t=72794

As quick fix use as few roboports with cell range as required to span the network, and roboports without cell range for charging.

New response