1.6.4 is using significantly more time than 1.6.3. In a large SpaceEx base, my UPS dropped form 80 to 40, time usage shows this all going to Ghost Scanner.

Once is expected from iterating through items_to_build_this for filtering hidden items.
After the first round it should have items cached.

I've just ran into the same issue, except my base isn't overkill sized. Not for SpaceExp scales anyways.
It's "only" 53 Roboports where I've run into the problem and the scanner still caused 80+ ms of processing each time it kicks in.
And no, this isn't after loading the game either, this is after having played for an hour.