Greetings!
So, I'm back to HD mods again :))))
Here's what I came up with...

Original problem:
The original code traverses the entire data.raw (all ~150+ prototype types in the game, without third-party mods) to find sprite paths and replace them with HD versions. This takes me ~9 seconds. I tested it for "factorio_hd_age_base_game_production" with several mods enabled, including global ones like Krastorio2, Nexus, Metal_and_Stars and a couple more. When enabling a large number of third-party mods, the number of entries increases, and for each sprite, all existing data.raw entries are constantly traversed.

Solution:
Added prototype type information to the mod config via the "type" parameter. For example:

["accumulator"] = { __type__ = "accumulator", ["*"] = {}, ["remnants"] = {} }
["steel-furnace"] = { __type__ = "furnace", ["*"] = {}, ["remnants"] = {} }

Now the code:

Scans the config and collects a list of used types (accumulator, furnace, mining-drill, etc.)
Traverses only these types in data.raw instead of all existing ones
The rest of the path replacement logic remains unchanged

Result:
Before: 9 seconds per pack (traversing all type entries)
Now: 1.2 seconds (traversing only ~10-15 required types)
Acceleration: ~750% (7.5 times faster)

One downside is that to reduce loading time, you'll have to add entity types to each config. Well, that's if you actually want to solve this problem. I'll link to the re-uploaded test mod if anyone's interested.

The advantage is that there is minimal modification of the main code.

For my own use, I'll rewrite it slightly differently, since currently each HD mod is essentially a standalone, separate mod, but the code in each one is the same. So, if we have 22 HD mods, we'll be executing the same code 22 times.
I'll make only one main "core" mod, and all the others will be texture packs only. That is, if a texture pack is present and enabled, it will load. The concept remains the same, but we'll avoid executing the same code more than 20 times, which will further reduce loading times.
However, I'm not yet sure how best to do this.

As a result, if in the current version I had a problem with the loading time of all 500+ enabled mods and all HD mods, and the time was over 14 minutes, then with this version I expect the loading time to be about 2 minutes.

Link to mod for testing:
https://drive.google.com/file/d/1JwkRSKTEe5j8tBqpJWb_JXH-Ojg9h30Y/view?usp=sharing

Hi, wow, I'm impressed. I'll try that out right away. If it works well, I'd like to implement it in all mods and give you a shout-out.

I also had the idea of doing everything via a main mod and then uploading the textures without the repetitive code. Unfortunately, I can't remember what stopped me from doing that. I'll think about it again.

Well, I hope your testing goes well too :)
I'm taking a time-out for now, as I'm really confused between the options. I was looking for the one with the minimum code changes, and now my head is a jumble of all the options, and I can't seem to figure out the code structure for a "core" mod. I understand what's needed, but I can't quite put it together :(

Well, and accordingly, as soon as I do this, I will share the option and my thoughts.

Huge respect for you figuring that out.

Yes, at some point you just see letters and brackets.

You changed the main code, which I accepted as given.
I'll need some time to figure out what you did there.

Apart from that, I have noticed that although the entities have an HD texture, the remains only load in SD with your changes.
So it seems to work only for entities but not for their modified or altered forms.

Yeah, there's a problem :(

What we have:
All the remnants are stored in the data.raw["corpse"] records. Previously, the code would go through all the records, including this one, and if it found a path match, it would replace the sprite.
Now, when we specify specific records for an entity, it no longer checks the data.raw["corpse"] record. Consequently, it doesn't find the path and doesn't replace the sprite.

What to do:
Somehow add the data.raw["corpse"] record to the bypass. And here, too, I'm having problems, because if you add a record, it checks all entities, regardless of whether we specify a match by path ["remnants"] = {} or not.

Overall, I had to work hard, and it turned out to be more difficult than I thought
I think I've figured out a solution, but need to test it with other HD mods.

I've reuploaded the mod, and if it's not too much trouble, please test it with other mods.

Just note that I intentionally removed the ["remnants"] parameter from one of the entities in the config:
Instead of:

["chemical-plant"] = { __type__ = "assembling-machine", ["*"] = {}, ["remnants"] = {} },

it now says:

["chemical-plant"] = { __type__ = "assembling-machine", ["*"] = {} },

since I tested it on it, and then need to return the record with ["remnants"] = {}.

Result:
Now all the specified entities, except ["chemical-plant"], have their remnants sprites replaced with HD ones.
Ahh, and I also removed the logs from the added code, since in fact they were only used for testing, the rest of the logs were not changed.
I hope I haven't screwed up anywhere else :)))

Overall, the result looks good, loading takes just 1 second, and those with a more powerful processor will get even faster (or, conversely, for some, it will be slower):

3.786 Loading mod factorio_hd_age_base_game_production 1.1.2 (data-final-fixes.lua)
4.776 Loading mod ...

But again, need to test it in other mods, and if there are any errors, need to think about how to fix them.

Just in case, the re-downloaded mod should have the "processRemnants(entity_name)" function added to the end of the code.

What I really don't understand right now is why the loading time is changing, even though I've enabled another mod... and purely theoretically, the time should be either the same or longer, but not shorter... ????

   3.514 Loading mod factorio_hd_age_base_game_production 1.1.2 (data-final-fixes.lua)
   4.371 Loading mod ...

What I really don't understand right now is why the loading time is changing, even though I've enabled another mod... and purely theoretically, the time should be either the same or longer, but not shorter... ????

In my experience, background processes always affect loading times. It's possible that a background process is interfering with your reproducible results.
In this context, I'm wondering what kind of hardware you're using. Specifically, I'd like to know what type of processor you have, and if you've installed the game on an NVMe drive. I can't remember if you've told me this already. I thought about the processor because you said that loading an individual HD mod takes 9 seconds. For my friend and me, it only takes about 1 up to 1,5 seconds per mod. This leads me to assume that your hardware is weaker and that background processes therefore have an even greater impact on performance.

In general, I have been considering what other factors need to be taken into account.

1. It should be possible to assign multiple type entries per folder. Depending on the folder, several types may be represented. For example, the Recycler folder contains both normal and remnant textures. This means that two type variables would have to be defined for one folder.

2. Assigned type entries should be automatically applied to all subfolders.

Unfortunately, I don't have enough time at the moment to examine the code thoroughly and try things out myself.
I'm hoping that, between Christmas and New Year's, I'll find at least one day when I can commit to this task entirely.

However, if you are still interested in sharing your results with me, I would be very happy to hear from you. I would also be happy to share my thoughts and feedback with you, but I can't promise when I will be able to do so.

Processor:

System Information: [CPU: AMD Ryzen 5 5600, 6 cores, 12 cores, RAM: 20887/65462 MB, Page: 22671/65462 MB, Virtual Memory: 4359/134217727 MB, Extended Virtual: 0 MB]

And if we talk about a clean boot, with only mods enabled:
1. base
2. elevated-rails
3. quality
4. space-age

Then loading one mod (for example, factorio_hd_age_base_game_production) takes me:

1,866 Loading the mod factorio_hd_age_base_game_production 1.1.2 (data-final-fixes.lua)
2,726 ...
4,601 Details ModManager.cpp:645: Mod loading time: 3.24068

According to the logs, loading a mod takes just under 1 second, and loading all mods takes 3.2 seconds.

But if I enable other mods (roughly the minimum set I need) that add data.raw entries, the picture starts to change:

3.468 Loading mod factorio_hd_age_base_game_production 1.1.2 (data-final-fixes.lua)
11.266 ...
12.668 Detailed ModManager.cpp:645: Mod loading time: 11.3916

In this case, loading takes about 8 seconds, but if I enable all the mods I play with, then, as I already said, it's infinite :)))

What does the existing code do? It takes the path from the config, scans all existing entries (for example, 150 by default), checks the path, and if it finds a match, replaces it, and then iterates through the entries in the config. And here's the problem: other mods either add entries or replace them, and the current code scans them too. Let's say there are not 150 entries, but 300, and they all need to be scanned for matches. That's why the time increases. But why, say, 8 times instead of 2 times? Honestly, I don't know how this happens; I don't fully understand the principle. Perhaps it depends on where these entries are applied, in date/date.update/date.final.fix.

No problem, I expect to release a reworked mod for testing in a week, two at the most, meaning with all HD modules. I just need to compile the current structure correctly to avoid conflicts.
It's a shame there are no user statistics on the HD mods used. For example, if 90% of users always enable all HD mods, then the structure could be reworked, completely replacing them without modules, leaving only the basic ones (base, rails, quality, space age). But in this case, we need to create a modular system so that we can enable only the necessary ones, so it will take time.
But I also have some questions (rhetorical ones), for example, why enable only buildings if, say, all decorative ones won't be included, or enable some and not others. I understand that it depends on the computer and video card resources, but if some are enabled and others are not, then it looks so-so :(

By the way, I'll try cheating a bit on the remains and moving them into a separate module. That way, they'll fit perfectly into the modular system, and there won't be any conflict with the reworked code, since it will only iterate through records with remains. :)

“Wow… that was a sweaty match!” ))))
Basically, the mod is ready, but there are a few nuances.
Loading times are excellent:

4.320 Loading mod ~Factorio_HD_Base 2025.11.15 (data-final-fixes.lua) 
4.487 Loading mod ~Factorio_HD_Core 2025.10.30 (data-final-fixes.lua) 
4.534 Loading mod ~Factorio_HD_Space_Age 2025.11.15 (data-final-fixes.lua) 
4.669 Loading mod ~Factorio_HD_Elevated_Rails 2025.11.21 (data-final-fixes.lua) 
4.719 Loading mod ~Factorio_HD_Quality 2025.11.21 (data-final-fixes.lua)
4,768 Loading ...

Replacing all sprites (that is, reading and writing all prototypes) takes less than 1 second. The remaining time is the sprite loading itself. I didn't measure this last time, and I don't think the time has changed. Loading sprites into memory itself takes about 60 seconds for me:

11,283 Parallel sprite loader initialized (threads: 11, bitmaps: 13,319)
74,907 Sprites loaded

Now about the nuances:

1. The entire code has been rewritten, both the main code and the configs. Only the sprites themselves remain from the original :)

2. On the one hand, the configs have become a bit more complex, but on the other, they're simpler; they're now more flexible and customizable. The hardest part was creating them and entering all the data, but since that process has already been completed, working with them has become much easier.

3. There's no modular texture pack system, meaning all the sprites aren't divided into modules, as they were before.
   Nowadays, they're quite easy to separate into modules. You could even make a main mod and make everything else into texture packs, as already discussed, but since I haven't needed to separate them yet, I haven't done that.

4. There's a nuance in the mod names: each one is preceded by a tilde "~."
   This is done so that the symbol itself comes after the entire alphabet, meaning the mod loads only after all other possible mods, with the exception of those whose names also contain a tilde as the first character.
   However, this also creates a problem: the tilde is a control character in the Factorio engine, and specifying any mod (any mod at all) whose first character is "~" as a dependency for any mod won't work. I'm referring to the dependency in the info.json file. In all other cases, it has no effect.
   This means that ~HD_Base can't be specified as a dependency for ~HD_Space Age, but ~HD_Space Age currently uses tile sprites from ~HD_Base. I haven't figured out a way around this yet, as I didn't want to duplicate sprites, as was the case in the original versions. I don't know what else to come up with to ensure HD mods are loaded last. I haven't come up with anything else. Other characters don't work, although I haven't tested with a simple name without leading characters.

5. Added minor checks: if the path doesn't match, the sprite isn't loaded. Likewise, if the sprite exceeds its size, an error is returned stating that this shouldn't be done. :)

6. Changing the mod name is pretty simple; all the paths are bound to a single variable in the main code.

7. Okay, I think there was something else, I don't remember :(

All prototypes have been checked for correct sprite replacement; everything is fine, no problems.
But I haven't tested it with all the mods enabled yet. I only enabled the same ones as last time, and there are still no issues. Everything is fine, and I don't think there will be any once I enable all the main ones.
I also had the idea of ​​somehow calculating whether certain sprites affect the frame rate/ups so I could add an option to disable them, but I don't know how. The only thing I know is that in space, when there are a lot of meteorites and they fly apart during attacks, it puts a lot of strain on the game, but I don't know whether sprite size has any effect on this either; it seems the number of particles flying apart has a greater impact. Maybe I'll figure something out later.

I'll send you the link in 10-15 minutes while it's uploading to the cloud.

Regarding point 4:
That is, it is currently impossible to enable HD_Space_Age alone without HD_Base, similar to how it is impossible to enable the Space Age expansion without the base game.
To do this, you need to copy all the tile sprites from HD_Base (as before) and replace the paths in the main HD_Space_Age code.

Download and test link:
https://drive.google.com/file/d/1PvG6IBWVbqlFL2OHK_pAbDPxO9pVrN7X/view?usp=sharing

Wow, I'm really excited about what you've created. I'll try to take a closer look at it over the weekend.

At first glance, what you've created at the end looks to me as if all sprites are replaced directly. That was my initial approach until I came across the texture base mod.

I rejected that at the time because it wasn't able to change textures flexibly if they were also used by mods for other things or had already been changed by mods. It also saved me the effort of entering everything.

I'm currently considering whether it's possible to combine both approaches and use an option in the menu to specify which approach the mod should use. Something along the lines of:
Option 1 – Fast loading, only the textures of the main game are replaced
Option 2 – Slower loading, the textures are replaced everywhere they occur.

Regarding renaming the mod, that is not possible. To do so, you would have to re-upload the mod under a completely new name, and I would not want to do that.
Since everything is loaded in the third loading phase (data-final fixes.lua), I don't think “~” is necessary either.

At first glance, what you've created at the end looks to me as if all sprites are replaced directly. That was my initial approach until I came across the texture base mod.

Almost a direct replacement, but in a semi-automated mode. Essentially, this version does the same thing, but with a targeting system that tells us what to replace and where. The difference is that we don't search everything; we search specifically.

I'm currently considering whether it's possible to combine both approaches and use an option in the menu to specify which approach the mod should use. Something along the lines of:
Option 1 – Fast loading, only the textures of the main game are replaced
Option 2 – Slower loading, the textures are replaced everywhere they occur.

Please note that the structure has been slightly changed, the "data" folder has been removed, meaning the "base", "space_age", etc. folders are now in the root.

Regarding renaming the mod, that is not possible. To do so, you would have to re-upload the mod under a completely new name, and I would not want to do that.
Since everything is loaded in the third loading phase (data-final fixes.lua), I don't think “~” is necessary either.

No, I'm not talking about renaming the mod. Alternatively, we could release version 2.0 or an extended version, if that's what you're interested in. :)
The current version is quite sufficient for local use. I might finish the module system later (but I don't see much need for it right now), and I'll most likely add another texture pack for third-party mods like "Krastorio 2," "Alien Biome," and a few other "global" mods that add graphics; fortunately, that's pretty easy to do now. :)
But I could use some advice on how to properly upscale sprites. I assume that after upscaling (or during upscaling), the sprite has 8-bit color rendition, not 32-bit. What software is best for upscaling? :))))

Hi,

I've come up with an idea how we can combine your idea of faster loading with my lack of time.

What do you think about publishing your own mod on the mod portal? I would call it 'Factorio HD Age: Fast Loader'.

The basic idea is to use an if-function in my mods to suspend loading using the current method, allowing your mod to load the textures faster.

Of course, you would then need to invest some time to adapt it accordingly. This would probably also involve using if-functions in the Data-final-Fixes.lus file to determine which HD:Age mods are installed.

You could also add settings that disable loading via the fast method for individual mods, which would reactivate the method I use. This way, each user can decide how to load each texture pack and choose between the different methods as needed.

PROS:
- The fast loader functionality becomes available to everyone.
- You get credit for your work.
- I would be able to implement the if loops within my limited time budget.
- The slower loading method remains as a fallback.

Neutral:
- You would have your own mod to take care of.

CON:
- You would still have a lot of work ahead of you.

What do you think?
If you like the idea, I can explain in more detail how I imagine it working.

No-no-no :))))
I made this version solely because I wanted to play with HD textures, but I was really annoyed that with all the mods enabled, the loading time was about 15 minutes, and I really wanted to :)))
So, I decided, "How can this be? I need to figure this out," and finally got around to it, and that's how this version came to be.
I'm avoiding the portal because I don't have enough knowledge at the moment to fully support it, and there are a ton of local mods I haven't figured out yet. :)
Perhaps, if later you decide not to merge or publish this version, I'll consider making it publicly available, but for now I don't want to bother with that. :)

Hi,
OK, I was thinking too far ahead. :))
You did this out of your own interest and wanted to share your results so that others could benefit, too.
If you are still interested in sharing your progress with me, I would be very happy
I would like to implement this, but, as I said, I'm very busy at the moment.
Should you have any questions about the release of a "fast loader" mod in the future, please do not hesitate to get in touch.

To answer your question from yesterday...

But I could use some advice on how to properly upscale sprites. I assume that after upscaling (or during upscaling), the sprite has 8-bit color rendition, not 32-bit. What software is best for upscaling?

I use either the chaiNNer with models from OpenModelDB or Topaz Gigapixel for upscaling. You need to experiment with each texture to see what produces the best results.
For PNG optimisation and compression, I use batch files that I have written myself.

Boss!
Help me out! :)))
I'm in the final stages of testing, or rather, almost finished, but there's still one last "finishing touch"...
I'm creating upscaled tile sprites for the "Alien Biome" mod. It's quite common, but when enabled, it replaces almost all tiles created in the base game or add-on, effectively eliminating the HD effect.
When using "Topaz Gigapixel AI", the tiles appear with striped artifacts along the edges :(((
Your sprites are very well done, and I assume they were made with "chaiNNer." Could you share the preset settings (file or at least a screenshot) and the model you used to upscale the tiles?

Here's what we got from our testing.
Test parameters: all mods (over 500) are enabled, measuring the time from game launch to full loading (until the game menu screen appears).
Without HD mods, the loading time was 149 seconds.
With HD mods, the loading time was 196 seconds.
So, the difference was 47 seconds, and essentially, this is the loading time for HD sprites. I think we did it. :))))

And this is the video memory:

196.280 Video memory usage: 11595.17 MB (Atlases: 11224.81 MB, Textures: 370.36 MB)

By the way, thanks for the tip on "chaiNNer", it's a cool program with excellent implementation, and for the batch files :)))

Hi,

When using "Topaz Gigapixel AI", the tiles appear with striped artifacts along the edges

Take a close look at the edges of the textures at 1000x zoom.
The trick with the ground textures is to break them down before upscaling (128x128, 256x256, and 512x512 tiles) so that the edges of the individual tiles match the edges of the image. Otherwise, the upscalers will try to create a transition. Depending on the texture, I then performed the upscaling with ChaiNNer or Gigapixel and then merged the individual images back together.

eliminating the HD effect

Did you integrate the textures manually, as with my mods, or did you use automated code?
The Factorio code itself does not allow scaling and will always load, for example, a 128x128px tile at position x:512px y:128px of the texture, regardless of the image size.
This makes the textures appear muddy and blurry if you change the image source automated.

Oh, that sounds pretty complicated :(
I made an additional mod for third-party mods that uses mixed mode: specifically for these tiles, I wrote a different code, a shortened version, which does the replacement, but I'm having trouble with the upscaling itself and the file sizes :((((
https://drive.google.com/file/d/1SLO-3-7M8F_hSrjuOLM9qLEaEEYVRTK4/view?usp=sharing

Oh, that sounds pretty complicated

Take your time to see how I implemented this with the textures.
You can use Chainner to split up the textures, but dealing with thousands of small files afterwards is a pain.

but I'm having trouble with the upscaling itself and the file sizes

What exactly do you mean by file size, and what exactly is your problem with upscaling?

Well, the thing is, I got sprites with artifacts, and the sprite size increases ten to fifteen times :)
I'm completely clueless about upscaling :(
Okay, I'll try to figure it out with Chainner over time.

Hi,
I found some time to add a quick load option to the Quality and Base Game - Logistics mod.

I revised the existing code by removing all irrelevant and obsolete steps for the purpose of replacing upscaled textures (with always the same Settings). I also converted some recursive functions into iterative ones and added early termination points to prevent unnecessary loop iterations.

If you would like to, you can try it out and let me know how much of an improvement you get.
In my case, I managed to reduce the loading times for the Logistics and Quality packages from 2.5 and 0.5 seconds, respectively, to 0.25 and 0.19 seconds.

If I find the time, I will check the results in detail (hence the experimental status). If I am satisfied with the results, I will roll out the whole thing to all mods, always with the fallback option.

Well, the thing is, I got sprites with artifacts, and the sprite size increases ten to fifteen times :)
I'm completely clueless about upscaling :(

Could you please link me an example (the original texture and the unstatisfying result) and I'll take a look at it.

Hello!
Regarding mod loading, I downloaded both and here are the results. I tested with all mods enabled, from the game launch until the main menu screen appeared.

In the default version, the loading time was:

28,211 Loading mod factorio_hd_age_base_game_logistics 1.1.3 (data-final-fixes.lua)
104,248 Loading mod --- loaded in 76 seconds ---

117,569 Loading mod factorio_hd_age_quality 1.1.5 (data-final-fixes.lua)
180,059 Loading mod --- loaded in 62.5 seconds ---

286,412 Sprites loaded --- total loading time ---

When loading with "fast" mode enabled, the loading time was:

26,941 Loading mod factorio_hd_age_base_game_logistics 1.1.3 (data-final-fixes.lua)
30,784 Loading mod --- loaded in ~3.9 seconds ---

43,306 Loading mod factorio_hd_age_quality 1.1.5 (data-final-fixes.lua)
48,375 Loading mod --- loaded in 5 seconds ---

145,957 Sprites loaded --- total loading time ---

The result is excellent, the overall loading time has been cut in half!
The loading time for the mods themselves has been reduced significantly! Excellent solution!

Regarding the sprites, of course, here's the link. There are two files: the original and the sprite upscaled using "Topaz Gigapixel AI."
https://drive.google.com/file/d/1ecd1MPBwymlooas4zPNp2S04xcl-a057/view?usp=sharing

I managed to reduce the loading times by a little more. I would be grateful if you could let me know how well it performs for you.

I took a look at your upscaling and noticed two things.

1. There is something wrong with the transparency. Either you used an upscaling model that does not support transparency, or there is something wrong with your Gigapixel settings.

2. The edges of the individual tiles in the sprite sheet are not clearly defined in your upscaling. This is the phenomenon I mentioned earlier: you have to break the texture down into its individual tiles before you can upscale it.

I managed to reduce the loading times by a little more. I would be grateful if you could let me know how well it performs for you.

  27.726 Loading mod factorio_hd_age_base_game_logistics 1.1.4 (data-final-fixes.lua)
  30.449 Loading mod --- loaded in 2.7 seconds ---

  43.134 Loading mod factorio_hd_age_quality 1.1.6 (data-final-fixes.lua)
  46.740 Loading mod --- loaded in 3.6 seconds ---

Thank you.
From my point of view, those are acceptable loading times. :)
Since I don't see any further potential for improvement with this method I will gradually implement this for all mods.

Hi,
I've now updated all the mods and I'm achieving an average time saving of around 87%.

Hi!
This is absolutely fantastic news! I'm glad it all worked out! :))))