This modpack bundles all of the Factorio HD Age mods for the Base Game and the Space Age DLC. It doesn't contain any content itself. It only references to the other Factorio HD Age mods as required dependencies. A total of at least 13 GB of VRAM is recommended.
Collections of mods with tweaks to make them work together.
Transportation of the player, be it vehicles or teleporters.
Augmented or new ways of transporting materials - belts, inserters, pipes!
Trains are great, but what if they could do even more?
New ways to deal with enemies, be it attack or defense.
Armors or armor equipment.
Changes to enemies or entirely new enemies to deal with.
Map generation and terrain modification.
New Ores and resources as well as machines.
Things related to oil and other fluids.
Related to roboports and logistic robots.
Entities which interact with the circuit network.
Changes to power production and distribution.
More than just chests.
Greetings!
So, I'm back to HD mods again :))))
Here's what I came up with...
Original problem:
The original code traverses the entire data.raw (all ~150+ prototype types in the game, without third-party mods) to find sprite paths and replace them with HD versions. This takes me ~9 seconds. I tested it for "factorio_hd_age_base_game_production" with several mods enabled, including global ones like Krastorio2, Nexus, Metal_and_Stars and a couple more. When enabling a large number of third-party mods, the number of entries increases, and for each sprite, all existing data.raw entries are constantly traversed.
Solution:
Added prototype type information to the mod config via the "type" parameter. For example:
["accumulator"] = { __type__ = "accumulator", ["*"] = {}, ["remnants"] = {} }
["steel-furnace"] = { __type__ = "furnace", ["*"] = {}, ["remnants"] = {} }
Now the code:
Scans the config and collects a list of used types (accumulator, furnace, mining-drill, etc.)
Traverses only these types in data.raw instead of all existing ones
The rest of the path replacement logic remains unchanged
Result:
Before: 9 seconds per pack (traversing all type entries)
Now: 1.2 seconds (traversing only ~10-15 required types)
Acceleration: ~750% (7.5 times faster)
One downside is that to reduce loading time, you'll have to add entity types to each config. Well, that's if you actually want to solve this problem. I'll link to the re-uploaded test mod if anyone's interested.
The advantage is that there is minimal modification of the main code.
For my own use, I'll rewrite it slightly differently, since currently each HD mod is essentially a standalone, separate mod, but the code in each one is the same. So, if we have 22 HD mods, we'll be executing the same code 22 times.
I'll make only one main "core" mod, and all the others will be texture packs only. That is, if a texture pack is present and enabled, it will load. The concept remains the same, but we'll avoid executing the same code more than 20 times, which will further reduce loading times.
However, I'm not yet sure how best to do this.
As a result, if in the current version I had a problem with the loading time of all 500+ enabled mods and all HD mods, and the time was over 14 minutes, then with this version I expect the loading time to be about 2 minutes.
Link to mod for testing:
https://drive.google.com/file/d/1JwkRSKTEe5j8tBqpJWb_JXH-Ojg9h30Y/view?usp=sharing
Hi, wow, I'm impressed. I'll try that out right away. If it works well, I'd like to implement it in all mods and give you a shout-out.
I also had the idea of doing everything via a main mod and then uploading the textures without the repetitive code. Unfortunately, I can't remember what stopped me from doing that. I'll think about it again.
Well, I hope your testing goes well too :)
I'm taking a time-out for now, as I'm really confused between the options. I was looking for the one with the minimum code changes, and now my head is a jumble of all the options, and I can't seem to figure out the code structure for a "core" mod. I understand what's needed, but I can't quite put it together :(
Well, and accordingly, as soon as I do this, I will share the option and my thoughts.
Huge respect for you figuring that out.
Yes, at some point you just see letters and brackets.
You changed the main code, which I accepted as given.
I'll need some time to figure out what you did there.
Apart from that, I have noticed that although the entities have an HD texture, the remains only load in SD with your changes.
So it seems to work only for entities but not for their modified or altered forms.
Yeah, there's a problem :(
What we have:
All the remnants are stored in the data.raw["corpse"] records. Previously, the code would go through all the records, including this one, and if it found a path match, it would replace the sprite.
Now, when we specify specific records for an entity, it no longer checks the data.raw["corpse"] record. Consequently, it doesn't find the path and doesn't replace the sprite.
What to do:
Somehow add the data.raw["corpse"] record to the bypass. And here, too, I'm having problems, because if you add a record, it checks all entities, regardless of whether we specify a match by path ["remnants"] = {} or not.
Overall, I had to work hard, and it turned out to be more difficult than I thought
I think I've figured out a solution, but need to test it with other HD mods.
I've reuploaded the mod, and if it's not too much trouble, please test it with other mods.
Just note that I intentionally removed the ["remnants"] parameter from one of the entities in the config:
Instead of:
["chemical-plant"] = { __type__ = "assembling-machine", ["*"] = {}, ["remnants"] = {} },
it now says:
["chemical-plant"] = { __type__ = "assembling-machine", ["*"] = {} },
since I tested it on it, and then need to return the record with ["remnants"] = {}.
Result:
Now all the specified entities, except ["chemical-plant"], have their remnants sprites replaced with HD ones.
Ahh, and I also removed the logs from the added code, since in fact they were only used for testing, the rest of the logs were not changed.
I hope I haven't screwed up anywhere else :)))
Overall, the result looks good, loading takes just 1 second, and those with a more powerful processor will get even faster (or, conversely, for some, it will be slower):
3.786 Loading mod factorio_hd_age_base_game_production 1.1.2 (data-final-fixes.lua)
4.776 Loading mod ...
But again, need to test it in other mods, and if there are any errors, need to think about how to fix them.
Just in case, the re-downloaded mod should have the "processRemnants(entity_name)" function added to the end of the code.
What I really don't understand right now is why the loading time is changing, even though I've enabled another mod... and purely theoretically, the time should be either the same or longer, but not shorter... ????
3.514 Loading mod factorio_hd_age_base_game_production 1.1.2 (data-final-fixes.lua)
4.371 Loading mod ...
What I really don't understand right now is why the loading time is changing, even though I've enabled another mod... and purely theoretically, the time should be either the same or longer, but not shorter... ????
In my experience, background processes always affect loading times. It's possible that a background process is interfering with your reproducible results.
In this context, I'm wondering what kind of hardware you're using. Specifically, I'd like to know what type of processor you have, and if you've installed the game on an NVMe drive. I can't remember if you've told me this already. I thought about the processor because you said that loading an individual HD mod takes 9 seconds. For my friend and me, it only takes about 1 up to 1,5 seconds per mod. This leads me to assume that your hardware is weaker and that background processes therefore have an even greater impact on performance.
In general, I have been considering what other factors need to be taken into account.
It should be possible to assign multiple type entries per folder. Depending on the folder, several types may be represented. For example, the Recycler folder contains both normal and remnant textures. This means that two type variables would have to be defined for one folder.
Assigned type entries should be automatically applied to all subfolders.
Unfortunately, I don't have enough time at the moment to examine the code thoroughly and try things out myself.
I'm hoping that, between Christmas and New Year's, I'll find at least one day when I can commit to this task entirely.
However, if you are still interested in sharing your results with me, I would be very happy to hear from you. I would also be happy to share my thoughts and feedback with you, but I can't promise when I will be able to do so.
Processor:
System Information: [CPU: AMD Ryzen 5 5600, 6 cores, 12 cores, RAM: 20887/65462 MB, Page: 22671/65462 MB, Virtual Memory: 4359/134217727 MB, Extended Virtual: 0 MB]
And if we talk about a clean boot, with only mods enabled:
1. base
2. elevated-rails
3. quality
4. space-age
Then loading one mod (for example, factorio_hd_age_base_game_production) takes me:
1,866 Loading the mod factorio_hd_age_base_game_production 1.1.2 (data-final-fixes.lua)
2,726 ...
4,601 Details ModManager.cpp:645: Mod loading time: 3.24068
According to the logs, loading a mod takes just under 1 second, and loading all mods takes 3.2 seconds.
But if I enable other mods (roughly the minimum set I need) that add data.raw entries, the picture starts to change:
3.468 Loading mod factorio_hd_age_base_game_production 1.1.2 (data-final-fixes.lua)
11.266 ...
12.668 Detailed ModManager.cpp:645: Mod loading time: 11.3916
In this case, loading takes about 8 seconds, but if I enable all the mods I play with, then, as I already said, it's infinite :)))
What does the existing code do? It takes the path from the config, scans all existing entries (for example, 150 by default), checks the path, and if it finds a match, replaces it, and then iterates through the entries in the config. And here's the problem: other mods either add entries or replace them, and the current code scans them too. Let's say there are not 150 entries, but 300, and they all need to be scanned for matches. That's why the time increases. But why, say, 8 times instead of 2 times? Honestly, I don't know how this happens; I don't fully understand the principle. Perhaps it depends on where these entries are applied, in date/date.update/date.final.fix.
No problem, I expect to release a reworked mod for testing in a week, two at the most, meaning with all HD modules. I just need to compile the current structure correctly to avoid conflicts.
It's a shame there are no user statistics on the HD mods used. For example, if 90% of users always enable all HD mods, then the structure could be reworked, completely replacing them without modules, leaving only the basic ones (base, rails, quality, space age). But in this case, we need to create a modular system so that we can enable only the necessary ones, so it will take time.
But I also have some questions (rhetorical ones), for example, why enable only buildings if, say, all decorative ones won't be included, or enable some and not others. I understand that it depends on the computer and video card resources, but if some are enabled and others are not, then it looks so-so :(
By the way, I'll try cheating a bit on the remains and moving them into a separate module. That way, they'll fit perfectly into the modular system, and there won't be any conflict with the reworked code, since it will only iterate through records with remains. :)