Kafkatorio Events


Sends in-game information to a server over the internet (requires additional infrastructure)

2 years ago
1.1
65

g Development link? I'd like to help out

2 years ago

Exciting project idea! I was thinking about developing an in-browser renderer (to display live train maps for example) and this could really help out. Do you have a GitHub project for this or any ideas on how I could help out?

2 years ago

Hey zablex, thanks a lot for the interest! Displaying live trains would certainly be possible - but it would need a lot more work. At the moment I'm trying to fix it so the required servers are deployable (e.g. all the URLs are hardcoded to localhost), and also sharing a demo video. I'll try and get it to a working state.

And thanks for the offer of help! The tech I'm using is

  • TypeScript to Lua to make the mod - the mod logs JSON messages to the server log
  • a shell script that will scrape the Factorio game server log and upload messages to a Kafka server. This isn't ideal as it requires the Factorio server is running in a Docker container, and the shell script has elevated permissions
  • a Kafka server (no special changes)
  • a Kafka Streams application (Kotlin) that will process the messages (this generates PNGs of the map terrain, and other events processing)
  • a websocket + map-tile Kotlin web server that will serve the PNGs and broadcast updates (using Http4k, but I want to migrate to Ktor)
  • a Leaflet webmap (I'm using Kotlin+KVision, but I think I might try Jetbrain's Compose Multiplatform)

I'm least experienced with frontend development, so help there would be good. I chose Kotlin+KVision for the challenge, but it can be written in anything. It would even be possible to connect directly to the Kafka server and generate metrics (in Grafana, for example).

What would really help is persuading the Factorio devs to add a websocket broadcast server https://forums.factorio.com/viewtopic.php?t=57281, or UDP out (like I know the Forza games have https://github.com/geeooff/forza-data-web). That would cut out the hacky shell script. Apart from that, maybe figuring out a better way to get data out - maybe over a RCON connection? I think Clusterio does that.

2 years ago

I've posted an announcement of the POC here: https://forums.factorio.com/viewtopic.php?f=190&t=102841&p=570598#p570598

Some more technical updates

  • I've found a workaround for the shell script - Docker syslog log driver. Docker will forward the Factorio logs to my backend server. It works nicely! However it has issues with connectivity and if the backend server isn't available then the logs go missing. I may try creating my own Docker log driver plugin.
  • I've migrated from http4k to Ktor - it's much more convenient and stable.
  • I've made some optimisations around tile grouping and tile generation so tile updating is really fast now (relatively speaking - it's not real time but the map will update in a matter of seconds)
2 years ago

Hi zablex, I've made the source code available - please check it out!

https://github.com/adamko-dev/kafkatorio/

New response