What colour is fire? Trick question, it depends on the chemistry of the fuel and the temperature of the furnace.
Nicer Fuel Glow exchanges Factorio's standard white glow for a tasteful "pine embers" to "Cherenkov blue" scale, along with some one-off exceptions for nuclear fuel (Radium green), electric furnaces (hot-metal orange) and oil refineries.
Affects: Boilers; stone, steel and electric furnaces; burner inserters, oil refineries. Mod entities adapted from those should also be supported, as well as reactors with the "use_fuel_glow_color" tag.
All modded fuels are supported, but a little configuration will be required for good results. Please consult the settings in-game.
A short video demonstration is available here, courtesy of JD-Plays.
For non-burner beautification, I highly recommend Heat Glow, along with Lightorio and Disco Science.