Hmm... a tricky issue. As you say, placing the heat pipes, then the inserter, allows both the inserter and the hidden heat pipe to conduct heat. Placing the inserter first, then heat pipes, only allows the hidden heat pipe to conduct heat; the inserter remains unconnected at 15 degrees. If I disable the control.lua that places the hidden heat pipe, then the heat inserter conducts heat and works perfectly in both situations.
I'm not entirely sure how to make sure it works both ways... the core issue appears to be the existence of the hidden heat pipe. Placing the inserter when heat pipes are already laid out, the heat inserter makes the initial connections, and then the script adds a hidden heat pipe underneath that also makes the connections.
But when the heat inserter is placed first, no initial connections are made, then the script adds the hidden heat pipe, which appears to override any future connections that would be made afterwards.
And of course, the entire reason that the hidden heat pipe was added, is because inserter-type entities can't thaw out buildings next to them. Tricky tricky...
EDIT: That being said, I'm not entirely sure why rotating the heat inserters fixes the connection issues; there's no script events tied to inserter rotation, at least on my end. I also tried playing with the collision boxes of both entities, and it didn't seem to make any difference in the connection logic.