Mortar Turrets, while effective at their job, have to lead their shots quite far in order to hit their targets, because of their range. This means that when bugs start to get close to your walls, and say you've got a mortar on an adjacent wall or deeper within your base, there's a bad tendency for the mortar to over-lead the shot and deal damage to friendly structures, even ones deeper within your base than you might expect. Rocket turrets have a similar problem, but the damage tends to be limited to the walls and some friendly turret damage, and is compensated by the fact that rocket turrets are so effective.

I don't know if there's an effective fix to mortar targeting to make it so they don't cause damage so deep into your own base. My suggestion would be to limit the firing arc on mortars so you limit more effectively which mortars fire where. The enemy mortar turret that you can find sometimes can keep its 360 targeting though.