Ghosts display the arm in the position it's built in, which is always 180° from the drop-off. The arm only moves to the pickup position after it is built and receives power.
To see the correct pickup direction on ghosts you need to enable Show inserter arrows in "Alt-mode" in Interface Settings