This happens to me all the time. If I break a factory with a full inventory, it will give me the standard gui error about not being able to insert into inventory. But the factory will just disappear.
Also, this will happen even if you have one or more spare slots on your toolbar. It only checks and tries to insert into your inventory space.