[snarfed][manton] one simple technique is to encrypt users' private keys in storage, password-protect the master key you encrypt them with, and require typing in that password when you boot your backend servers. depends on details of your stack and hosting setup, but works well
