Only read our public key if it wasn't already in the private key file.