Where to store SSH keys safely (besides ~/.ssh)

For Git over SSH and server login, the platform-standard location is still ~/.ssh — with correct permissions, passphrases, and often ssh-agent. The question “besides ~/.ssh” usually means one of three things: you want stronger protection (hardware keys), you want to avoid key sprawl, or you are mixing SSH keys with other developer secrets (API tokens) and want one mental model.

This guide separates those cases.

1. Baseline: ~/.ssh is not wrong

OpenSSH expects:

~/.ssh/id_ed25519       # private key (chmod 600)
~/.ssh/id_ed25519.pub   # public key
~/.ssh/config           # Host aliases, IdentityFile paths
~/.ssh/known_hosts      # server fingerprints

Verify permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

Official SSH documentation: OpenSSH manual pages (platform man pages: man ssh, man ssh_config).

2. Strong upgrade: passphrases + ssh-agent (macOS)

Always encrypt private keys at rest with a passphrase. On macOS, the keychain can store the passphrase so you are not typing it every push — this is a convenience vs exposure trade-off; understand your org policy.

Add keys to agent:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

GitHub documents macOS Keychain integration for passphrases: Working with SSH key passphrases.

3. Stronger upgrade: hardware security keys (FIDO2 / ed25519-sk)

For GitHub, GitLab, and many orgs, SSH keys backed by YubiKey (or similar) mean the private key does not exist as a normal file on disk in the same way. See GitHub’s guide Generating a new SSH key for a hardware security key.

When this wins: high-assurance orgs, protection against disk forensics on a stolen powered-on machine (still not magic against all attacks).

4. Per-client, per-service keys (reduce blast radius)

Instead of one id_rsa for everything:

# ~/.ssh/config (excerpt)
Host github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_ed25519_github
  IdentitiesOnly yes

Host gitlab.acme.internal
  User git
  IdentityFile ~/.ssh/id_ed25519_acme_gitlab
  IdentitiesOnly yes

If one key is compromised, revoke only that deploy key in the provider UI.

5. What not to do

  • Copy private keys to Slack, email, or tickets — ever.
  • Store private keys in iCloud-synced notes or repos (even “private”).
  • Use no passphrase on laptops that leave the house.

6. When a developer vault (e.g. PassStore) fits

PassStore is aimed at API keys, tokens, and env-style secrets — not replacing OpenSSH’s key files. Many teams use:

  • ~/.ssh for SSH key material (possibly hardware-backed).
  • PassStore for everything else (Stripe, OpenAI, database URLs, signing secrets) — Security.

That split keeps SSH semantics standard while still getting encrypted vault ergonomics for non-SSH material.

7. Rotation

If a private key might have leaked: remove the public key from GitHub/GitLab, generate a new keypair, re-add to services, and audit git remotes for unexpected URLs. Pair with rotate API keys safely for PATs used alongside SSH.

Related