Managing Secrets with HashiCorp Vault

Secure storage and management of API keys, credentials, and sensitive data

Managing sensitive information like API keys, database credentials, and certificates is a critical security challenge for modern applications. HashiCorp Vault provides a centralized, secure solution for storing, accessing, and managing secrets with robust access controls, audit logging, and encryption at rest.

This guide demonstrates practical secrets management using Vault’s Key-Value (KV) secrets engine, covering everything from initial setup to production operations.

Prerequisites

• HashiCorp Vault installed and running
• Vault CLI configured and authenticated
• Administrative access to enable secrets engines
• API keys or credentials to store (for testing purposes)

Understanding Vault Secrets Engines

Vault uses secrets engines to handle different types of sensitive data. The Key-Value (KV) engine is the most commonly used for storing static secrets like API keys, passwords, and configuration data.

Step 1: Verify and Enable Secrets Engine

Check Current Secrets Engines

Before storing secrets, verify which engines are available:

vault secrets list

Expected Output:

Path          Type         Accessor              Description
----          ----         --------              -----------
cubbyhole/    cubbyhole    cubbyhole_12345678    per-token private secret storage
identity/     identity     identity_87654321     identity store
sys/          system       system_abcdef123      system endpoints used for control

Enable KV Secrets Engine

If the KV engine isn’t mounted at the secret/ path, enable it:

# Enable KV v2 (recommended for production)
vault secrets enable -path=secret kv-v2

# Verify successful enablement
vault secrets list

Step 2: Store API Keys and Credentials

Store Individual API Keys

Store your sensitive API keys using organized path structures:

# Store OpenAI API key
vault kv put secret/api-keys/openai \
  api_key="sk-proj-your-openai-key-here" \
  organization="your-org-id"

# Store Anthropic API key  
vault kv put secret/api-keys/anthropic \
  api_key="your-anthropic-key-here"

# Store additional service credentials
vault kv put secret/api-keys/github \
  token="ghp_your-github-token" \
  username="your-github-username"

Store Application Configuration

Group related secrets for easier management:

# Database credentials
vault kv put secret/myapp/database \
  username="db_admin" \
  password="secure_db_password" \
  host="db.example.com" \
  port="5432"

# Application settings
vault kv put secret/myapp/config \
  app_secret="your-app-secret-key" \
  jwt_secret="jwt-signing-key" \
  encryption_key="aes-256-key"

Step 3: Retrieve Secrets

View Complete Secrets

Retrieve all key-value pairs for a secret:

# Get full secret with metadata
vault kv get secret/api-keys/openai

# Get secret in JSON format (useful for automation)
vault kv get -format=json secret/api-keys/openai

Sample Output:

====== Secret Path ======
secret/data/api-keys/openai

======= Metadata =======
Key                Value
---                -----
created_time       2025-09-25T10:30:00.123456Z
custom_metadata    
deletion_time      n/a
destroyed          false
version            1

========= Data =========
Key             Value
---             -----
api_key         sk-proj-your-openai-key-here
organization    your-org-id

Extract Specific Values

Get individual field values for use in scripts and applications:

# Get only the API key value
vault kv get -field=api_key secret/api-keys/openai

# Get database password
vault kv get -field=password secret/myapp/database

# Store value in environment variable
export OPENAI_API_KEY=$(vault kv get -field=api_key secret/api-keys/openai)

Step 4: Advanced Secrets Management

Version Management (KV v2 Only)

KV v2 maintains version history for all secrets:

# View secret metadata and versions
vault kv metadata get secret/api-keys/openai

# Get specific version
vault kv get -version=1 secret/api-keys/openai

# Update secret (creates new version)
vault kv put secret/api-keys/openai \
  api_key="sk-proj-updated-key" \
  organization="new-org-id"

Secret Lifecycle Management

Manage secret versions and deletions:

# Soft delete (recoverable)
vault kv delete secret/api-keys/old-service

# Delete specific versions
vault kv delete -versions=1,2 secret/api-keys/openai

# Permanently destroy versions (irreversible)
vault kv destroy -versions=1,2 secret/api-keys/openai

# Undelete soft-deleted secret
vault kv undelete -versions=3 secret/api-keys/old-service

Patch Operations

Update individual fields without affecting others:

# Update only the API key, preserve other fields
vault kv patch secret/api-keys/stripe \
  api_key="sk_live_new-stripe-key"

# Add new field to existing secret
vault kv patch secret/myapp/config \
  redis_url="redis://cache.example.com:6379"

Step 5: Integration with Applications

Environment Variable Loading

Create scripts to load secrets into environment variables:

#!/bin/bash
# load-secrets.sh

# Load API keys
export OPENAI_API_KEY=$(vault kv get -field=api_key secret/api-keys/openai)
export ANTHROPIC_API_KEY=$(vault kv get -field=api_key secret/api-keys/anthropic)

# Load database credentials  
export DB_USERNAME=$(vault kv get -field=username secret/myapp/database)
export DB_PASSWORD=$(vault kv get -field=password secret/myapp/database)
export DB_HOST=$(vault kv get -field=host secret/myapp/database)

# Start application with loaded secrets
./start-application.sh

Application Integration Examples

Python Application:

import subprocess
import os

def get_vault_secret(path, field):
    """Retrieve secret from Vault"""
    result = subprocess.run([
        'vault', 'kv', 'get', '-field=' + field, path
    ], capture_output=True, text=True)
    
    if result.returncode == 0:
        return result.stdout.strip()
    else:
        raise Exception(f"Failed to retrieve secret: {result.stderr}")

# Usage
openai_key = get_vault_secret('secret/api-keys/openai', 'api_key')
db_password = get_vault_secret('secret/myapp/database', 'password')

Troubleshooting Common Issues

Error making API request.
Code: 403. Errors: * preflight capability check returned 403

# Enable KV v2 at secret/ path
vault secrets enable -path=secret kv-v2

Error: no handler for route 'secret/mykey'

# Verify secrets engine is mounted
vault secrets list

No value found at secret/data/mykey

# List available secrets
vault kv list secret/

Security Best Practices

Path Organization

secret/
├── api-keys/
│   ├── openai
│   ├── anthropic
│   └── github
├── databases/
│   ├── production
│   └── staging
└── applications/
    ├── webapp/config
    └── api/credentials