Welcome to another exciting lesson of our course - Mastering Cloud Engineering with AWS and Python. Today, we delve into managing encryption keys with the AWS Key Management Service (AWS KMS). In previous lessons, we covered securing AWS resources and managing secrets using AWS Secrets Manager and AWS SSM Parameter Store. Now, we will extend that knowledge by learning how to create, control, and use AWS KMS keys and how to encrypt and decrypt data using AWS KMS.

The AWS Key Management Service (AWS KMS) is a managed service that helps you create and manage cryptographic keys for data protection. It supports various key types, including:

• AWS KMS keys: AWS KMS keys are the core resources in AWS KMS, used to encrypt up to 4 KB of data directly and manage other cryptographic operations. There are two types of AWS KMS keys:

  • AWS Managed KMS Keys: Automatically created and managed by AWS for specific integrated services. These keys are used for default service encryption and are managed by AWS, including their lifecycle operations like rotation.
  • Customer Managed KMS Keys: Created and managed by you, offering more control, including policy management, key rotation, and auditing. Unlike AWS managed KMS keys, these incur usage charges.

• Data Keys: Generated by AWS KMS and used to encrypt your data outside of KMS. Data keys are encrypted under an AWS KMS key and can be decrypted to plaintext only when needed, ensuring secure data handling practices.

• Asymmetric Keys and Key Pairs: For operations requiring separate keys for encryption and decryption or digital signing, AWS KMS supports asymmetric KMS keys, generating a public and a private key pair. The private key remains secure within KMS while the public key can be used externally.

AWS KMS is designed for security, meeting compliance requirements with its hardware security modules (HSMs) that are FIPS 140-2 validated, making it suitable for managing sensitive data across AWS services. To learn more, refer to the official AWS KMS documentation.

Note: The term customer master key (CMK) is an old term that means the same as AWS KMS key. You might still encounter CMK in some contexts, but know that it refers to AWS KMS keys.

To grasp the distinction and practical use of AWS KMS keys and data keys in AWS Key Management Service (AWS KMS), consider a secure document storage service. This service allows users to securely store and access confidential documents in the cloud, highlighting the necessity for both AWS KMS keys and data keys.

• Use of AWS KMS Keys:

  • Central Management and Policy Control: AWS KMS keys serve as the foundation of the encryption strategy, enabling centralized control over encryption policies. For example, specific policies can restrict encryption and key generation activities to certain application components or users, reinforcing security and access control.
  • Audit and Compliance: The ability to audit the use of AWS KMS keys through AWS CloudTrail helps meet compliance requirements. These logs provide valuable insights into when and by whom the AWS KMS keys were accessed, which is crucial for security audits.

• Use of Data Keys:

  • Efficient Document Encryption: Given the direct encryption with AWS KMS keys in AWS KMS is not suited for large volumes of data due to size and cost constraints, data keys are generated to encrypt the actual documents. Each uploaded document triggers the generation of a unique data key, enhancing efficiency and scalability.
  • Document Encryption Process: When a document is uploaded, the application utilizes an AWS KMS key to generate a data key in two forms: a plaintext version for immediate encryption of the document, and an encrypted version stored alongside the encrypted document. This ensures that the encryption process is both secure and efficient.
  • Secure Access and Sharing: To access a document, the encrypted data key is decrypted using the AWS KMS key into plaintext, then used to decrypt the document. This process allows for secure and scalable document access, with each document encrypted under a unique key, significantly minimizing the risk of a security breach.

This example illustrates how AWS KMS keys and data keys complement each other within a secure encryption strategy. AWS KMS keys facilitate the high-level management and policy enforcement, making them the backbone of the encryption architecture. In contrast, data keys, generated for specific encryption tasks, offer the flexibility and efficiency needed for encrypting large amounts of data. Together, they enable a secure, scalable, and compliant document storage service.

AWS KMS keys are primary resources in AWS KMS; they are used to encrypt and decrypt up to 4 KB of data. Essentially, you interact with an AWS KMS key when you encrypt or decrypt data. In creating an AWS KMS key, we specify certain parameters like Description, KeyUsage, and Origin, which customize the key's creation and its usability in AWS. Let's create an AWS KMS key:

• KeyUsage Parameter: The KeyUsage parameter determines how the AWS KMS key will be used. The allowed values are:

  • ENCRYPT_DECRYPT: The AWS KMS key can be used for both encryption and decryption operations.
  • SIGN_VERIFY: The AWS KMS key is used for digital signing operations or to verify digital signatures, rather than encryption or decryption.
  • GENERATE_VERIFY_MAC: The AWS KMS key is used to generate and verify message authentication codes (MACs), providing data integrity and authenticity.
  • GENERATE_KEY_PAIR: The AWS KMS key is used to generate key pairs, typically for asymmetric cryptographic operations.

• Origin Parameter: The Origin parameter defines the source of the key material for the AWS KMS key. Possible values are:

  • AWS_KMS: Key material is generated and managed within AWS KMS.
  • EXTERNAL: Key material is generated outside of AWS KMS and imported. This option allows you to use your key material in AWS KMS operations but keep it in a hardware security module (HSM) that you control.
  • AWS_CLOUDHSM: Key material is generated and managed in AWS CloudHSM but used within AWS KMS. This combines the integration and ease-of-use features of AWS KMS with the control and security of AWS CloudHSM.
  • EXTERNAL_KEY_STORE: Key material is generated and managed in an external key store but used within AWS KMS. This allows for tighter control and compliance by integrating with an organization's existing key management infrastructure.

The KeyUsage and Origin parameters together determine the capabilities and properties of your AWS KMS key. By understanding and configuring these parameters, you can tailor your AWS KMS key to suit your specific security and operational requirements.

To gain insights into a specific KMS key, you can use the describe_key function. This function provides detailed information about the specified key, such as its state, creation date, and various settings. Below is a short example that describes a key and its response parameters:

The response from the describe_key operation contains several key pieces of information:

• KeyId: The unique identifier of the AWS KMS key.
• Arn: The Amazon Resource Name (ARN) of the key.
• CreationDate: The date and time when the key was created.
• KeyUsage: Specifies the cryptographic operations for which the key is used, e.g., ENCRYPT_DECRYPT.
• KeyState: The current state of the key, such as Enabled, Disabled, or PendingDeletion.
• Description: The description you provided when creating the key.
• Origin: The source of the key material, such as AWS_KMS, EXTERNAL, or AWS_CLOUDHSM.
• KeyManager: Specifies whether the key is managed by AWS or the customer (values like AWS or CUSTOMER).

By using the describe_key function, you can easily retrieve detailed metadata about your AWS KMS keys, assisting with key management and audit requirements.

Now, we examine how to encrypt and decrypt data using the AWS KMS key we just created. First, we need to extract the KeyId from the AWS KMS key response. Afterward, we can use this key to encrypt and decrypt data. As an example, let's encrypt the text string 'Hello, AWS!', and then decrypt it.

As highlighted previously, directly using AWS KMS keys for encryption and decryption tasks isn't typically advisable, especially for files larger than 4KB, where it becomes impractical due to AWS KMS limitations. With this understanding, let's shift our focus towards the pivotal role of data keys in ensuring efficient and secure encryption workflows.

The GenerateDataKey operation provides both a plaintext and an encrypted version of a data key. This process enables you to encrypt data using the plaintext data key and subsequently store only the encrypted version of the data key alongside your encrypted data, thus maintaining a secure environment.

When you need an encrypted data key without its plaintext form—for heightened security—the GenerateDataKeyWithoutPlaintext operation is used. This approach avoids exposing the plaintext key, enhancing data protection:

Decrypting an encrypted data key is straightforward with the Decrypt operation. This is vital for accessing your encrypted data securely and efficiently:

This suite of operations equips you with the capability to manage encryption keys effectively, ensuring data is encrypted and decrypted as needed while maintaining security best practices. Through these methodologies, AWS KMS and Boto3 empower developers and security professionals to protect sensitive information comprehensively.

In this lesson, we've focused on AWS KMS and its management of data keys. Actual encryption and decryption with these keys require the cryptography library in Python, extending beyond our scope. For further exploration, consult the cryptography library's documentation: Cryptography Library, where you can find detailed examples and guidance on implementing encryption and decryption.

Key rotation in AWS Key Management Service (AWS KMS) ensures heightened security for your cryptographic operations. For customer-managed AWS KMS keys, AWS KMS allows you to enable key rotation, automatically generating new cryptographic material for the AWS KMS key every year, while keeping the key ID the same. This feature is particularly valuable for maintaining security without disrupting key-dependent resources or services. In contrast, AWS-managed KMS keys, which are maintained by AWS for encrypted AWS service data, are rotated by AWS every three years with no required action from the customer. Note that the automatic rotation feature is distinctly applied to AWS KMS keys, and does not extend to data keys generated by these AWS KMS keys. Managing the lifecycle and rotation of data keys, used directly for encrypting and decrypting data, is the responsibility of the implementing application, following its own security practices and requirements.

To enable key rotation for an AWS KMS key, use the following Boto3 code snippet:

After enabling key rotation, you may want to verify the status to ensure it's active. Use the get_key_rotation_status operation for this purpose:

There might be scenarios where you no longer need an AWS KMS key, and retaining it could pose unnecessary risks. AWS KMS allows you to schedule the deletion of a key, providing a buffer period during which you can cancel the deletion if needed. This feature ensures that keys are not deleted accidentally and that you have a grace period to reverse the operation if the deletion was not intended.

To schedule the deletion of an AWS KMS key and specify the waiting period, use the following code:

Congratulations! You have learned how to manage encryption keys using AWS KMS via the AWS SDK for Python (Boto3). You should understand what AWS KMS is, what an AWS KMS key is, and how to encrypt and decrypt data using these keys. You should also know how to manage data keys, enable key rotation, and schedule key deletion.

In the next practice exercises, you will apply these concepts, using Boto3 to manage keys in AWS KMS. If this is the final lesson of this course, you’ve come a long way - well done! Remember, practice is key to mastery, so keep exploring and experimenting!