CMS Encryption in Go: Secure Data with OpenSSL and CGO

What We’ll Build #

After deep diving into OpenSSL’s CLI capabilities to encrypt and decrypt with CMS, we’ll build a small Go CLI tool that performs CMS encryption using OpenSSL’s libcrypto API through CGO. It uses authenticated encryption and public key–based key management.

The final usage of the tool will look like this:

cms-encrypt -cert "$CMS_CERT" -out ./message.p7m -message "Hello World"

Using the corresponding private key, the encrypted message can be decrypted with OpenSSL:

openssl cms -decrypt -in ./message.p7m -inkey "$CMS_KEY" -inform PEM
Hello  World

For the full working example, see the example section.

Introduction #

The Go ecosystem lacks CMS encryption support. Better support is available in OpenSSL. To use OpenSSL, we leverage Go’s built-in CGO interface to call into OpenSSL’s libcrypto C-library.

By the end of this post, you’ll have a functional CMS encryption CLI, and you’ll understand how to integrate OpenSSL into your Go applications using CGO.

Prerequisites

This example relies on CGO and the OpenSSL C libraries. CGO supports pkg-config to correctly discover required paths for the compiler and linker. Ensure that the following is installed on your system:

• pkg-config
• openssl >= 3.0 and its header files

Ensure OpenSSL can be discovered using pkg-config:

pkg-config --modversion openssl

A note for macOS users

macOS’ native OpenSSL distribution cannot be symlinked due to restrictions on macOS. Install OpenSSL separately, for example, using brew. To allow pkg-config to find the installation, ensure to set the following environment variable (change the path if you are not using brew):

export PKG_CONFIG_PATH="$(brew --prefix openssl)/lib/pkgconfig"

To verify your installation, run the following and compare the output:

pkg-config --cflags --libs openssl
-I/opt/homebrew/Cellar/openssl@3/3.4.1/include -L/opt/homebrew/Cellar/openssl@3/3.4.1/lib -lssl -lcrypto

Usage in Go

With this, we can setup CGO and wire up our dependencies to libcrypto APIs in encrypt.go as follows:

package main

/*
#cgo pkg-config: openssl
#include <openssl/cms.h>
*/
import "C"

The comment, where we configure pkg-config and include C-Headers - the so-called preamble - must immediately precede the import of the CGO runtime using import "C".

Using OpenSSL libcrypto with CGO #

Before we build the command line tool, let’s first dissect libcrypto’s APIs and how to use them in Go. The API surface is large and powerful. Keep libcrypto’s API docs handy as a reference.

CMS Encryption APIs #

The API around CMS uses much of the terminology introduced in RFC 2315. Read my primer on CMS for a jump start if you have a hard time understanding what is going on. Since we want to perform encryption with CMS, the function CMS_encrypt is a good starting point:

CMS_ContentInfo *CMS_encrypt(STACK_OF(X509) *certs, BIO *in, const EVP_CIPHER *cipher, unsigned int flags);

The documentation describes several modes of behavior and calling conventions depending on which flags are set. The flag providing most flexibility is CMS_PARTIAL. With this one, we can

• set certs to NULL and add recipients later on using CMS_add1_recipient_cert
• and set in to NULL requiring us to call CMS_final with the content after all recipients have been configured.

In addition, we are not interested in any SMIME encodings. We disable those by using CMS_BINARY. When encrypting large inputs, the flag CMS_STREAM is providing great value but is not relevant for this example.

The parameter cipher controls how the content is encrypted. CMS supports AEADs, and we will use one for this example. OpenSSL supports various, but we stick with the straightforward cipher AES-128 in GCM mode which is supported via the function EVP_aes_128_gcm.

CMS_encrypt allocates memory and returns a pointer. To prevent leaking memory, we need to deallocate that memory with the corresponding function CMS_ContentInfo_free. A typical pattern with CGO is to call the memory deallocating functions with defer to ensure memory is freed at some point.

With that knowledge we are ready to initialize our CMS structure:

cms := C.CMS_encrypt(nil, nil, C.EVP_aes_128_gcm(), C.CMS_PARTIAL|C.CMS_BINARY)
if cms == nil {
  return nil, fmt.Errorf("CMS_encrypt failed: %w", getOpenSSLError())
}
defer C.CMS_ContentInfo_free(cms)

Before we proceed with the encryption, we need some more helper methods. One of them is getOpenSSLError and is explained in the following section.

Error handling #

Many functions in libcrypto handle errors by returning NULL and provide detailed error information via ERR_get_error and ERR_error_string. To expose these errors via Go’s error interface, we use this helper:

func getOpenSSLError() error {
  errCode := C.ERR_get_error()
  if errCode == 0 {
  	return nil
  }
  errStr := C.ERR_error_string(errCode, nil)
  return errors.New(C.GoString(errStr))
}

This retrieves the current error and maps it into the corresponding error string, which is then finally wrapped into a standard Go error.

Certificates and BIOs #

CMS_add1_recipient_cert requires us to use the X509 C-struct. Usually, X509 certificates are either passed around using PEM encoding or DER encoding (check out my post on PEM and DER encoding of X509 certificates if that is new for you). libcrypto supports parsing both encodings.

For this example, we assume that we get certificates as Go type x509.Certificate and that some calling code is performing the actual parsing from a file. We use libcrypto’s d2i_X509 to decode the bytes into the X509 data type. With this, we can come up with the following helper method:

func loadCert(cert *x509.Certificate) (*C.X509, error) {
	if cert == nil || len(cert.Raw) == 0 {
		return nil, errors.New("invalid certificate input")
	}

	certLength := C.long(len(cert.Raw))
	ptr := (*C.uchar)(C.CBytes(cert.Raw))
	defer C.free(unsafe.Pointer(ptr))

	cCert := C.d2i_X509(nil, &ptr, certLength)
	if cCert == nil {
		return nil, fmt.Errorf("error parsing DER certificate: %w", getOpenSSLError())
	}

	return cCert, nil
}

Since memory is allocated, remember to free it with X509_free.

One more note on passing data: libcrypto has many functions with BIO in its name. All BIO-related functions deal either with inputting or outputting data via a diverse set of mechanisms. One of those mechanisms is to simply load all data from memory into a byte array using BIO_get_mem_data:

long BIO_get_mem_data(BIO *b, char **pp);

A seemingly easy “function” but with one specific detail noted somewhere in the docs:

It is implemented as a macro.

That little detail kept me busy for quite some time since using it via CGO always resulted in

could not determine what C.BIO_get_mem_data refers to

After some research on CGO and its inner workings, it became clear:

To use macros anyhow, you need to have a small wrapper function, for example, in the preamble:

/*
long MACRO_BIO_get_mem_data(BIO *b, char **pp) {
	return BIO_get_mem_data(b, pp);
}
*/
import "C"

With this in place, writing a convenient wrapper method to load all data in a BIO into memory is straightforward:

func bytesFromBio(bio *C.BIO) ([]byte, error) {
  var ptr *C.char
  size := C.MACRO_BIO_get_mem_data(bio, &ptr)
  if size <= 0 {
  	return nil, errors.New("no output data")
  }
  return C.GoBytes(unsafe.Pointer(ptr), C.int(size)), nil
}

Build a CMS Encryption CLI Tool in Go #

After we gathered all the individual pieces, we are ready to put everything together. The goal is a single high-level Go function wrapping low-level CMS encryption and to use it from the CLI interface code.

High-level Go encryption function #

Let’s begin with the high-level encryption interface. We work through it step by step. To view the full implementation all at once, see encrypt.go.

func EncryptCMS(data []byte, recipient *x509.Certificate) ([]byte, error)

Step 1: Initialize CMS structure #

To begin, we initialize the CMS structure by setting the previously discussed flags and ciphers:

cms := C.CMS_encrypt(nil, nil, C.EVP_aes_128_gcm(), C.CMS_PARTIAL|C.CMS_BINARY)
if cms == nil {
  return nil, fmt.Errorf("CMS_encrypt failed: %w", getOpenSSLError())
}
defer C.CMS_ContentInfo_free(cms)
```

Step 2: Load and add recipient certificate #

Next, we load the certificate with our helper method and add it as a recipient to our CMS structure:

cert, err := loadCert(recipient)
if err != nil {
  return nil, fmt.Errorf("load cert: %w", err)
}
defer C.X509_free(cert)

if C.CMS_add1_recipient_cert(cms, cert, 0) == nil {
  return nil, fmt.Errorf("CMS_add1_recipient_cert failed: %w", getOpenSSLError())
}

Step 3: Finalize CMS with message input #

Since we are running in partial mode, we need to finalize the CMS structure with the plaintext intended for encryption. CMS_final signature wants a BIO to read the data from. Since we are purely handling all data in-memory at this point, we use a BIO based on a memory buffer:

bioIn := C.BIO_new_mem_buf(unsafe.Pointer(&data[0]), C.int(len(data)))
if bioIn == nil {
  return nil, errors.New("failed to create input BIO")
}
defer C.BIO_free(bioIn)

if C.CMS_final(cms, bioIn, nil, 0) != 1 {
  return nil, fmt.Errorf("CMS_final failed: %w", getOpenSSLError())
}

Step 4: PEM encode output #

With the finalized CMS structure, we now need to deal with encoding the structure and where to store it. CMS knows of a PEM encoding and the corresponding method PEM_write_bio_CMS_stream writes a CMS structure PEM encoded into a BIO. We again use a memory-backend BIO since we handle all data in-memory. After PEM_write_bio_CMS_stream returns, the PEM encoded CMS structure is in the BIO, and we extract it again into a plain Go byte-array by using our utility bytesFromBio.

bioOut := C.BIO_new(C.BIO_s_mem())
if bioOut == nil {
  return nil, errors.New("failed to create output BIO")
}
defer C.BIO_free(bioOut)

if C.PEM_write_bio_CMS_stream(bioOut, cms, nil, 0) != 1 {
  return nil, fmt.Errorf("i2d_CMS_bio_stream failed: %w", getOpenSSLError())
}

return bytesFromBio(bioOut)
}

CLI interface #

What remains is our CLI interface that calls our previous function. The parameters for our CLI are organized in the following structure:

type opts struct {
	CertificatePath string
	Message         string
	OutFile         string
}

This struct is input to the Run function of our CLI which simply reads in the certificate from a file and parses the PEM content, performs encryption of the passed content using our EncryptCMS function and finally outputs the CMS structure either on stdout or writes it into the given file.

func Run(options opts) error {
	// Read the certificate from the given path
	certPEM, err := os.ReadFile(options.CertificatePath)
	if err != nil {
		return fmt.Errorf("failed to read certificate: %w", err)
	}

	// Decode the PEM block
	block, _ := pem.Decode(certPEM)
	if block == nil || block.Type != "CERTIFICATE" {
		return errors.New("error to decode PEM block containing certificate")
	}

	// Parse the certificate
	cert, err := x509.ParseCertificate(block.Bytes)
	if err != nil {
		return fmt.Errorf("error parsing certificate: %w", err)
	}
	
	// Perform encryption for the recipient
	encrypted, err := cms.EncryptCMS([]byte(options.Message), cert)
	if err != nil {
		return err
	}

	// Output the CMS structure as file or on stdout
	if options.OutFile != "" {
		fd, err := os.OpenFile(options.OutFile, os.O_WRONLY|os.O_CREATE, 0644)
		if err != nil {
			return err
		}
		defer fd.Close()

		_, err = fd.Write(encrypted)
		fmt.Println("Wrote CMS content to file:", options.OutFile)
		return err
	} else {
		fmt.Println("Encrypted using CMS:")
		fmt.Println(string(encrypted))
	}

	return err
}

The corresponding main method uses Go’s built-in flag package to parse the CLI parameters, fill the option struct, and pass control to the Run method:

func main() {
	var options opts
	flag.StringVar(&options.CertificatePath, "cert", "", "Certificate path")
	flag.StringVar(&options.Message, "message", "", "Message to encrypt")
	flag.StringVar(&options.OutFile, "out", "", "Output file")
	flag.Parse()

	if options.CertificatePath == "" {
		fmt.Println("missing required argument: -cert")
		return
	}
	if options.Message == "" {
		fmt.Println("missing required argument: -message")
		return
	}

	err := Run(options)
	if err != nil {
		fmt.Println("Error running the command:", err)
	}

	return
}

The full implementation is in main.go.

Example #

We are ready and we can compile our CLI tool. Clone the repository to your local machine and ensure that you have a Go SDK ready on your local machine:

git clone https://github.com/adippel/cms-encryption-go.git && cd cms-encryption-go

Compile the binary:

go build -o cms-encrypt ./main

Before using it, we need a certificate and private key to encrypt and decrypt messages:

CMS_CERT=recipient.crt
CMS_KEY=recipient.key
openssl req -new -newkey ec -pkeyopt group:prime256v1 -noenc -keyout $CMS_KEY \
	-out $CMS_CERT -outform PEM -x509 -days 365 \
	-subj "/O=alexdippel.de/OU=CMS Test Laboratory/CN=ecc-recipient" \
	-addext keyUsage=critical,keyAgreement

With that, we can perform the encryption:

./cms-encrypt -cert "$CMS_CERT" -out ./message.p7m -message "Hello World"

To inspect the resulting structure, use the print mechanisms of the OpenSSL CLI:

openssl cms -print -cmsout -in ./message.p7m -inform PEM

Using the corresponding private key, the encrypted message can be decrypted again:

openssl cms -decrypt -in ./message.p7m -inkey "$CMS_KEY" -inform PEM

Conclusion #

Go’s ecosystem lacks first-class CMS encryption support, but you can fill that gap using OpenSSL’s mature CMS capabilities and CGO. In this post, we built a CLI tool that performs CMS encryption with recipient certificates, using AES-GCM for authenticated encryption.

We explored:

• How to use CMS_encrypt, CMS_add1_recipient_cert, and CMS_final
• Passing data between Go and C
• Handling OpenSSL’s errors in idiomatic Go

From here on, potential extensions of the example might be:

• CMS supports encrypting for multiple recipients: Allowing adding several recipient certificates.
• Large file support:
  • Introduce streaming support using Go’s io.Reader together with OpenSSL BIOs.
  • Avoid base64 encoding overhead by allowing binary DER/BER encoding.
• Allowing changing the cipher used for encryption.

You can find the complete source code and instructions in the GitHub repository. If you have questions or run into trouble, feel free to drop me a message.

Related Posts #

This is part of a series of CMS articles that I have published. Please see also:

• Using OpenSSL to Encrypt and Decrypt with CMS
• End-to-End security for sensitive data using CMS (aka PKCS7)