OpenPGP Web Key Directory
Table of Contents
1. Introduction
If you use OpenPGP/GnuPG to secure your email communication, you know that sharing your public key with your contacts is an important step of the process. Traditionally it was done either by sending it as an attachment on the first email, or by publishing it on the so-called "keyserver network", where your contacts can find and fetch it. Both of these methods have their own problems and are not recommended.
The recommended way for publishing your public key is through Web Key Directory (WKD), where mail clients will be able to locate and fetch it automatically. This means that once you publish your key on the WKD, your contacts will be able to:
- Verify your signed emails automatically, because their mail clients can find and fetch your public key automatically.
- Send you immediately encrypted messages, without asking for your public key first, because their mail clients can find and fetch it automatically.
The ability to fetch automatically public keys from a WKD is already supported by almost all the mail clients (Thunderbird, KMail, Outlook, etc.) All you need to do is to publish your public key to a WKD server. Let's see how to build such a server and how to publish your key.
2. How WKD works
Before building a WKD server, let's try to understand first how it
works by following the steps that a mail client would do to locate the
public key corresponding to a certain email address. Let's say that
the email address is <user@example.org>
.
A mail client will check first whether a WKD for the domain example.org
exists.
It does it by checking first for the presence of:
https://openpgpkey.example.org/ .well-known/openpgpkey/example.org/policy
If it fails, then it checks for the presence of:
https://example.org/.well-known/openpgpkey/policy
The first one is called the advanced method, and the fallback one is
called the direct method. The file policy
is usually just an empty
file.
Then, depending on which WKD method exists for example.org
, it tries
to download the public key from one of these locations (lines are
broken for rendering purposes):
https://openpgpkey.example.org/.well-known/openpgpkey/example.org/ hu/nmxk159crbcuk3imqiw13gkjmfwd8mqj?l=user https://example.org/.well-known/openpgpkey/ hu/nmxk159crbcuk3imqiw13gkjmfwd8mqj?l=user
The directory hu
stands for hashed-userid
, and
nmxk159crbcuk3imqiw13gkjmfwd8mqj
is indeed a kind of hash of user
,
the local-part of the email address. For the time being we don't need
to know how the WKD client finds or generates this hash.
3. Building a WKD
First, let's build a WKD with the direct method, since it is a bit simpler than the advanced method.
This requires that we have access to the webserver of the domain
example.org
, and that the server supports HTTPS.
Then we will see also the advanced method, which additionally
requires access to the DNS server of the domain example.org
.
3.1. Create the directory of public keys
If the DocumentRoot is /var/www/html/
we can build the web key
directory like this:
mkdir -p /var/www/html/.well-known/openpgpkey/ touch /var/www/html/.well-known/openpgpkey/policy
Now we just need to export the public key, transfer it to the web server, and save it at the file:
.well-known/openpgpkey/hu/nmxk159crbcuk3imqiw13gkjmfwd8mqj
The name of the file nmxk159crbcuk3imqiw13gkjmfwd8mqj
is the
hashed-userid that we saw on the previous section. We can get it
from a gpg
command like this:
$ gpg --with-wkd-hash --fingerprint user@example.org pub rsa3072 2021-04-22 [SC] [expires: 2023-04-22] 901D C530 A8E1 DEBA FED0 6C25 C802 3646 1A8D CFC2 uid [ultimate] user@example.org nmxk159crbcuk3imqiw13gkjmfwd8mqj@example.org sub rsa3072 2021-04-22 [E]
To export the public key we can use a command like this:
gpg --no-armor --export \ user@example.org > nmxk159crbcuk3imqiw13gkjmfwd8mqj
The option --no-armor
makes sure to export it in binary format (not
ASCII armored), as required by the WKD specs.
The same way we can publish the public keys for more email addresses,
like user1@example.org
, user2@example.org
, etc. At the end, the
directory should look like this:
/var/www/html/.well-known/ └── openpgpkey ├── hu │ ├── nmxk159crbcuk3imqiw13gkjmfwd8mqj │ ├── sxpkq64cy1wikgh8o8eddrx6bg8urzu8 │ └── wgrbabzq3fs5uryhxq96e8nnwxae78fw └── policy
3.2. Webserver configuration
The web server of the WKD:
- Should disable directory listing for the
hu/
directory. - Should have the right CORS headers.
- Should use
application/octet-stream
as the Content-Type for the data of public keys.
With apache2 the configuration should look like this:
<Directory "/.well-known/openpgpkey/hu"> Options -Indexes ForceType application/octet-stream Header always set Access-Control-Allow-Origin "*" </Directory>
This requires that the apache2 modules mime
and headers
are
enabled:
a2enmod mime headers
With nginx the configuration should look like this:
location /.well-known/openpgpkey/hu/ { autoindex off; default_type "application/octet-stream"; add_header Access-Control-Allow-Origin * always; }
3.3. Test the WKD
We can use wget
or curl
to download a published key from the WKD,
just for testing. Let's also use gpg-wks-client
in order to find out
the hash of the userid. It can be installed like this:
apt install gpg-wks-client alias gpg-wks-client='/usr/lib/gnupg/gpg-wks-client -v' gpg-wks-client --help
We can find the hash of user@example.org
like this:
$ gpg-wks-client --print-wkd-hash user@example.org nmxk159crbcuk3imqiw13gkjmfwd8mqj user@example.org
With this hash we can construct the URL manually and download the public key:
wget -q -O user-key.pub \ https://example.org/.well-known/openpgpkey/hu/ nmxk159crbcuk3imqiw13gkjmfwd8mqj?l=user
Other ways for testing that the published key is accessible via WKD are these:
Using
gpg --locate-keys
:env GNUPGHOME=$(mktemp -d) gpg -v --locate-keys user@example.org
- Using a WKD validator like this one: https://metacode.biz/openpgp/web-key-directory
3.4. Publish the keys of an organization
We have seen already how to publish keys manually one by one, but as the number of keys to be published gets large, it becomes tedious and error-prone to manage them. However it is possible to publish them in bulk.
Let's say that we have in our GnuPG keyring the public keys of the members of an organization (for example they were sent to us by attachment, and we imported them). We can export all these keys into a WKD format like this:
mkdir wkd gpg --list-options show-only-fpr-mbox \ --list-keys "@example.org" \ | gpg-wks-client --install-key --directory wkd/
The GnuPG keyring is searched for all public keys (--list-keys
)
matching the defined pattern (@example.org
), and the output is
generated as fingerprint user_id
values that look like this:
901DC530A8E1DEBAFED06C25C80236461A8DCFC2 user@example.org D67E52B28F276D2AE5250B4EFF09740FC5FD1300 user1@example.org 38186EF3FDA463907B494A4AEDF1E29CB878858F user2@example.org
With this as input, gpg-wks-client
creates a WKD directory structure
that looks like this:
$ tree wkd/ wkd/ └── example.org ├── hu │ ├── nmxk159crbcuk3imqiw13gkjmfwd8mqj │ ├── sxpkq64cy1wikgh8o8eddrx6bg8urzu8 │ └── wgrbabzq3fs5uryhxq96e8nnwxae78fw └── policy
Now we can just use rsync
to synchronize the directory hu/
with
the one on the webserver, for example:
rsync -a --delete \ ./wkd/example.org/hu/ \ webserver:/var/www/html/.well-known/openpgpkey/hu/
3.5. The advanced method of WKD
With the advanced method, the key is published in a location like:
https://openpgpkey.example.org/.well-known/openpgpkey/example.org/hu/
,
instead of https://example.org/.well-known/openpgpkey/hu/
, which is
the location for the direct method. Everything else is the same.
Notice that this requires the subdomain openpgpkey.example.org
to be
resolvable. On the other hand, if we want to use the direct method,
we should make sure that this subdomain is not resolvable, since WKD
clients will first try the advanced method, and only if the
openpgpkey
subdomain is not resolvable will fall back to the
direct method.
While the direct method is simpler because it does not need any DNS modifications, the advanced method is more flexible because:
- It allows us to use a WKD server that is different from the webserver.
- It allows us to support more than one email domain in the same WKD server.
4. WKD server with docker-scripts
We can install a WKD server with Docker and docker-scripts. This container actually supports a WKS (Web Key Service) server as well, but we will see this later.
This container supports the advanced method of WKD, so we need to
define a DNS record for the subdomain openpgpkey.example.org
, which
resolves to the IP of the host where this container is installed.
4.1. Install dependencies
This container depends on docker, docker-scripts, and wsproxy, which is a container that acts as a reverse proxy for the HTTPS requests.
Install docker:
curl -fsSL https://get.docker.com -o get-docker.sh sudo sh get-docker.sh
Install docker-scripts:
apt install m4 make git clone https://gitlab.com/docker-scripts/ds \ /opt/docker-scripts/ds cd /opt/docker-scripts/ds/ make install
Install WebServer Proxy:
ds pull wsproxy ds init wsproxy @wsproxy cd /var/ds/wsproxy/ vim settings.sh ds make
4.2. Install the WebKey container
Add records like these on the DNS configuration of the domain
example.org
:webkey.example.org. IN A 10.11.12.13 openpgpkey.example.org. IN CNAME webkey.example.org.
The subdomain
webkey
can be named to anything, but the subdomainopenpgpkey
is not up to us because it is part of the WKD specification.Get the scripts:
ds pull webkey
Create a directory for the container:
ds init webkey @webkey.example.org
Fix the settings:
cd /var/ds/webkey.example.org/ vim settings.sh
We will see later that this container can be used both as a WKD and a WKS (Web Key Service) server. For the time being we are using it only as a WKD server, so we can comment out the settings
PORTS
andALLOWED_NETWORKS
(although it doesn't harm if we don't comment them).As
HOSTNAME
setwebkey.example.org
and asWEBKEY_DOMAINS
setexample.org
(without the prefixopenpgpkey
, it will be added automatically by the configuration scripts).Make the container:
ds make
4.3. Checking and testing
After installation, let's check some configurations:
- On
/var/ds/webkey.example.org/
there should be the subdirectorywkd/
. (There is also the subdirectorywks/
but we can safely ignore it for now). On wsproxy, the configuration file
/var/ds/wsproxy/sites-enabled/webkey.example.org.conf
should containServerName
andServerAlias
like this:ServerName webkey.example.org ServerAlias openpgpkey.example.org
The configuration of apache2 inside the container should look like this:
$ ds exec cat /etc/apache2/conf-enabled/wkd.conf Alias /.well-known/openpgpkey /host/wkd <Directory /host/wkd> Require all granted Options -Indexes </Directory> <Directory ~ "/host/wkd/.*/hu"> ForceType application/octet-stream Header always set Access-Control-Allow-Origin "*" </Directory>
If we add keys to the directory wkd/example.org/hu/
they will be
available immediately for being accessed through WKD. Initially only
the public key for the email address keys@example.org
is published
(which is used for the WKS, as we will see later). We can use this
published key to test whether WKD works correctly:
Install
gpg-wks-client
:apt install gpg-wks-client alias gpg-wks-client='/usr/lib/gnupg/gpg-wks-client -v' gpg-wks-client --help
Check that the public key of
keys@example.org
is published on WKD:$ gpg-wks-client -v --check keys@example.org gpg-wks-client: public key for 'keys@example.org' found via WKD gpg-wks-client: gpg: Total number processed: 1 gpg-wks-client: fingerprint: B3883F4D3D926E19DAD558D968BA810F826B7DF7 gpg-wks-client: user-id: keys@example.org gpg-wks-client: created: Tue 27 Apr 2021 07:03:45 PM CEST gpg-wks-client: addr-spec: keys@example.org
Alternatively, check it using a WKD validator like this one: https://metacode.biz/openpgp/web-key-directory
Get the key and import it on the GnuPG keyring:
$ gpg-wks-client --print-wkd-url keys@example.org https://openpgpkey.example.org/.well-known/openpgpkey/ example.org/hu/mfnkhjkbtuim3hwf51o7qepzxnrpi1r5?l=keys $ wget -qO- $(gpg-wks-client --print-wkd-url keys@example.org) \ | gpg --import gpg: key 68BA810F826B7DF7: public key "keys@example" imported gpg: Total number processed: 1 gpg: imported: 1
Alternatively, we can get it with
gpg --locate-keys
:$ gpg --delete-keys keys@example.org $ gpg --list-keys keys@example.org $ gpg -v --locate-keys keys@example.org gpg: using pgp trust model gpg: error retrieving 'keys@example.org' via Local: No public key gpg: pub rsa3072/68BA810F826B7DF7 2021-04-27 keys@example.org gpg: key 68BA810F826B7DF7: public key "keys@example.org" imported gpg: Total number processed: 1 gpg: imported: 1 gpg: auto-key-locate found fingerprint B3883F4D3D926E19DAD558D968BA810F826B7DF7 gpg: automatically retrieved 'keys@example.org' via WKD pub rsa3072 2021-04-27 [SC] [expires: 2023-04-27] B3883F4D3D926E19DAD558D968BA810F826B7DF7 uid [ unknown] keys@example.org sub rsa3072 2021-04-27 [E]
Notice the message:
automatically retrieved 'keys@example.org' via WKD
.
5. Web Key Service (WKS)
What is a WKS? So far we have seen how to publish keys manually (one by one or in bulk). For a large organization it is more suitable if each member can publish and update his key himself. One way to implement this is with a web interface, where the members can upload their public key, after authenticating themselves somehow. Another way is to send the public key to the WKD by email.
WKS allows users to publish their public key by email, through a specific email protocol.
5.1. How WKS works
To understand how WKS works, let's see how a key is published through it.
First of all, the user (or his mail client) needs to know the email
address where he can send the key. By the WKS specification, this
email address can be found on the file submission-address
on this
URL (the line is broken for rendering purposes):
https://openpgpkey.example.org/.well-known/openpgpkey/example.org/ submission-address
This file contains a single line with an email address like this:
keys@example.org
Then, to publish the key, these steps are followed:
- The user (or his mail client) sends by attachment to the submission address the public key corresponding to his email address. This is the key publication request.
- The WKS replies with an encrypted message that contains a nonce and the fingerprint of the key. The message is encrypted with the public key that is received by attachment (that is to be published on the WKD). This is the request verification step.
- The user decrypts the confirmation message with his private key and sends back the nonce. This reply message should be encrypted with the public key of the submission address, which can be retrieved from the WKD. This is the request confirmation step.
- Upon receiving the nonce and checking that it is correct, the WKS publishes the key of the user to the WKD and sends a notification message to the user, informing him that the key has been published. This is the key publication step.
The steps (2) and (3) above (request verification and request confirmation) ensure that:
- The key submission request is not fake (it is coming from the user that owns that email address).
- The public key that is submitted for publication is genuine (since the user is able to decrypt a message encrypted with it).
A WKS is usually integrated with the mail server of the domain
example.org
. But it can also be installed on a separate server and
work as an extension to the mail server.
5.2. Install the WKS container
The docker container that we installed on the section (4.) can serve
both for WKD and WKS. Installation is the same, just make sure to
uncomment PORTS
and ALLOWED_NETWORKS
on settings.sh
, and then
rebuild with ds make
.
Make sure to add to the ALLOWED_NETWORKS
the IP addresses (or
networks) of the mail servers that are going to be served by this WKS
container. For security reasons, it accepts SMTP connections only from
the allowed networks. You will understand why this is needed on the
next section, which explains how it works.
After installation, we can make a quick sanity check of the installed container like this:
cd /var/ds/webkey.example.org/ ds play tests/test1.sh
5.3. How the WKS container works
This WKS container works as an external extension to one or more existing mail servers.
- When the mail server receives an email for
keys@example.org
(which is the submission-address for the mail domain@example.org
) it forwards it tokeys@webkey.example.org
and sends it by SMTP on the port10025
of the WKS server (with hostnamewebkey.example.org
). For security reasons, the WKS server will accept this SMTP connection only if the IP of the mail server is listed on itsALLOWED_NETWORKS
variable. The WKS server has an alias that forwards this email to the local account
webkey
(which belongs to the local userwebkey
):$ ds exec postconf mydestination mydestination = webkey.example.org, localhost
$ ds exec cat /etc/aliases keys webkey
The WKS server also has enabled maildrop processing:
$ ds exec postconf mailbox_command mailbox_command = /usr/bin/maildrop -d ${USER}
The
webkey
account has a mailfilter like this:$ cat wks/.mailfilter logfile maildrop.log #cc archive/ # debug to "| gpg-wks-server --directory /host/wkd \ --receive --send &>>maildrop.log"
It pipes the arriving emails to the command
gpg-wks-server
, which processes them and sends any replies through the mailserver.- For the messages and replies from
gpg-wks-server
to get through, the mail server has to add the IP of the WKS server on its trusted hosts. It also has to rewrite the sender address fromwebkey@webkey.example.org
tokeys@example.org
, in order to avoid any SPF failures. For more details about this check the next section.
5.4. Configuration on the mailserver
On the mail server we need to make these configurations in order to integrate it with the WKD+WKS server:
Add a virtual alias from
keys@example.org
tokeys@webkey.example.org
. It might be done like this:postconf 'virtual_alias_maps = hash:/host/config/virtual_alias_maps' cat << EOF >> /host/config/virtual_alias_maps keys@example.org keys@webkey.example.org EOF postmap /host/config/virtual_alias_maps postfix reload
This ensures that key-submission emails and submission-confirmation emails are forwarded to the WKS server.
Tell the mailserver to send (transport) the mails for the domain
webkey.example.org
tosmtp:[webkey.example.org]:10025
. It may be done like this:postconf 'transport_maps = hash:/host/config/transport_maps' cat << EOF > /host/config/transport_maps webkey.example.org smtp:[webkey.example.org]:10025 EOF postmap /host/config/transport_maps postfix reload
Allow the WKS server to send replies (emails) through the mailserver, without authentication, by adding its IP to the list of
mynetworks
. It might be done like this:postconf 'mynetworks = /host/config/trusted_hosts' cat << EOF >> /host/config/trusted_hosts 12.34.56.78 EOF postfix reload
The envelope of the emails (replies) sent from the WKS server will have a sender address like
webkey@webkey.example.org
. This may confuse the SPF check of mail providers (in case the emails of the clients are forwarded to external addresses). As a result these emails may be considered less trusty and may end up as spam.To prevent this we should configure the mailserver to rewrite this address to
keys@example.org
. It might be done like this:postconf 'smtp_generic_maps = hash:/host/config/smtp_generic_maps' cat << EOF > /host/config/smtp_generic_maps webkey@webkey.example.org keys@example.org EOF postmap /host/config/smtp_generic_maps postfix reload
5.5. Publishing a key through WKS
Let's assume that we have an authenticated mail account on the
mailserver, with username test1@example.org
and password pass1
. We
want to publish the GnuPG public key for the email address
test1@example.org
.
We will use the command gpg-wks-client
to generate the necessary
emails and replies that should be sent to the WKS. But we also need a
tool for sending authenticated emails from the command line, and
msmtp
is a good one. Let's install these dependencies first:
apt install gpg-wks-client msmtp alias gpg-wks-client='/usr/lib/gnupg/gpg-wks-client'
Before sending the key, let's make sure that WKS is supported by our mailserver:
$ gpg-wks-client -v --supported test1@example.org gpg-wks-client: provider for 'test1@example.org' supports WKS
Now we can follow these steps to publish the key:
Send a key publishing request like this:
gpg --list-keys test1@example.org gpg-wks-client --create \ AB97233AD0EB0180882D1227799020EF6FF16876 \ test1@example.org \ | msmtp \ --read-envelope-from --read-recipients \ --tls=on --auth=on \ --host=smtp.example.org --port=587 \ --user=test1@example.org --passwordeval="echo pass1"
The command
gpg-wks-client --create
creates a request email in the format that is required by the WKS, andmsmtp
sends it through the mailserver, with authentication.- Soon WKS will send as followup an email with subject Confirm your key publication.
Save it as a text file:
Confirm-your-key-publication.eml
Send the confirmation email with the command
gpg-wks-client --receive
, like this:cat Confirm-your-key-publication.eml \ | gpg-wks-client --receive \ | msmtp \ --read-envelope-from --read-recipients \ --tls=on --auth=on \ --host=smtp.example.org --port=587 \ --user=test1@example.org --passwordeval="echo pass1"
We should receive a notification email that the key has been published. For a quick check we can use the command:
gpg-wks-client -v --check test1@example.org
Note: Key publishing process was automated by Thunderbird+Enigmail. However Thunderbird 78 dropped Enigmail. Its functionality was supposed to be merged to Thunderbird, but not all of it is already merged. In particular the features that facilitate and automate the interaction with a WKS server are missing. Let’s hope that they will be added back soon.
6. References
- https://wiki.gnupg.org/WKD
- https://datatracker.ietf.org/doc/draft-koch-openpgp-webkey-service/
- gpg-wks-client
- gpg-wks-server
- https://www.uriports.com/blog/setting-up-openpgp-web-key-directory/
- https://spacekookie.de/blog/usable-gpg-with-wkd/
- https://shibumi.dev/posts/how-to-setup-your-own-wkd-server/