rework former 18.2 and 18.3 into new 18.3 on merging and updating certificates

book/source/adv/certificates.md

@@ -19,126 +19,113 @@ OpenPGP certificates are crucial for a wide range of applications, from secure e
 
 In this chapter, we aim to delve deeper into the advanced concepts surrounding OpenPGP certificates, focusing on their validity, expiration, and the critical role they play in ensuring the security and reliability of cryptographic communications. By exploring these concepts, we aim to provide readers with a comprehensive understanding of how OpenPGP certificates function within the ecosystem, their practical applications, and best practices for managing certificate validity and expiration to maintain a secure cryptographic environment.
 
-## Certificate validity and expiration
+## Certificate validity, expiration, and revocation
 
-Maintaining the validity of OpenPGP certificates is essential for the security and reliability of cryptographic communications. Expiration dates play a key role in this process, acting as a built-in check that prompts users to review and renew their certificates periodically.
+OpenPGP certificates are integral to establishing and maintaining secure communication channels. These certificates, being composites of various components linked by [signatures](../signing_components), embody the trust and authentication mechanisms underpinning the OpenPGP standard. This section explores the dual aspects of certificate validity: expiration and revocation, and how they govern the lifecycle of a certificate and its individual components.
 
-Certificates are composites of components that are linked together using [signatures](../signing_components).
+### Understanding certificate expiration
 
-A certificate can be valid or invalid as a whole. However, even when a certificate is valid, individual components (subkeys or identities) of it can be invalid.
+Certificates and their components within the OpenPGP framework are subject to expiration, a mechanism designed to ensure timely review and renewal of cryptographic credentials. Expiration delineates a clear validity period, beyond which a certificate or its specific components, such as subkeys or identities, are considered invalid.
 
-In this section, we discuss the validity of certificates and their components. This discussion is closely related to [signature validity](/verification), and builds on that concept.
+Certificates can "expire," rendering them and their individual components invalid unless renewed. OpenPGP software will refuse to encrypt email using an expired certificate, adhering to the expressed preferences in the certificate's metadata. This refusal acts as a safeguard, prompting certificate owners to update or renew their certificates to maintain operational security.
 
-The validity of the signatures that link a certificate is a necessary precondition. Two concepts are particularly central to the validity of certificates and components:
+Expiration dates are set using *Key Expiration Time* subpackets for subkeys, and [*signature expiration time* subpackets](https://www.ietf.org/archive/id/draft-ietf-openpgp-crypto-refresh-12.html#signature-expiration-subpacket) for identity components. An expired binding signature invalidates the component it is associated with, emphasizing the critical role of timely updates.
 
-- Expiration
-- Revocation
+### The role of expiration in certificate freshness
 
-### Expiration
+The expiration mechanism serves a crucial function beyond merely invalidating old or compromised certificates. It acts as a proactive catalyst for certificate renewal, ensuring that the OpenPGP ecosystem remains vibrant with up-to-date cryptographic keys and identities.
 
-Certificates and components can "expire," which renders them invalid. Each component of a certificate can have an expiration time, or be unlimited in its temporal validity.
+Utilizing expiration dates fulfills two primary objectives: it compels the polling for certificate updates, such as from a keyserver, and it provides a passive mechanism for certificates to "time out." This is particularly vital if a certificate owner loses control over their certificates or is unable to issue a revocation.
 
-The OpenPGP software of a sender will refuse to encrypt email using an expired certificate, or using an encryption component key that is expired. The sender's software rejects encryption to the key, essentially as a courtesy to the certificate owner, respecting the preferences expressed in their certificate metadata.
+By mandating regular updates through expiration, the OpenPGP standard ensures that certificates reflect the current cryptographic stance of their owners, thereby enhancing the overall security and reliability of the network. This mechanism ensures that certificates maintain their relevance and trustworthiness, fostering a culture of active security management and vigilance among users.
 
-The expiration mechanism in OpenPGP is complemented by a mechanism to extend/renew expiration time.
+### Introduction to certificate revocation
 
-Using the expiration mechanism is useful for two reasons:
+Revocation is a critical security mechanism in the OpenPGP standard, allowing for the invalidation of a certificate or its components. Unlike expiration, which is inherently time-based, revocation is an explicit declaration that a certificate or component should no longer be trusted or used.
 
-- Expiration of a certificate means that it cannot be used anymore. This forces users of that certificate (or their OpenPGP software) to poll for updates for it. For example, from a keyserver.
-- It is a passive way for certificates to "time out," e.g., if their owner loses control over them, or isn't able to broadcast a revocation, for any reason.
+OpenPGP certificates are designed as "append only" data structures, meaning that once a component or signature is added, it cannot simply be removed. Instead, to invalidate a certificate or component, revocation signatures are issued and appended to the certificate. This method ensures that the historical record is preserved, maintaining a full audit trail of actions taken over the certificate's lifecycle.
 
-Component keys use *Key Expiration Time* subpackets for expressing the expiration time. Identity components rely on the [*signature expiration time*](https://www.ietf.org/archive/id/draft-ietf-openpgp-crypto-refresh-12.html#signature-expiration-subpacket) subpacket of their binding signature. If a binding signature expires, the binding becomes invalid, and the component is considered expired.
+### Revocation mechanisms and types
 
-### Revocation
+Revocation can apply to individual components of a certificate, such as User IDs and subkeys, allowing specific elements to be marked as invalid without affecting the overall certificate. However, revoking the *primary User ID* or the primary key with a [*Key revocation signature*](https://www.ietf.org/archive/id/draft-ietf-openpgp-crypto-refresh-12.html#name-key-revocation-signature-ty) (type ID `0x20`) has a more significant effect, marking the entire certificate and all its components as invalid and unusable.
 
-Since OpenPGP certificates act as ["append only" data structures](append-only), existing components or signatures cannot simply be "removed." Instead, they can be marked as invalid by issuing revocation signatures. These additional revocation signatures are added to the certificate.
+The OpenPGP standard facilitates various [*Reasons for Revocation*](https://www.ietf.org/archive/id/draft-ietf-openpgp-crypto-refresh-12.html#reason-for-revocation), each denoting the specific rationale behind the revocation. These reasons allow for nuanced distinctions between "soft" and "hard" revocations, impacting how the revocation affects the certificate's use:
 
-Each component, such as User ID and a subkey, can be revoked without affecting the rest of the certificate.
+- *Soft revocations* (e.g., *Key is superseded*, *Key is retired*, *User ID is no longer valid*) suggest that the revoked component may still have valid uses before the revocation time, allowing for historical verification of signatures.
+- *Hard revocations* signal a significant trust breach, such as key compromise, and are treated as valid from the moment of the key's creation, effectively invalidating it for all time.
 
-The *primary User ID* is an exception: when it is revoked, the entire certificate is considered invalid.
+### Semantics of revocations
 
-Revoking the primary key with a [*Key revocation signature*](https://www.ietf.org/archive/id/draft-ietf-openpgp-crypto-refresh-12.html#name-key-revocation-signature-ty) (type ID `0x20`) also marks the entire certificate, including all of its components, as invalid and unusable.
+Revocation semantics play a crucial role in the OpenPGP trust model, particularly in distinguishing between hard and soft revocations. Hard revocations are irreversible and signal that a certificate or component should never be used again, addressing scenarios where a private key is compromised. In contrast, soft revocations allow for the possibility that a component was valid in the past but is no longer appropriate for current use, such as when a subkey is retired in favor of a new one.
 
-### Semantics of Revocations
+This distinction is vital for evaluating the validity of components or signatures at a specified reference time. Hard revocations invalidate a component at all points in time, including before the creation of the revocation signature, to prevent misuse by attackers. Soft revocations, however, leave the door open for the component's use before the revocation, acknowledging its past validity[^undo-revocations].
 
-In contrast to expiration, revocation is typically final and not withdrawn[^undo-revocations].
+By understanding the mechanisms and semantics of revocation, OpenPGP users and implementers can more effectively manage and interpret the validity and trustworthiness of certificates within the ecosystem.
 
 [^undo-revocations]: While some revocations can be reverted, undoing revocations is an uncommon workflow. Unlike expirations, which are commonly undone by extending the expiration time.
 
-A revocation indicates that the component should not be used. Revocation signatures over components use a [*Reason for Revocation*](https://www.ietf.org/archive/id/draft-ietf-openpgp-crypto-refresh-12.html#reason-for-revocation) subpacket to specify further details about the reason why the component or certification was revoked. The OpenPGP format specifies a set of distinct [values for *Reasons for Revocation*](https://www.ietf.org/archive/id/draft-ietf-openpgp-crypto-refresh-12.html#table-10), and additionally provides space for a human-readable free text field for comments about the revocation.
-
-Some libraries, such as Sequoia PGP, expose these distinct reasons for users, enabling nuanced machine-readable statements by the revoker. Other implementations focus mainly on the distinction between "hard" and "soft" revocations.
-
-Of the defined revocation types, *Key is superseded*, *Key is retired* and *User ID is no longer valid* are considered "soft" revocations. Any other reason (including a missing *reason for revocation* subpacket) means that the revocation is "hard."
-
-The distinction between hard and soft revocations plays a role when evaluating the validity of a component or signature at a specified reference time: Hard revocations have unbounded [temporal validity](temporal-validity), they are in effect even before their creation time and therefore invalidate the revoked component or signature at all points in time.
-
-By contrast, a soft revocation leaves the revoked component or signature valid before the creation time of the revocation signature. A soft revocation can technically be overridden, for example, with a newer binding signature (the new binding signature and its metadata then shadow the revocation and re-connect and re-validate the component).
-
-Hard revocations address the following problem: If a private key was compromised, then the attacker can issue signatures using that key. This means, the attacker could issue a signature dated before the revocation, impersonating the owner of the key. A recipient of that signature would mistakenly consider this signature valid if the issuing key has been soft revoked. This is a problem.
-To counteract this problem, it is reasonable to clearly mark compromised keys as suspect at any point in time. That's what hard revocations do.
-
-On the other hand, if the subkey was merely retired using a soft revocation, and the certificate holder moved to a different subkey, then the signatures in the past, made by the retired key, are still valid.
-
-(append-only)=
-## Certificates are effectively append-only data structures
-
-OpenPGP certificates act as *append-only data structures*, in practice. Packets that are associated with a certificate cannot be "recalled", once they were published. Third parties (such as other users, or keyservers) may keep and/or distribute copies of those packets.
-
-While it is not possible to *remove* elements, once they were publicly associated with an OpenPGP certificate, it is possible to invalidate them by adding new metadata to the certificate. This new metadata could set an *expiration time* on a component, or explicitly *revoke* that component. In both cases, no packets are removed from the certificate.
-
-Invalidation resembles removal of a component in a semantical sense. The component is not a valid element of the certificate anymore, at least starting from some point in time. Implementations that handle the certificate may omit the invalid component in their representation.
-
-We have to distinguish the "packet level" information about a certificate from an application-level view of that certificate. The two may differ.
-
-### Reasoning about append-only properties in a distributed system
-
-OpenPGP is a decentral and distributed system. Users can obtain and transmit certificate information about their own, as well as other users', certificates using a broad range of mechanisms. These mechanisms include keyservers, manual handling, [Web Key Directory](https://datatracker.ietf.org/doc/draft-koch-openpgp-webkey-service/) (WKD) and [Autocrypt](https://en.wikipedia.org/wiki/Autocrypt).
-
-Different users' OpenPGP software may obtain different views of a particular certificate, over time. Individual users' OpenPGP instances have to reconcile and store a combined version of the possibly disparate elements they obtain from different sources.
-
-In practice, this means that various OpenPGP users may have differing views of any given certificate. For various reasons, not all users will be in possession of a fully up-to date and complete version of a certificate.
-
-There are various potential problems associated with this fact: Users may not be aware that a component has been invalidated by the certificate holder. Revocations may not have been propagated to some third party. So for example, they may not be aware that the certificate holder has rotated their encryption subkey to a new one, and doesn't want to receive messages encrypted to the previous encryption subkey.
-
-One mechanism that addresses a part of this issue is *expiration*: By setting their certificates to expire after an appropriate interval, certificate holders can force their communication partners to refresh their certificate, e.g. from a keyserver[^mgorny].
-
-[^mgorny]: See, for example, [here](https://blogs.gentoo.org/mgorny/2018/08/13/openpgp-key-expiration-is-not-a-security-measure/): "Expiration times really serve two purposes: naturally eliminating unused keys, and enforcing periodical checks on the primary key."
-
-Good practices, like setting appropriate expiration times, can mitigate the complexity of the inherently distributed nature of certificates.
-
-However, such mitigations by definition cannot address all possible cases of outdated certificate information in a decentralized, asynchronous system such as OpenPGP. So a defensive approach is generally appropriate when reasoning about the view of certificates that different actors have.
-
-When thinking about edge cases, it's useful to "assume the worst." For example:
-
-- Recipients may not obtain updates to a certificate in a timely manner (this could happen for various reasons, including, but not limited to, interference by malicious actors).
-- Data associated with a certificate may compound, and a certificate can become too large for convenient handling, even in the course of normal operations (for example, a certificate may receive very many legitimate third-party certifications). If such a problem arises, then by definition, the certificate holder cannot address it: remember that the certificate holder cannot "recall" existing packets.
-
-### Differing "views" of a certificate exist
-
-Another way to think about this discussion is that different OpenPGP users may have a different view of any certificate. There is a notional "canonical" version of the certificate, but we cannot assume that every user has exactly this copy. Besides propagation of elements that the certificate holder has linked to a certificate, third-party certifications are by design a distributed mechanism. A third-party certification is issued by a third party, and may or may not be distributed widely by them, or by the certificate holder. Not distributing third-party certifications widely is a workflow that may be entirely appropriate for some use cases[^tpc-privacy].
-
-[^tpc-privacy]: The two parties to a certification (the issuer and the target of the certification) may prefer not to publish their mutual association. Also see {ref}`social-graph-metadata-leak`.
-
-As a general tendency, it is desirable for OpenPGP users to have the most complete possible view of all certificates that they interact with.
-
-However, there are contexts in which it is preferable to only use a subset of the available elements of a certificate. We discuss this in the section {ref}`minimization`.
-
 (certificate-merging)=
-## Merging
+## 18.3 Merging and updating certificates
 
-As described above, OpenPGP certificates are effectively [append-only](append-only) data structures. As part of the practical realization of this fact, OpenPGP software needs to *merge* different copies of a certificate.
+OpenPGP's design as a decentralized and distributed system poses unique challenges and considerations for certificate management. This section explores strategies for merging certificates, handling unauthenticated information, and the implications of OpenPGP's append-only data structure on certificate validity and trust.
 
-For example, Bob's OpenPGP software may have a local copy of Alice's certificate, and obtain a different version of Alice's certificate from a keyserver. The goal of the implementation is to add new information about Alice's certificate, if any, to the local copy. Alice may have added a new identity, replaced a subkey with a new subkey, or revoked some components of her certificate. Or, Alice may have revoked her certificate, signaling that she doesn't want communication partners to use that certificate anymore. All of these updates could be crucial for Bob to be aware of.
+### Strategies for merging certificates
 
-Merging two versions of a certificate involves making decisions about which packets should be kept. The versions of the certificate will typically contain some packets that are identical. No duplicates of the exact same packet should be stored in the merged version of the certificate. Additionally, if the newly obtained copy contains packets that are in fact entirely unrelated to the certificate, those should not be retained (a third party may have included unrelated packets, either by mistake, or with malicious intent).
+Merging certificates in OpenPGP involves reconciling various pieces of information from different sources to construct a unified and up-to-date view of a certificate. This process is crucial when different copies of a certificate, each potentially containing unique data, need to be consolidated into a single, coherent certificate view.
+
+For example, Bob’s OpenPGP software may have a local copy of Alice’s certificate, and obtain a different version of Alice’s certificate from a keyserver. The goal of the implementation is to add new information about Alice’s certificate, if any, to the local copy. Alice may have added a new identity, replaced a subkey with a new subkey, or revoked some components of her certificate. Or, Alice may have revoked her certificate, signaling that she doesn’t want communication partners to use that certificate anymore. All of these updates could be crucial for Bob to be aware of.
+
+Key strategies for effective certificate merging include:
+
+- **Timestamp verification**: Compare the timestamps on certificate components from different sources. Retain the most recent information, such as the latest revocation status or key expiration dates, to ensure the certificate view is current.
+- **Signature validation**: Verify the signatures on each component of the certificate. Only incorporate components with valid signatures into the merged view, discarding any data that cannot be authenticated.
+- **Conflict resolution**: When conflicting information is present (e.g., differing subkey sets from two sources), prioritize data from more reliable or directly obtained sources, such as WKD or a direct exchange with the certificate owner.
+- **Revocation checks**: Pay special attention to revocation signatures. A revocation signature supersedes all other information about the revoked component, marking it as invalid regardless of other data.
+
+Implementing these strategies helps OpenPGP software and users to manage the complexities of certificate information distributed across the decentralized OpenPGP ecosystem, ensuring that the operational view of a certificate is both comprehensive and accurate.
 
 ### Handling unauthenticated information
 
-For information that *is* related to the certificate, but not bound to it by a self-signature, there is no generally correct approach. The receiving implementation must revolve these cases, possibly in a context-specific manner. Such cases include:
+In the OpenPGP ecosystem, certificates often accumulate information that is not directly authenticated by the certificate owner, such as third-party certifications or subkeys. Handling this unauthenticated information requires careful consideration to maintain the trustworthiness of the certificate.
 
-- Third-party certifications. These could be valuable information, where a third party attests that the association of an identity to a certificate is valid. On the other hand, they could also be a type of spam.
-- Subpackets in the unhashed area of a signature packet. Again, these could contain information that is useful to the recipient. However, the data could also be either useless, or even misleading/harmful.
+For information that *is* related to the certificate but not bound to it by a self-signature, implementations need to navigate these cases thoughtfully, often in a context-specific manner. Challenges include:
+
+- **Third-party certifications**, which may either be valuable endorsements of an identity’s association with a certificate or could serve as spam.
+- **Subpackets in the unhashed area of a signature packet**, which could contain either useful or potentially misleading information.
+
+The key is to balance the inclusion of potentially valuable third-party information with the risk of incorporating misleading or harmful data. Strategies may include user-configurable trust settings, software that prioritizes data verified by trusted sources, and mechanisms for users to manually verify or reject unauthenticated information.
+
+(append-only)=
+### OpenPGP certificates as append-only data structures
+
+OpenPGP certificates act as *append-only data structures*, in practice. Once packets are associated with a certificate and published, they cannot be "recalled." This ensures the integrity and auditability of certificate changes over time. Third parties, including other users and keyservers, may keep and distribute copies of these packets.
+
+Invalidation of components, achieved by adding new metadata such as expiration times or revocation signatures, does not remove packets from the certificate. Instead, it semantically "removes" the component's validity, allowing implementations to omit these invalid elements from their operational view of the certificate.
+
+### Reasoning about append-only properties in a distributed system
+
+The decentralized nature of OpenPGP allows users to share and receive certificate information through various mechanisms, including keyservers, manual handling, [Web Key Directory](https://datatracker.ietf.org/doc/draft-koch-openpgp-webkey-service/) (WKD), and [Autocrypt](https://en.wikipedia.org/wiki/Autocrypt). This can lead to different users having different views of a certificate at any given time.
+
+These differing views necessitate a process within OpenPGP software to reconcile and store a combined version of the certificate elements obtained from disparate sources. This aspect underlines the importance of assuming a defensive posture when reasoning about the validity and completeness of certificate information in OpenPGP's distributed environment.
+
+When considering edge cases within this distributed system, it's prudent to "assume the worst." For instance:
+
+- Recipients may not obtain updates to a certificate in a timely manner, for various reasons including, but not limited to, interference by malicious actors. This delay in receiving updates could lead to the use of outdated or compromised keys, undermining the security of communications.
+
+- Data associated with a certificate may compound over time, leading to a certificate that is too large for convenient handling. This situation can arise when a certificate accumulates a significant number of legitimate third-party certifications. In such cases, the certificate holder is limited in their ability to manage the certificate's size, as the OpenPGP standard does not allow for the direct removal of existing packets.
+
+### Differing views of a certificate
+
+Variability in certificate views among OpenPGP users highlights the system's inherent complexity and the challenges in ensuring all parties have an accurate, up-to-date understanding of a certificate's status. Defensive practices, such as regular certificate updates and careful consideration of certificate expiration and revocation, are vital in navigating these challenges.
+
+Different OpenPGP implementations may handle these challenges in various ways, emphasizing the need for users to understand the potential for discrepancies and adopt practices that ensure the security and reliability of their cryptographic communications.
+
+The diversity of certificate views, influenced by distributed third-party certifications, underscores the complexity of managing privacy and trust within OpenPGP. Users must navigate these challenges with an understanding that not all certificate information may be equally available or desirable to share widely, emphasizing a cautious approach to certificate data management[^tpc-privacy]..
+
+One mechanism that addresses a part of this issue is *expiration*: By setting their certificates to expire after an appropriate interval, certificate holders can force their communication partners to refresh their certificate, e.g., from a keyserver[^mgorny]. This approach not only helps in naturally eliminating unused keys but also enforces periodical checks on the primary key, thereby promoting a consistent and up-to-date view of the certificate across different users.
+
+[^tpc-privacy]: The two parties to a certification (the issuer and the target of the certification) may prefer not to publish their mutual association. Also see {ref}`social-graph-metadata-leak`.
+[^mgorny]: See, for example, [here](https://blogs.gentoo.org/mgorny/2018/08/13/openpgp-key-expiration-is-not-a-security-measure/): "Expiration times really serve two purposes: naturally eliminating unused keys, and enforcing periodical checks on the primary key."
 
 (minimization)=
 ## Certificate minimization