gale.org
Home > Development > Revolution > Old Proposal > Beland's Counter@, Meacham/LaForge Extension

Looking to skip all boring stuff? You'll regret it later, but here you go.


Requirements

If you can't agree with anything else in this document, I hope you can agree with these requirements. (If you can't agree with the requirements, I'm sure you won't agree with anything else.) Conventions as per IETF.

R1. The proposal MUST define a single, uniform 'location' syntax unifying keys and categories.
A single 'location' string will identify both the category of a message and the key with which it is encrypted.
R2. It MUST be possible to reconstruct a location from the key and category of a message.
There MUST be one-to-one mapping between locations and valid (key,category) pairs. There MAY exist "invalid" (key,category) pairs which do not correspond to any location. Clients MAY define "backdoor" techniques for advanced users to construct such "invalid" messages. Clients MUST NOT require users to understand these techniques in the normal course of operation.
R3. Standard user addresses MUST be valid locations under this syntax.
Users will still be able to send to ordinary, email-like addresses of the form "user@some.domain". When used, these addresses MUST have the expected behavior of encrypting the message for that user and delivering it to a "personal category".
R4. The same location MUST refer to the same key and category regardless of context.
The result of interpreting a location MUST NOT depend on any attribute or state of the interpreter. The result MAY depend on globally available state, including the characteristics of accessible keys.
R5. Any subcategory of a valid location MUST also be a valid location.
If a valid location is formed from a key and a category, it MUST be possible to form a different valid location from the same key and any subcategory of the original category. This requirement should satisfy the common need for "encrypted groups" and "personal subcategories".
R6. The hierarchy of keys and categories defined by locations MUST be defined by delimited 'words'.
The existing Gale key hierarchy is defined by words, but categories are currently defined by a sequence of characters. The proposal MUST unify both into a delimited-word system, and the proposal MUST define the standard delimiter used.
R7. The proposal SHOULD define 'alias' or 'shortcut' mechanisms for convenient entry of locations.
Depending on their needs, some clients MAY choose to use nonstandard alias mechanisms. The alias mechanism SHOULD be locally configurable. All clients SHOULD accept and display full standard locations as well as shortened or altered forms. Any aliases MUST be fully expanded before a message is sent to a Gale server. It MUST NOT be necessary to configure or understand these aliases to use Gale.
R8. Location syntax MUST be easy for end users to comprehend and generate.
This requirement is ill-defined but very important. Complex, confusing, or error-prone syntax MUST NOT be used.

Assumptions

The proposal makes some assumptions. If you disagree with these, you will probably not like the proposal.

We assume that all categories have a 'root', or a point in the hierarchy beyond which it is no longer generally useful to subscribe. For example, while users of the current system may subscribe to 'group/ofb' or 'group/slackers.net', it is not generally useful to subscribe to 'group', or to the empty string ''. Likewise, it is sensible to subscribe to a user's individual category '@some.domain/user/name/', but (except for specialized analysis purposes) it is not sensible to subscribe to all user categories at '@some.domain/user/'.

Furthermore, we assume that it is reasonable to expect that all messages posted within the scope of categories defined by such a 'root' will be encrypted with the same key. This assumption implies that users will have fewer private keys than subscriptions, since each private key will require at least one unique subscription.


Proposal

Examples

Old  New (long form)  New (using aliases)  
gsend -c pub/comp/linux  gsend pub.comp.linux@ofb.net  gsend pub.comp.linux  
gsend -c local/seattle/weather  gsend local.weather@seattle.wa.us  gsend seattle.weather  
gsend tlau@cs.washington.edu  gsend tlau@cs.washington.edu  gsend tlau@uw  
gsend -c group/ofb/haqrz egnor  gsend ofb.haqrz@ofb.net egnor@ofb.net  gsend ofb.haqrz egnor  
gsend -C @ugcs.caltech.edu/user/egnor/mail egnor  gsend egnor.mail@ugcs.caltech.edu  gsend egnor.mail@ugcs  
gsend -C group/slackers.net/admin slackers  gsend group.admin@slackers.net  gsend slack.admin  
gsend -c pub/toe-suckers:/tangent  gsend pub.toe-suckers@ofb.net /tangent  gsend pub.toe-suckers /tangent  
gsend -c pbu/typo  gsend pbu.typo
ERROR: No key found for 'pbu@ofb.net'
  gsend pbu.typo@ofb.net
ERROR: No key found for 'pbu@ofb.net'
  
gsub @ugcs.caltech.edu/user/egnor: @ofb.net/user/egnor:pub:-pub.toes  gsub egnor@ugcs.caltech.edu egnor@ofb.net pub@ofb.net -pub.toes@ofb.net  gsub egnor@ugcs egnor pub -pub.toes  
gsub group/cabal  gsub cabal@secret.com
WARNING: You have no key for 'cabal@secret.com'!
     
GALE_ID=webgale.whuang  GALE_ID=whuang-webgale@ofb.net  GALE_ID=whuang-webgale  

Definition

This is the full location format:

key
ofb
subkey*
-admin
subcategory*
.haqrz.evil

@
domain
ofb.net

* Can be repeated.

As discussed in the requirements, this proposal defines a 'location' syntax that encompasses both the key used in the message and its category. In this proposal, there is a one-to-one mapping between locations and categories, and a many-to-one mapping from locations to keys. This means that a message's category fully determines the key used to encrypt that message. All messages are processed with some key, but "null keys" (such as "pub@ofb.net") may be created; messages processed with these keys are not encrypted. (These keys are still signed as usual.)

To be perfectly clear, the category includes the key. If you use the location "pub.foo.bar@ofb.net", the category used is "pub.foo.bar@ofb.net", and the key used is "pub@ofb.net".

The only special characters are "@", ".", and "-". Everything else may be part of a domain, key, or category component. Additionally, "-" may be part of a category or domain component. Domain components are further restricted by the DNS standard, and key components will in general use be further restricted by e-mail standards and username limits.

Internal Form

We define an "internal form" for these locations. To convert from a location in standard form to internal form, remove the "@domain", reverse the components of the domain, and affix "@", the reversed domain, and "/" to the beginning of the category. Thus, the standard form location "test.foo@ugcs.caltech.edu" becomes "@edu.caltech.ugcs/test.foo" in internal form. If the domain is missing (due to condiments or unexpanded aliases), the internal form is the same as the standard form: "pub.stuff", "/offensive.toesucking".

The advantage of internal form is that, unlike standard form, the syntax is strictly ordered from most generic to most specific, allowing simple computation of subscription semantics. This is also the order of key signing: ROOT signs "edu", "edu" signs "caltech", "caltech" signs "ugcs", and "ugcs" signs "test". ("foo" is a subcategory, not a key.) This reordering is analogous to the common reordering of personal names, where "Daniel Trawick Egnor" becomes "Egnor, Daniel Trawick".

The internal form can be distinguished from the standard form by its use of the leading "@". Clients could accept the internal form in addition to the standard form for input, though it is quite unnecessary to do so.

Aliases

Aliases are used to eliminate repeated typing. They are defined by a set of expansion rules set by individual users and site maintainers. Their operation is best defined in terms of the internal form:

The first word of the internal form is extracted. (This word may be part of the domain, if a leading "@" is present, or not otherwise.) The system and user alias tables are searched for that word. If the word is not found, alias expansion stops. Otherwise, the word is replaced by the internal form of the word's definition in the alias table. (Various rules are invoked here to keep the syntax valid.) Alias expansion is run anew on the result. No alias is allowed to be expanded more than once for the same input.

If, after alias expansion, no domain portion is present, the local default domain is added to the location.

Aliases may be used to create short names for domains ("@ugcs" expands to "@ugcs.caltech.edu"), to create aliases for remote users or forums ("pub" becomes "pub@ofb.net", and therefore "pub.stuff" becomes "pub.stuff@ofb.net")

Compatibility

The internal form is very similar to today's directed categories (with the domain reversed). By unreversing the domain, we could use that mechanism directly without changing the servers at all. My proposal at this point is to create a new protocol version which uses the 'location' syntax (or possibly internal form) directly, but to include code in the protocol handler to automatically translate categories for old servers.

Implementation Plan
Mumble. To be determined.

Variations

Delimiters

Delimiter choice remains controversial, as always. For discussion purposes, let us use "ABC" to represent the use of "A" as a subkey delimiter, "B" as a key/category separator, and "C" as a subcategory delimiter.

The current proposal is thus "- . .". The Meacham proposal suggests using ". / /" (among other changes). The category separator could be a space, which frees the ":" character for use a delimiter.

Delim.  Problems  
- . .  In the domain component, '-' is not a delimiter, and frequently occurs inside words. Since subkeys share many attributes with domain components (and both participate in a single key-signing hierarchy), this is confusing and inconsistent. Furthermore, many usernames contain '-'. All told, '-' is probably unsuitable for any delimiting purpose.  
. . .  If the subkey delimiter and the key/category separator are the same character, keys and categories must be distinguished programmatically in some nonsyntactic fashion. Proposals include simply attempting to look up all possible keys (and using the longest one that matches) or setting a special "terminator bit" in "leaf keys" to indicate the transition to "category space". These methods tend to be cumbersome.  
. / .  Using a different key/category separator and delimiter makes the user forcibly aware of the distinction between the "key part" and the "category part". Categories "pub/comp.linux@ofb.net" certainly look strange to Gale veterans, and the punctuation potpourri will arguably confuse newbies as well. (Others do argue that the distinction is important and should be highlighted rather than swept under the rug.)  
. / /   Slashes are ugly, and they connote filesystem semantics which are out of place in a system based on e-mail addresses. Only a radical slash loyalist could love these delimiters.   
. : .
: . .
  The colon has a very low precendence, typographically. Categories like "pub:comp.linux@ofb.net" look uncomfortably like URLs that use the protocol "pub"; in any case, "comp.linux@ofb.net" seems entirely subsidiary to "pub", which is at odds with the actual semantics of the location string. This problem unfortunately remains no matter where ":" is placed or what it is combined with. Some argue that this is not as disastrous as it seems, because while the perceived binding may be incorrect, at least (domain aside) the hierarchy is valid (everything except the domain is subsidiary to "pub").  
/ . .  While subkeys are rare, and subkey/subcategory combinations rarer still, the "/" character is so much weightier than the "." that users will generally assume that the subkey is more strongly connected to the category than it is to the main key. As above, some argue that this is not a serious problem.  

Keys

John Meacham and others have a counterproposal which omits the key part entirely to indicate a public (unencrypted) category, e.g. "/pub/toes/big@ofb.net". (Remember, they suggest ".//" delimiters.) By allowing the key to be completely absent, the need for "null keys" is removed.

Format

We could rearrange the entire string, using "key@some.domain/category" instead of "key.subcat@some.domain". Chris Beland argues strongly in favor of such a system in his counterproposal document. This proposal no longer requires keys and categories to match; under such a system, it is necessary to define some convention to make sure that "egnor@ofb.net" and "tlau@ofb.net" refer to distinct categories. I think this question is fundamentally related to the assumptions I make; if you don't believe those assumptions, then you will want to keep category and key independently variable; in that case, Beland's proposal is the logical syntax.

Chris Beland later submitted a second counterproposal, which (near as I can tell) advocates a combined URI/e-mail syntax with rewrite rules to convert from one to the other, as well as more flexible category matching (allowing substring match in addition to hierarchical prefix matching).

Pro "key.subcat@some.domain"  Pro "key@some.domain/category"  
Locations more closely resemble e-mail addresses.  Locations can be easily transformed into URIs.  
The binding "(key.subcat)@(some.domain)" is more syntactically evident than "(key@(some.domain))/subcat", requiring fewer levels of operator precedence.  The most specific components (the end of the subcategory and the beginning of the key) are clearly visible at the ends of the location, rather than buried int he middle.  
No mapping is necessary for private categories.  Categories and keys can be mixed and matched.  
Public discussion areas must be explicitly created by domain administrators.  Public discussion areas do not have to be explicitly created by domain administrators.  
The key is derived from the category, relieving the user of the need to understand the distinction between "keys" and "categories".  Categories and keys are syntactically distinct, reflecting their semantic differences.  

Condiments

Condiments remain an issue. I don't like the leading slash syntax, and its meaning has not been elucidated. Beland's proposal deems condiments to be part of the message content, not the location, but this precludes server-side condiment filtering.