How to Use Checkgroups and Control Messages

The purpose of this page is to provide information to News Service Providers about how to configure their news server to automatically create or remove newsgroups, thereby keeping their lists of Big-8 newsgroups in compliance with the Checkgroups message posted monthly in news.announce.newgroups.

What is a Control Message?

• "Control messages are specially-formatted messages that tell news servers to take various actions" ([1]).
• Control messages for the Big-8 hierarchies are PGP-signed and are, for the most part, used to add or subtract newsgroups from Checkgroups.
• From Allbery's Usenet Hierarchy Administration FAQ:

Most hierarchies now PGP-sign all control messages. The PGP signature is visible in the X-PGP-Sig header of the control message.

Background information and (now somewhat outdated) instructions are at:

ftp://ftp.isc.org/pub/pgpcontrol/README.html

and the exact format of the signature is at:

ftp://ftp.isc.org/pub/pgpcontrol/FORMAT

What is Checkgroups?

• It is the canonical list of groups that constitute the Big-8.

• "The 'application/news-checkgroups' body part contains a complete list of all the newsgroups in a (sub)hierarchy, their newsgroup-descriptions and their moderation status" (News Article Format and Transmission draft 13.01).

• Since 1996, when tale (David C. Lawrence) created the PGP key used by news.announce.newgroups moderators, there has been a reliable mechanism for identifying the newsgroups that belong in the Big-8 namespace. The Big-8 Technical Team updates the list of valid Big-8 groups whenever a group is added or removed, and that list is sent out monthly both as a checkgroups control message and as a posting to news.announce.newgroups. This trustworthy list is also available from the ISC ftp server.

Why should I care about following control messages or Checkgroups?

• In brief: because your customers will be best served if you provide them with access to groups that are well-propagated on news servers run by other NSPs.

• One of the beauties of Usenet is that it is sustained by the free cooperation of news administrators throughout the world who agree to carry the same newsgroups as their competitors. When news administrators consistently follow the Big-8 control messages and/or use Checkgroups to align their server with other news servers, a post made on their own system will be broadcast to other compliant news servers around the world and their server, in turn, will receive the posts that are initiated on other news servers. It is a win/win situation for all involved, both news administrators and their customers.

• Customers are best served by having a list that is reliable and well-propagated.
  • Reliable: groups on the list are alive, not dead; bogus groups are weeded out and missing groups are supplied.
  • Well-propagated: carried on news servers outside the orbit of their own NSP.

• Automated compliance will help make sure that posts to moderated groups get inserted into the right stream to be approved and circulated worldwide. When a moderated group is misconfigured, the posts do not show up on properly configured servers elsewhere in the world. Customers do not get to participate in the group by having their posts appear where all other participants in the group can see them.

• News administrators who verify the control messages themselves or who use Checkgroups keep their server in sync with thousands of other news servers around the world, thus providing their customers with the largest possible audience for their Big-8 posts and with the largest number of posts to read.

• Your fellow news administrators at other NSPs or in non-profit organizations may, in one sense, be competitors for your customers, but in a larger sense are your colleagues. All of you benefit when there is a reliable method for carrying the same set of Big-8 newsgroups.

How Do I Verify PGP Signatures on Control Messages?

General

Our key signature is '0xB88DA9C1', and our key is available through most standard PGP key servers (including subkeys.pgp.net), or at http:archives.eyrie.org/software/pgpcontrol/PGPKEYS

Our key fingerprint is:

 F5 35 58 D3 55 64 10 14  07 C6 95 53 13 6F D4 07

A good starting point for verifying control messages is 'pgpcontrol', which comes with INN, or separately at http://archives.eyrie.org/software/pgpcontrol/

For an explanation of PGP, try this document: "Pretty Good Privacy" by David E. Ross

INN

As noted, above, software to handle PGP control messages comes with INN. More information is available as part of the INN documentation - Configuring INN to Verify Control Messages

Diablo

From http://www.openusenet.org/diablo/faq/faq.html#Utilities0 :

 dsyncgroups - Synchronise the active file with a another news server

High Winds

From http://contrib.highwinds-software.com/support/contributed_details.aspx?contributeID=16 :

 control.pl is a perl script that should be run from cron periodically to
 process newgroup and rmgroup control messages.  It uses the same
 control.ctl file format as INN, making for a smooth transition from INN.
 It also supports the pgpverify control message authentication mechanism.

Other Software Packages / By Hand

• Download pgpverify at ftp://ftp.isc.org/pub/pgpcontrol/
• Configure pgpverify to use gpgv, not to use syslog, and to not look for INN shell variables (all should be fairly obvious modifications to the script, but must be custom to your local setting
• Run 'gpg --recv-keys 0xB88DA9C1' to download the appropriate Big-8 signing key from the public key servers. (We use subkeys.pgp.net.)
• Run 'pgpverify < msg' on the control messages you want to verify. ('pgpverify -test < msg' will give you a lot of useful debugging information as well.)

While we cannot offer a simple drop-in solution for custom news servers (they are custom, after all), this should offer a good starting point for writing your own scripts to automatically verify control messages. Set up your server to pass control messages to this script, and if the control message passes, call another script to modify your group list accordingly.

How do I automate compliance with control messages?

• One of the easiest methods is to sync with the newsgroups list published in the ISC ftp index. This list is updated hourly. News admins can pull down the newsgroups file, extract the hierarchies they care about, and then use that to update their group list. That's a lot easier than setting up pgp signature validation.
• If administrators are using a major usenet news package like inn, diablo, dnews, etc., then of course the capability and configuration generally comes with the package, but that doesn't help with custom software.

How do I verify a control message by hand?

• Download pgpverify at ftp://ftp.isc.org/pub/pgpcontrol/
• Configure pgpverify to use gpgv, not to use syslog, and to not look for inn shell variables (all should be fairly obvious modifications to the script, but must be custom to your local setting
• Run 'gpg --recv-keys 0xb88da9c1' to download the appropriate big-8 signing key from the public key servers. (we use subkeys.pgp.net.)
• Run 'pgpverify < msg' on the control messages you want to verify.

How can I check checkgroups regularly?

Doug Mclaren has written a script to show the differences between checkgroups and your nntp master list:

# This script is designed to compare the active file on your NNTP
# server vs. the master list kept at ftp.isc.org to see which groups
# should (could) be added, removed or changed.  It does not require
# any special privileges to run beyond access to a NNTP server and to
# the Internet, but of course you will need admin access to the NNTP
# server itself to make any changes.
#
# It will not actually make any changes by itself, but can instead
# give you a list of commands that can be run (if you're running innd)
# to make your server match the master list.  (If you're not running
# innd, send me the equivilent commands for your server and I'll add
# support for it.)

• The current version of this script is available from Doug's web site.
• Sample output:
  • Easynews
  • austin.rr.com

Very temporary FAQ maintainer: moleski@canisius.edu