What newsgroups are and how they work

Concisely, newsgroups are a means of public discussion. Newsgroup articles (messages) look like email, but millions of people all over the world can read them.

This note describes how newsgroups work in the sense of what happens to articles after they are posted. In order to find out how newsgroups work in the sense of how to use your news-reading software, please look for documentation for that software or post a question to an appropriate newsgroup.

• If you are using a web browser, such as Netscape, or Microsoft Internet Explorer, try comp.infosystems.www.browsers.ms-windows or comp.infosystems.www.browsers.mac .
• If you are using Microsoft Outlook Express, try microsoft.public.outlookexpress.general .
• If you are using another newsreader, try news.software.readers or news.newusers.questions .

Questions and answers

Can someone give a brief, not-too-technical description of where a message goes when I post it into a newsgroup?

Newsgroup articles are distributed via news servers, which contain databases of articles. Internet service providers (ISPs), schools and other organisations operate news servers.

Do all the messages travel to a central site, and do all the sites that want it pick it up there?

No, there is no central server on Usenet. A newsgroup article propagates from one server to another, starting from the server where it is first posted.

Moderated newsgroups are sort of an exception, in that all articles are first forwarded via email to a moderator for approval. The moderator posts them on his or her news server; from there, they propagate as described below. Different moderators use different news servers.

Do all messages travel to all news servers?

Ideally, all articles in a newsgroup would travel to all news servers that carry the newsgroup. When you post an article, the result is tens of thousands of copies, all over the world.

More specifically, when you post an article, it goes first to your local news server (operated e.g. by your ISP or school). Your server then sends copies of the article to its "neighbours", that is, to servers with which it has agreed to exchange articles. Those servers, in turn, send copies to their neighbours. Eventually, every server that carries the newsgroup has a copy.

In what order?

Most servers normally forward articles more or less in the order of arrival. This sequence can become scrambled for various reasons, which is why you often see responses before the original article arrives.

How do they know which servers want them?

News server administrators arrange among themselves which newsgroups they exchange. The receiving server's admin tells the sending server's admin which newsgroups he or she wants to receive. The sending server's admin then configures his or her server to send only those newsgroups.

How do they know which sites they have already visited? How does a message avoid the same site twice?

There are two methods. Servers usually use both of them, in sequence:

1. The Path: header line shows the sites that the article has travelled through, so far, between the originating server and the current server. If the receiving server appears in the Path: line, the sending server does not try to send the article, because it knows that the receiving server already has received a copy.
2. The Message-ID: header line contains an identifying code that is different for every article. Before transmitting the article, the sending server asks the receiving server, in effect, "Do you have an article with Message-ID such-and-such?" The receiving server responds either "No, please send me a copy" or "Yes, I have it already", whereupon the sending server either sends the article or goes on to the next one.

How long do messages stay posted?

Each news server removes old articles, usually once a day, to make space for new ones. This is called expiring. Most servers do this based on the number of days an article has been on that server. The expiration time varies from one server to another, and can vary from one newsgroup to another on the same server. It might be less than a day (such as for groups that carry binary content), or it might be two weeks or more. Therefore, even after an article expires from your own server, it is probably still visible on many other servers.

Note, however, that even though you cannot see an article any more, it may still be present on your server. Most news-reading software keeps track of which articles you have read, and shows those articles to you only once. This way, you do not have to wade through the same articles repeatedly. There should be a command, button or something that "shows all", "shows previously-read" or "unmarks" articles so that you can see all the articles that have not yet expired from your server.

Behind the scenes, how does a newsreader communicate with a news server?

An example of how an article posting takes place on the news transfer protocol level is available on the [file:///C:/misc/telnet-troubleshooting/nntp.html "How to test an NNTP (news) server using telnet"] page. You can score guru points with your friends by reading and posting news without using a newsreader. ☺