How to download binaries from USENET news on ISIS

When people see or find out about my relatively large collection of graphics files and that I got them off of the Internet, they often ask me, "How do you that?" or tell me, "I tried to do that one night, but it didn't work." This HOWTO is intended to convey my method of downloading binaries off of USENET, a method that is relatively straightforward and uncomplicated. It is directed at users of ISIS here at UNC, but the method should be usable by users of other UN*X-based systems with USENET news access.

Getting Started

First of all, you need an ISIS account. If you do not already have an ISIS account, you may obtain one by connecting to isisa.oit.unc.edu or email.unc.edu and logging in as "freemail". You can do this from any of the on-campus computer labs or over the University dialup lines. ISIS will ask you to enter some information about yourself in order to verify your affiliation with the University of North Carolina at Chapel Hill. Once ISIS verifies your affiliation, an account will automatically be generated for you and be activated in about 24 to 48 hours.

If you will be accessing ISIS over the dialup lines, you will need a terminal program. Your terminal software should support the Zmodem file transfer protocol. I suggest Telix for PC users. COMit, a Windows-based program distributed with some brands of modems is UNACCEPTABLE for downloading files with Zmodem (to put it simply, it does not work.) As I am only a casual Mac user, I cannot recommend a good Mac terminal program (other than to get a real computer and some real terminal software.) If someone has a good Mac terminal program they would like to recommend, please let me know, and I will add it to this page. User Services should have terminal programs suitable for use with ISIS for both PCs and Macs.

If you will be downloading binaries from the on-campus computer labs, all you really need is some floppy disks on which to store them if you want to keep them. The PCs in the campus computer labs have Lview, an excellent MS Windows-based freeware graphics viewing program, installed on the network. The PCs also have a program called WPLANY which is adequate to play *short* sound files. At the time of writing, the campus PCs do not have any software to play animation files (.MOVs, .AVIs, .FLCs, .MPGs, etc.).

Setting up a temporary directory to hold decoded binaries

Ok, by this point you should have your ISIS account and a means to access ISIS, be it dialup access of through one of the campus computer labs. Log into ISIS with your user ID and password. After you log in, you should be greeted by the ISIS menu. Select option 7 to exit the menu to a tcsh shell prompt.

Because of all the users ISIS supports, ISIS enforces some pretty draconian space restrictions. As of the time of writing, users were restricted to 3MB of storage. You could easily run up against the 3MB limitation when decoding binary files from USENET, especially if you start decoding multimedia files. However, since we only need to have the files on ISIS long enough to download them, we can "avoid" the space restriction by using the /scratch filesystem. The /scratch filesystem exists for users who need storage in excess of their disk quotas for short periods of time. The /tmp filesystem could also be used for the same purpose, but there is a different /tmp filesystem for each ISIS node, and the /tmp file system on one ISIS node is not broadcast to the nodes. So, the administrators of ISIS created the /scratch filesystem for users to use as temporary storage and configured it so that its contents are broadcast to all ISIS nodes. We will use the /scratch file system as a "way station" to store our binaries long enough to download them. First, change your working directory to /scratch with the following command:

cd /scratch

Next, create a subdirectory for yourself. For simplicity sake, use YOUR username.

mkdir astudent

Now, change your working directory to your newly-created subdirectory.

cd astudent

You can store files in the /scratch file system for about 24-48 hours. The SysOps of ISIS routinely nuke the /scratch filesystem in order to free up space, so do not leave any valuable files in the /scratch filesystem. While your subdirectory in the /scratch filesystem exists, you can make it your working directory with the following command:

cd /scratch/astudent

If you get a message that the directory does not exist, then the /scratch filesystem has been purged. Just create a new subdirectory using the above technique. If at anytime you want to return to your home directory, just enter

cd

at the command prompt. To go back to the ISIS menu, just enter

exit

at the command prompt.

Starting TIN

Now, we need to invoke the TIN, the USENET newsreader we will be using to select and extract our binaries off of USENET. Unlike the some other newsreaders like RN and TRN, TIN allows us to decode binaries straight off of USENET simply by selecting all parts of the file and entering a command. Invoke TIN at the command prompt with the following command:

tin

You should then see an intro message like this:

tin 1.2 PL2 [UNIX] (c) Copyright 1991-93 Iain Lea.
Connecting to news.oit.unc.edu...
Reading news active file...

TIN may then ask you whether you want to subscribe to certain new newsgroups with a prompt like this one:

Subscribe to new group alt.movies.independent (y/n/q) [n]:

If you want to subscribe to the newsgroup, hit y. If you don't, hit n. If you hit either y or n, TIN will ask you the same question for all of the new newsgroups that have just become available. If you do not want to subscribe to any of the new newsgroups, hit q and you will bypass all of them.

You should then see a screen like this:

                     Group Selection (news.oit.unc.edu  4654)     h=help

       1    61  alt.1d                                                         
       2   104  alt.3d                                                         
       3   267  alt.abortion.inequity                                          
       4     4  alt.abuse.offender.recovery                                    
       5    16  alt.abuse.recovery                                             
       6   960  alt.activism                                                   
       7   261  alt.activism.d                                                 
       8     8  alt.adjective.noun.verb.verb.verb                              
       9   132  alt.adoption                                                   
      10     1  alt.aeffle.und.pferdle                                         
      11    15  alt.agriculture.fruit                                          
      12    30  alt.agriculture.misc                                           
      13    10  alt.aldus.freehand                                             
      14   127  alt.aldus.pagemaker                                            
      15    33  alt.alien.vampire.flonk.flonk.flonk                            
      16   781  alt.alien.visitors                                          Are

       <n>=set current to n, TAB=next unread, /=search pattern, c)atchup,
    g)oto, j=line down, k=line up, h)elp, m)ove, q)uit, r=toggle all/unread,
      s)ubscribe, S)ub pattern, u)nsubscribe, U)nsub pattern, y)ank in/out

TIN has placed us at the top of the list of newsgroups. Binary files on USENET are posted for the most part to the newsgroups in the alt.binaries.* hierarchy. Multimedia files can be found in the alt.binaries.multimedia.* hierarchy of groups, graphics files can be found in the alt.binaries.pictures.* hierarchy, and sound files can be found in the alt.binaries.sounds.* hierarchy. Two groups to always keep an eye on are alt.binaries.pictures.utilities and alt.binaries.sounds.utilities. These group contain postings of software that can be used to view or hear the files you will be downloading off of USENET.

For a rundown of some of the major binaries groups, hit / (the search command) and then enter alt.binaries, like so:

Search forwards []> alt.binaries

You will then see a screen that looks something like this:

                     Group Selection (news.oit.unc.edu  4654)     h=help

      49     2  alt.bbs.pcbuucp                                                
      50    13  alt.bbs.unixbbs                                                
      51     1  alt.bbs.unixbbs.uniboard                                       
      52     2  alt.bbs.uupcb                                                  
      53     2  alt.bbs.waffle                                                 
      54    12  alt.beadworld                                                  
      55   187  alt.beer                                                       
      56     6  alt.beer.like-molson-eh                                        
      57    46  alt.best.of.internet                                           
      58   153  alt.bigfoot                                                    
      59   933  alt.binaries.multimedia                                        
      60        alt.binaries.pictures                                       ? (
      61   271  alt.binaries.pictures.d                                        
      62  8401  alt.binaries.pictures.erotica                                  
      63  1100  alt.binaries.pictures.erotica.blondes                          
      64    36  alt.binaries.pictures.erotica.d                                

       <n>=set current to n, TAB=next unread, /=search pattern, c)atchup,
    g)oto, j=line down, k=line up, h)elp, m)ove, q)uit, r=toggle all/unread,
      s)ubscribe, S)ub pattern, u)nsubscribe, U)nsub pattern, y)ank in/out

Unfortunately, TIN does not sort newsgroups in alphabetical order. You can tab through the groups with the UP and DOWN arrow keys or with k to go up a line or j to move down a line. When you reach the end of the first set of binaries groups, hit / and then the return key to find other binaries newsgroups.

The newsgroups for binary postings are named roughly by general subject. Newsgroups ending with .d are newsgroups set aside for text discussion of the pictures posted to other newsgroups. i.e. alt.binaries.pictures.d (Posting of picture requests and general discussion to the non-.d groups is frowned upon.)

Basic TIN commands

In order to use TIN effectively to locate and decode binary files, you need to learn a few of its basic commands so that you can navigate USENET with TIN effectively. Fortunately, TIN is nice enough to provide us with a short menu of some of the most commonly used commands. The highlited line indicates the current newsgroup or posting that is selected. You can move down the list of newsgroups or list of postings line-by-line with the DOWN ARROW key or the j command. Similarly, you can move up line-by line with the UP ARROW or the k command. Use CTRL-U to move up a page, and CTRL-D to move down a page. Pressing return open the currently highlited newsgroup or displays the currently highlited posting. $ will take you to the bottom of the newsgroups (where the most recent postings are located). Hitting 1 followed by return will take you to the first posting in the newsgroup. If you are reading a thread of postings (postings on the same subject), spacebar will page you through the postings in that thread, and at the end of the thread, it will take you to the next subject header and any thread under it.

q takes you "up a level". For instance, if you are reading a posting, q takes you back to the list of posting headers. Hitting q again takes you back to the list of newsgroups. If you hit q again, you will leave TIN. Q will bail you out of TIN no matter where you are. You get a "cheat sheet" of these and other commands simply by hitting h anywhere in TIN.

To jump to a specific newsgroup, use the g command. After you hit g, TIN will prompt you for the name of the newsgroup you want to jump to. Enter the name of the newsgroup, and press the return key. TIN will move you to the name of the newsgroup you selected. Let's give it a try. Say you want to jump to the newsgroup alt.binaries.pictures.misc. To jump to that group, press g, and TIN will give you a prompt that will look much something like this:

Goto newsgroup []>

Now, type alt.binaries.pictures.misc and hit return, like so:

Goto newsgroup []> alt.binaries.pictures.misc

After you press return, you should get a screen like this:

                     Group Selection (news.oit.unc.edu  4654)     h=help

      65   259  alt.binaries.pictures.erotica.female                           
      66    18  alt.binaries.pictures.fractals                                 
      67        alt.binaries.pictures.furniture                                
      68   113  alt.binaries.pictures.misc                                     
      69  3498  alt.binaries.pictures.supermodels                              
      70    71  alt.binaries.pictures.tasteless                                
      71   174  alt.binaries.pictures.utilities                                
      72        alt.binaries.sounds-armpit.noises                              
      73     4  alt.binaries.sounds.d                                          
      74    16  alt.binaries.sounds.erotica                                    
      75    42  alt.binaries.sounds.misc                                       
      76     2  alt.birthright                                                 
      77     5  alt.bitch.pork                                                 
      78    75  alt.bitterness                                                 
      79     5  alt.bogus.group                                                
      80     9  alt.bonsai                                                     

       <n>=set current to n, TAB=next unread, /=search pattern, c)atchup,
    g)oto, j=line down, k=line up, h)elp, m)ove, q)uit, r=toggle all/unread,
      s)ubscribe, S)ub pattern, u)nsubscribe, U)nsub pattern, y)ank in/out

Hit return to enter the newsgroup.

Decoding binaries

Before we can start decoding pictures, a discussion of the UU coding scheme is necessary. Binary files are composed of 8-bit data. However, USENET is designed to handle only 7-bit text data. How in the world, you may be wondering, can 8-bit binary files be transported over USENET? The answer is by encoding binary files into a 7-bit format that can be handled by USENET. The most popular and ubiquitous scheme used on USENET is the UU standard. In a nutshell, under the UU standard, 3 bytes of binary data are split up into 4 groups of 6 bits and then each group is "stuffed" with a value to turn it into an ASCII character. The encoded bit groups are then arranged on lines with a character at the beginning of the line indicative of the number of bit groups on that line and, depending on the encoder, a checksum at the end of the line. Binary files that have been UUENCODED are roughly 1/3 larger than they are in binary format. However, UUENCODED data can be handled by USENET like ordinary text data.

Because many binary files can be rather large, by convention large binary files are split into separate parts and posted as separate messages. Some older news servers can only handle individual messages that are no larger than 60K in length. Thus, by splitting larger binary files into smaller chunks, users of these servers can access the files. These older servers are gradually disappearing from the Internet, but the convention persists nevertheless, in part because it is easier for news clients to handle shorter files rather than large, massive ones. With the growing volume of multimedia postings, the size and number of separate parts is increasing. It is not uncommon to see postings of especially large binary files in excess of 50 parts in 150-200K chunks. This has led to substantial fragmentation of many larger binary postings (posting out-of-order, some parts not making it to the site, or not making it after the other parts have been purged). This is mainly a problem in the multimedia groups because of the large size of the files. This problem is likely to persist for some time, at least until methods of handling and transporting these large files are developed.

Descriptions of binary postings can often be found in files with a 0 in the part of the message subject containing the number of sections of the file. (i.e. (0/2)). Other descriptions can be found in the first section of the binary file. (A word of caution: BEWARE of postings with no description! You may not like what you see or hear!)

Ok, jump to and open up the group alt.binaries.pictures.misc using the method outlined above. You should then see a screen that looks somewhat like this:

                alt.binaries.pictures.misc (101T 112A 0K 0H R)          h=help


     1  +    REQ:Linda Wong pictures?                        Hitech Engineering
     2  +    How do I download these pictures                Boxtruck@aol.com  
     3  +    JAGGER 1965.JPG [1/1]                           Cody Jarrett      
     4  +    CHARLIE WATTS: 1965.JPG [1/1]                   Cody Jarrett      
     5  +    BILL WYMAN: 1965.JPG [1/1]                      Cody Jarrett      
     6  +    BRIAN JONES 1964.JPG [1/1]                      Cody Jarrett      
     7  +    MICK AND KEITH 1964.JPG [1/1]                   Cody Jarrett      
     8  +    Railroad roundhouses                            Jeffrey Williams  
     9  +    PLEASE READ>>>>>>>>IMPORTANT!!!!!!!!!!!!        Douglas Jackson   
    10  + 2  Does anyone know this guy? (1/1)                Ari Fisher (BIO)  
    11  + 4  REQ:PLEASE ZIP BINARIES BEFORE POSTING          solarfox@i-link.ne
    12  +    Don't: (was Re: REQ:PLEASE ZIP BINARIES BEFOR   Syed Zaeem Hosain 
    13  +    00296.jpg [1/1] (12hr)                          [Brad Brace]      
    14  +    Need Suggestion for an animator.                Lord Tilden Owsen 
    15  + 2  Public Apology (ie Scanning Service Posting)    Illya Kuryakin    
    16  +    HK Idols?                                       Frederick Wong    


      <n>=set current to n, TAB=next unread, /=search pattern, ^K)ill/select,
    a)uthor search, c)atchup, j=line down, k=line up, K=mark read, l)ist thread,
      |=pipe, m)ail, o=print, q)uit, r=toggle all/unread, s)ave, t)ag, w=post

Using the commands listed above, move through the newsgroup and look for a picture that you might like to view. For example (and just as an example), suppose you a posting containing an encoded rendering of a water molecule that you would like to download and view,

                alt.binaries.pictures.misc (101T 112A 0K 0H R)          h=help


    49  +    Art - hass-02.png 03/03 Childe Hassam           Harry Davis       
    50  +    00298.jpg [1/1] (12hr)                          [Brad Brace]      
    51  +    - FAX_forms.sea [00/02] FAX forms ready to fi   Pinochle          
    52  +    - FAX_forms.sea [01/02] FAX forms ready to fi   Pinochle          
    53  +    - FAX_forms.sea [02/02] FAX forms ready to fi   Pinochle          
    54  +    REQ: Carl Orff                                  TreborK910        
    55  +    aaa                                             mross@oregon.uoreg
    56  + 2  test                                            Nicholas Hollins  
    57  +    BOO                                             Nicholas Hollins  
    58  +    SEE THE PICTURES - INSTRUCTIONS  (PC)  -  FRE   seepics           
    59  +    Terry Hatcher                                   Mark Wilkes       
    60  +    REQ:Rush Album Covers                           Fromm  Richard W  
    61  + 3  REQ: Giger Pix                                  Chris Nituch      
    62  +    Rendering of a water molecule (1/2)             John Allen Marsh  
    63  +    Rendering of a water molecule (2/2)             John Allen Marsh  
    64  +    Rendering of a water molecule (0/2)             John Allen Marsh  

      <n>=set current to n, TAB=next unread, /=search pattern, ^K)ill/select,
    a)uthor search, c)atchup, j=line down, k=line up, K=mark read, l)ist thread,
      |=pipe, m)ail, o=print, q)uit, r=toggle all/unread, s)ave, t)ag, w=post

Notice that this picture has a description file associated with it (the posting with "(0/2)" in its subject field). If you want to read the description of this graphics file, highlight it and press return. The "(x/2)" in the subject header indicates the number of parts to the graphics file and the part posted in that particular posting. "(1/2)" indicates that it is the first part of a two part file, and "(2/2)" indicates that it is the second part of a two part file.

In order to decode this graphics file, we must "tag" all of its parts in order. Highlight the first part of the file (usually indicated by "(1/x)" in the subject field) and hit t for to tag the posting. After you hit t, the cursor moves the the next posting. Hit t again if the posting is the next part of the file, otherwise reposition the cursor before hitting t again. Once you have selected all parts of the file, your screen should look something like this:

               alt.binaries.pictures.misc (101T 112A 0K 0H R)          h=help


    49  +    Art - hass-02.png 03/03 Childe Hassam           Harry Davis       
    50  +    00298.jpg [1/1] (12hr)                          [Brad Brace]      
    51  +    - FAX_forms.sea [00/02] FAX forms ready to fi   Pinochle          
    52  +    - FAX_forms.sea [01/02] FAX forms ready to fi   Pinochle          
    53  +    - FAX_forms.sea [02/02] FAX forms ready to fi   Pinochle          
    54  +    REQ: Carl Orff                                  TreborK910        
    55  +    aaa                                             mross@oregon.uoreg
    56  + 2  test                                            Nicholas Hollins  
    57  +    BOO                                             Nicholas Hollins  
    58  +    SEE THE PICTURES - INSTRUCTIONS  (PC)  -  FRE   seepics           
    59  +    Terry Hatcher                                   Mark Wilkes       
    60  +    REQ:Rush Album Covers                           Fromm  Richard W  
    61  + 3  REQ: Giger Pix                                  Chris Nituch      
    62  1    Rendering of a water molecule (1/2)             John Allen Marsh  
    63  2    Rendering of a water molecule (2/2)             John Allen Marsh  
    64  +    Rendering of a water molecule (0/2)             John Allen Marsh  

      <n>=set current to n, TAB=next unread, /=search pattern, ^K)ill/select,
    a)uthor search, c)atchup, j=line down, k=line up, K=mark read, l)ist thread,
      |=pipe, m)ail, o=print, q)uit, r=toggle all/unread, s)ave, t)ag, w=post

Now that you have tagged all parts of the file, it's time to decode the file. Hit s to begin the decoding process. After you hit s, TIN will give you the following prompt:

Save a)rticle, t)hread, h)ot, p)attern, T)agged articles, q)uit:

You want to save all the tagged articles, so hit T. TIN will then prompt you for a "save filename".

Save filename []>

This prompt is somewhat misleading. The filename of the graphics file we want to download is included as part of the UUENCODED data, and it will be extracted and used when we finally decode the file into binary format. What TIN wants to know is what you want to call the file that will contain the raw UUENCODED data. Since you will not need the UUENCODED data after you have decoded the binary, it really does not matter what file you save it to. For lack of something better, lets have TIN save it to a file called picture.uue. Now, remember earlier when we created the temporary directory? Now, its time to press it into service. If you just enter picture.uue, TIN will save the UUENCODED data and the decoded binary to a subdirectory called News in your home directory. As noted above, you could very quickly use up your disk quota by saving binaries to that subdirectory, so we need to tell TIN to save it in our temporary directory that we created earlier. To do this, we need to enter the full pathlist of your temporary directory in addition to the filename. Using the example from above, the full pathlist of the temporary astudent directory is /scratch/astudent. Accordingly, the full pathlist of the temporary file we want to create would be /scratch/astudent/picture.uue. Enter that at the filename prompt and press return, like so:

Save filename []> /scratch/astudent/picture.uue

Next, TIN will ask you how it should process the file:

Process n)one, s)har, u)ud, l)ist zoo, e)xt zoo, L)ist zip, E)xt zip, q)uit:

Since we want to UUDECODE, hit u. TIN will then procede to save the encoded parts and then decode the file.

Saving...
Saving...1
Saving...2

-- Saved pattern to picture.uue.01 - picture.uue.02 --

Post processing...
Uudecoding...

TIN will then ask you if you want to delete the files containing the UUENCODED data:

Delete saved files that have been post processed? (y/n):

Unless you have something particular in mind that you want to do with them, just answer y. The decoded binary file should be in your temporary subdirectory.

Repeat this method for each file you want to decode.

Downloading the decoded binary files

Ok, you've decoded your binary files and you are ready to download them. If you have not already done so, leave TIN by hitting Q. To make sure all of your files are present, enter

ll

at the command prompt. ll gives you a long listing of the files in your current working directory. Assuming your files are there, we are ready to begin downloading.

Downloading over the dialup lines

To download your binary files to your home computer over the dialup lines, you need to use the Zmodem file transfer protocol. Using Zmodem is easy. To send an individual file to your home computer, use the following command:

sz -e filename

replacing filename with the name of the file you want to download. To download all of the files in your current working directory, enter

sz -e *

If your terminal program is working correctly, sz should trigger it to begin receiving the file. Most terminal programs display a box showing the progress of the file transfer. If for some reason Zmodem aborts the file transfer because of too many errors, you can usually just enter the appropriate szcommand again and the file transfer will usually, but not always, pick up where it left off. When the file transfer is complete, be sure to delete the binaries from your temporary directory in order to free up space for other users:

rm filename

to delete an individual file, or

rm *

to DELETE ALL FILES IN YOUR CURRENT WORKING DIRECTORY. BE EXTREMELY CAREFUL WHEN USING THIS COMMAND. ALWAYS GET A DIRECTORY LISTING BEFORE EXECUTING THIS COMMAND TO MAKE SURE YOU ARE NOT DELETING ANY FILE YOU WANT TO KEEP. YOU HAVE BEEN WARNED!!! Remeber, the /scratch filesystem is subject to being purged at any time, so do not leave anything in your temporary directory that you may want to keep. Either download it to your system or copy it to your home directory.

After you have downloaded your binaries to your home machine, you may do with them what you will. If you have downloaded a compressed archive containing a program, use an extraction utility like PKUNZIP to decompress the archive. If you have downloaded a graphics file, sound file, or a multimedia file, you will need an appropriate helper application. Paramount's Voyager site has a good list of basic helper applications. Select here to look over the list and get the helper applications you need. Once you download your helper applications, you can use sz to download them to your computer and then rm to delete them from ISIS after the download is complete.

Downloading from the on-campus computer labs

The PCs on campus have a MS-Windows based FTP client that can be used to download your binaries either to the local hard drive or to your floppy disk. Open the Network Services applications group and click on the FTP client. Connect to and login into ISIS. Once you are logged in, change your directory to your temporary directory. Your decoded binaries should appear as icons. Grab each of them with your mouse and drag them to the FTP client windows that represents either the local hard drive (C:\>) or your floppy disk (A:\>). Be sure to delete your binary files off of ISIS after you are finished with them.

As noted above, the on-campus PCs have Lview3.1, an excellent freeware graphics viewer, installed on them as well as WPLANY, a no-frills sound player. They should be in the same directory as Netscape on the network drive. You should be able to find it with the Windows File Manager. You can use Lview to view .JPGs, .GIFs, .BMPs, etc. and to listen to .WAVs and .AU's. As of the time of the writing of this HOWTO, no multimedia viewing files were installed on the PCs or the Macs, so viewing multimedia files from the on-campus machines is not presently possible.

Courtesy and consideration

Downloading binaries can be a time-consuming and resource-consuming endeavor. If you are downloading over the dialup lines, try to restrict your downloading to the late evening and early morning hours. If you are downloading from the on-campus computer labs, try to pick times when demand for computers is low. Remember, students who needs computers to write term papers or seminar papers ALWAYS have priority over recreational users.

If you find this page informative and useful, PLEASE drop me an email. Since I do not have a page counter, I have no other of knowing how much this page is accessed.

Have fun!

J. William Snyder, Jr. ()
Last Modified: May 20, 2005

Back to the Will's Home Page