.tr ~
.ds u \s-2UNIX\s+2
.de HX
.ps 10
..
.DS 2
.B
.S +2
UNIX Networking at Purdue
.R
.S -2
.DE
.H 1 "Introduction"
This paper describes a modest \*u network in use and developed
at Purdue University School of Electrical Engineering.
Features include:
.BL
.LI
Virtual terminal access.
Through the program \fBcon\fR a user can connect his physical terminal to a
\fIpseudo-terminal\fR on any other machine.
A virtual terminal protocol provides local/remote echoing by
propagation of \fBstty\fR/\fBgtty\fR functions.
.LI
Remote process execution.
The program \fBcsh\fR (for "connected shell") takes as arguments a machine name
and a string of one or more commands.
These commands then execute on the specified machine but with standard input
and output redirected to the local machine.
.LI
File transfer/remote device access.
\fBcsh\fR in conjunction with \fBcat\fR or \fBcpio\fR allows simple file or
directory transfers between machines.
.LI
User programmed net I/O.
The above programs \fBcon\fR and \fBcsh\fR access the net through a pool of
\*u \fIspecial files\fR.
By issuing "set teletype" (\fBstty\fR) functions on such an open file, programs
can directly connect to any machine in the network, disconnect,
wait for connection, send signals, and so on.
All of these capabilities are available to any user-written program.
.LI
Moderate bandwidth.
Measurements have shown 400K baud process to process within the same
machine and 250K baud process to process between two machines
separated by a DMC11 network link.
At this rate single block disk latency limits any further improvement
for file-file transfers.
.LI
Low system overhead.
Minimal internal buffering and copying is done.
Packets contain 512 bytes of user data.
The net buffer pool resides outside of kernel D space.
.LI
Generalized machine interconnection.
There are no master-slave relationships.
Up to 256 machines can be connected arbitrarily using any combination
of DMC11 and DA11-B network links.
Link device drivers are reentrant (code is shared over multiple
units).
Packet routing is determined by a simple static table in each
machine at sysgen time.
.LE
.cs R 30
.lg 0
.DS
.nf
.S -2

                                           host
              host                          imp
               |                             |
               |                             |     host
        host__imp_____imp__host             imp____imp
                       |                   host     |
                       |                            |
      (fig A)         host            (fig B)      imp
                                                   host

.fi
.S +2
.DE
.lg
.cs R
.H 1 "Machine interconnection"
In an ideal local network, the intermachine packet switching and routing
would take place in a
.I
subnetwork
.R
of highly reliable, single function processors (interface message
processors, \fIimp\fRs in ARPA-NET terminology), which connect
through physically short interfaces with the "real" machines of the
network (\fIhost\fRs in ARPA-NET).
The imps themselves are then connected through longer links to form
the backbone or subnetwork (fig A).
Some recent subnet designs are using loop or broadcast techniques
for the inter-imp links.
.P
Due to our limited budget and the time it would take to assemble a
working subnetwork, we have moved the imp function inside of each
host, at a cost of added complexity and reduced network reliability (fig B).
.H 1 "Special files"
Programs such as \fBcon\fR and \fBcsh\fR, their
\fIservers\fR (their counterparts in
the other host which wait for a connection requesting service),
and user-written software, all run in \fIuser address space\fR as normal
processes.
They read and write from network \fIspecial files\fR which are connected to
a special file and process in another host.
.P
The special files have names /dev/mx/A, /dev/mx/B, /dev/mx/C, and so on.
\fImx\fR stands for multiplex device and the last letter in each filename
just selects a different \fIminor\fR device of the \fImajor\fR mx device.
The different file names comprise a \fIpool\fR of network files;
the actual file name used has no significance.
.P
Before a process can do any network I/O, it must find a free mx file
from the pool by calling \fBmxfile\fR (see below).
At this point \fBstty\fR's may be issued to for example \fIconnect\fR
or \fIwait\fR for
a connection;
once the connection is open, data written at one end may be read at
the other and vice versa.
.H 1 "Connection names, host and socket numbers"
Hosts are identified by a one byte \fIhost number\fR.
Within a given host many processes may wish to hold network conversations
simultaneously so a unique inter-host connection naming convention
is needed.
A connection between a process in the \fIlocal host\fR and a process in a
\fIforeign host\fR is uniquely identified by four numbers, each one byte long:
\fIlocal host, local socket, foreign host,\fR and \fIforeign socket\fR.
The \fIsocket number\fR can be thought of as specifying a particular process
or service within the given host.
.P
A subset of the socket numbers at each host are reserved for
connections to system provided services.
For example a process called the \fIcon server\fR (started initally after
the system is booted) is always waiting for connections to local
socket number 1.
When a connection arrives the server \fBfork\fRs a child process to handle this
particular request while the parent opens a new mx file and goes back
to waiting for further socket 1 connects.
The child now reads and writes from the connected file
(at the other end is the user-initiated
\fBcon\fR process) and behaves as a
pseudo-terminal for the virtual terminal protocol.
Socket number 2 is used similarly by the \fBcsh\fR server.
.P
Note that many connections can exist to the same local host/socket
pair, since the entire 4 byte \fIconnection name\fR is still unique.
This is analogous to phone company PBX service where many incoming
calls can be in progress to the same number.
Note also that this naming convention eliminates the need for a
clumsy \fIinitial connection protocol\fR.
.H 1 "User callable function library"
A library is available to C programs for performing various network
functions.
The library can be loaded with a program by using the \%"-lmx" switch of
the C compiler (\fBcc\fR) command.
Most of the functions just reformat the arguments into the appropriate
\fBstty\fR call on the open network file descriptor supplied as
the first argument.
For completeness the resulting system call which the function makes is
shown in brackets; however users need not be concerned with
this level of detail.
The functions return -1 on failure.
.VL 5
.LI
\fIfd\fR = \fBmxfile\fR();  [\fIfd\fR = \fBopen\fR("\fI/dev/mx/x\fR",2);]
.br
A free network file from the pool "\fI/dev/mx/x\fR", is found and opened.
Its file descriptor (\fIfd\fR) is returned.
A valid network file descriptor must be supplied to most of the
functions listed below.
No reads or writes can be performed on this \fIfd\fR until a connection
is made.
.LI
\fBmxwait\fR(\fIfd,socket\fR);  [\fBstty\fR(\fIfd\fR,{3,\fIsocket\fR,0});]
.br
Wait for a connection to local socket number \fIsocket\fR from \fIany\fR host
in the network.
When the connection arrives, \fBmxwait\fR returns.
Socket numbers 1 through 5 are reserved for super-user owned servers.
.LI
\fBmxcon\fR(\fIfd,host,socket\fR);  [\fBstty\fR(\fIfd\fR,{1,\fIhost,socket\fR});]
.br
Connect the local network file \fIfd\fR to foreign \fIhost\fR, foreign
\fIsocket\fR.
When the connection is complete, \fBmxcon\fR returns.
The connection is completed if a process at the foreign host has issued
the corresponding \fBmxwait\fR, or issues it at some time in the future.
The local socket number is picked random from those
unallocated at the local host.
If no path through the network currently exists to \fIhost\fR,
\fBmxcon\fR fails immediately.
The standard \*u \fBalarm\fR clock signal can be used to timeout \fBmxcon\fR
or \fBmxwait\fR, if this is desired.
.LI
\fBmxscon\fR(\fIfd,"hostname",socket\fR);
.br
This call is identical to \fBmxcon\fR, except that here a string rather than
an integer is used for the host.
A table of standard host names to numbers is searched and \fBmxcon\fR called
with the result.
\fBmxscon\fR fails if \fIhostname\fR is not in its table.
.LI
\fBmxdis\fR(\fIfd\fR);  [\fBstty\fR(\fIfd\fR,{2,0,0});]
.br
The connected file \fIfd\fR is disconnected; further reads or writes are
now ignored.
A new \fBmxcon\fR or \fBmxwait\fR can now be issued on this \fIfd\fR.
A disconnect occurs automatically on a \fBclose\fR(\fIfd\fR) or
if the process exits.
.LI
\fBread\fR(\fIfd,buffer,count\fR);  \fBwrite\fR(\fIfd,buffer,count\fR);
.br
Read and write work normally but note the following:
(1) Read never returns more bytes than were written, and never more than
512 bytes (the host-host packet size).
In other words the read returns when ANY data is available;
it does not wait for the buffer to fill.
(2) So that 1-byte writes (which are pretty common in \*u) are not
horribly slow, they are buffered locally inside the kernel until
the packet is filled or a timer goes off.
.LI
\fBmxsig\fR(\fIfd,signal\fR);  [\fBstty\fR(\fIfd\fR,{6,\fIsignal\fR,0});]
.br
The process at the other end of the connection is sent a \*u \fBsignal\fR
number \fIsignal\fR.
See the \fI\*u programmers manual\fR section \fBsignal\fR(II).
.LI
\fBmxpgrp\fR(\fIfd\fR);  [\fBstty\fR(\fIfd\fR,{5,0,0});]
.br
The current process and all its children by \fBfork\fRs are placed in the
same unique \fIprocess group\fR.
If a signal (\fBmxsig\fR) is received over the net, all the members of the
process group receive it.
.LI
\fBmxeof\fR(\fIfd\fR);  [\fBstty\fR(\fIfd\fR,{7,0,0});]
.br
An end of file (EOF) is written onto the connected file \fIfd\fR, such that
all \fBreads\fR issued at the other end will now return a zero byte count
(EOF).
Further \fBwrites\fR at this end are ignored.
.LI
\fBmxserve\fR(\fIsocket\fR);
.br
This call is used mainly by the con and csh servers to answer
requests for service on \fIsocket\fR.
The basic actions are:
.B
.nf


   close all file descriptors
   while(1) {
      mxfile();               /* \fIfd\fB will = 0 */
      mxwait(0,socket);
      fork();                 /* split into 2 processes */
      if(child process) {
	 dup(0);  dup(0);     /* \fIfd\fB 0,1,2 now net file */
         return;
      }
      if(parent process) {
         close(0);
      }
   }
.fi

.R
.LI
Since most \*u processes do input from file descriptor 0 (standard
input) and output to file descriptor 1 (standard output),
such a process if \fBexec\fRed after calling \fBmxserve\fR
will do its I/O over
the network.
.LE
.cs R 30
.cs I 30
.lg 0
.DS
.S -2
.nf
          ___________________________
          |network application progs|
          |con, csh servers         |      user space
          |=========================|      =======
          |mx device driver         |      kernel space
          |\fI/dev/mx/x\fR special files  |
          |_ _ _ _ _ _ _ _ _ _ _ _ _|
          |net interface control    |
          |(host_host protocol)     |
          |_________________________|
          |      |__________________|
          | imp  | line driver da   |___>
          |      |__________________|
          |      | line driver dmc  |___>  to other machines
          |      |__________________|
(fig C)   |      | line driver dmc  |___>
          ___________________________
.fi
.S +2
.DE
.lg
.cs R
.cs I
.H 1 "Overview of internal organization"
The network software in each machine is split into two parts (fig C):
.VL 5
.LI IMP
The "imp" process receives buffers (packets) from the local host and
from neighboring imps.
The imp examines the destination address on each packet it receives and
looks up the host number in its (static) routing table.
This table maps host numbers to external line (link) numbers.
The packet is then enqueued for output via the appropriate line driver
and the driver is activated if idle.
A special routing table entry marks the local host number.
Packets for the local host are passed directly via a subroutine call
to the mx driver.
.LI MX
The mx device (\fI/dev/mx/x\fR) appears as a conventional device driver to
\*u.
\fBOpen, close, read, write,\fR and \fBstty\fR
calls on mx files pass control
to the mx driver in kernel space.
The driver generates packets containing host to host protocol and
passes these to the imp for delivery.
Host to host packets from foreign hosts are received from the imp
and decoded.
.LE
.H 2 "Host to host protocol"
Hosts communicate with each other by exchanging packets containing
\fIhost to host protocol\fR.
The structure of a host to host packet is shown below (fig D).
The first four bytes can be seen as the \fIconnection name\fR.
\fIbyte_count\fR contains the number of bytes in the array \fIdata[]\fR,
if any.
The maximum number of bytes a packet can contain is 512, which
is the same size as a disk block.
.DF
.nf
.S -2
.I

               struct packet {
                  char   source_host;
                  char   source_socket;
                  char   dest_host;
                  char   dest_socket;
                  char   host_control;              (fig D)
                  char   imp_control;
                  int    byte_count;
                  char   data[512];
               }

.S +2
.R
.fi
.DE
.P
The imp and host \fIcontrol\fR fields are usually zero,
in which case this is
a packet of \fIbyte_count\fR bytes of user data from the source host/socket
to the destination host/socket given in the header.
.P
The \fIhost_control\fR field when non-zero is a small integer op-code which
represents a host-host \fIcontrol message\fR.
Below are listed the op-code mnemonics and the function a packet
with that op-code performs.
.I
.VL 7
.LI con
connect.
.R
This packet requests that the connection named in the first four bytes
be established.
The connection is open when a pair of these is exchanged, one in each
direction.
After the connection is open, data packets may then be sent.
If the host receiving this request has a process with a matching
\fBmxwait\fR(\fIfd\fR,\fIsocket\fR) pending,
the matching \fIcon\fR is sent and the connection
is open.
If no one is currently listening (\fBmxwait\fR pending) to that socket at the
receiver, the \fIcon\fR is remembered but no reply is sent.
An \fBmxwait\fR then issued some time
later will find this remembered \fIcon\fR
and open the connection by replying.
.LI ~
Once \fIcon\fR has been sent, that connection name must not be used again
until disconnects (\fIdis\fR, below) are exchanged.
If \fIcon\fR is sent and no matching \fIcon\fR returns, the user or process
which issued the \fBmxcon\fR (causing \fIcon\fR
to be sent) will typically time
out and \fBclose\fR or \fBmxdis\fR
the offending mx file, which causes \fIdis\fR to be sent.*
.FS *
Rather than using a giant ARPA-NET-style finite-state machine to record
the state of connection establishment, it was found sufficient to just
store a bit vector: (\fIcon sent, con received, dis sent, dis received\fR)
in each host per connection, and just make tests on combinations
of these bits.
.FE
.I
.LI dis
disconnect.
.R
This requests that the connection named in the first four bytes
be broken.
The disconnect is complete when a pair of these are exchanged.
.I
.LI next
ready for next packet.
.R
After a data packet is sent and before more data can
be sent, this op-code must be received.
The \fInext\fR is sent by the consumer of the data packet when the
user process at that end reads the data from kernel into user space.\(dg
.FS \(dg
This flow-control scheme is crude and is the reason
we can only get 250K baud out of our 1M baud DMC11;
but it's simple and reasonably effective due to the large
packet size used.
Of course the remaining 750K bandwidth is available for several other
connections doing simultaneous full-speed transfers.
.FE
.I
.LI sig
signal.
.R
Sends interrupt signal number in \fIdata[0]\fR, to the process at the other
end of the connection.
Note that this bypasses the queued data-packet stream.
.I
.LI rst
reset.
.R
The source and dest socket numbers are not used.
The source host is telling the dest host to clear out and reset all
known connections between the two.
This is "broadcast" by each host when it comes up (normally
or after a crash) to clean up loose ends.
.I
.LI rrp
reset reply.
.R
Reply to \fIrst\fR.
Not currently implemented.
.LE
.H 2 "Imp-host protocol"
Rather than adding another independent layer of protocol on top of
host-host, a single byte is stolen from the host-host packet header,
and when non-zero indicates an imp to host or host to imp control
op-code.
There is currently only one op-code and it is imp to host:
.I
.VL 7
.LI dead
dead host.
.R
This packet is being returned because there is no path to the host it
was sent to.
The imp that discovered this fact simply reversed the source/dest fields,
set this op-code in the header, and requeued the packet for
transmission.
The second time a packet bounces off an imp like this, it is destroyed,
for both the source and dest host are then dead.
.LE
.H 2 "Imp-imp protocol"
There is no single imp-imp protocol at present but rather a different
protocol used by the line device drivers for each type of physical
link.
There is a more or less common type of imp-imp header that is
prefixed to the host-host packet header.
It contains a start of header (SOH) or flag character, a sequence
number, and an optional checksum.
Currently DMC11 and DA11-B line drivers are supported,
a low speed serial driver will be available in the near future.
.H 1 "Imp process / line driver interaction"
The imp process is activated whenever a packet arrives at the imp,
from either the local host or from neighboring imps.
The packets are placed on the imp's single \fItask\fR (or work) queue
and the imp activated by a level 1 \fIprogram interrupt request\fR.
The imp: (1) dequeues packets one at a time;  (2) examines their
destination;  (3) checks that the appropriate line is "alive";
(4) places them on that line driver work queue; and (5) activates
the driver.
.P
Each line driver has a standard set of entry points:
.VL 9
.LI TPACK
A packet has been placed on the drivers work queue.
.LI TTIMER
Each driver is entered here 10 times per second for use in
timeouts and in determining if the physical link is "alive".
.LI TSTART
Entered at network start-up time to handle preset details:
initial buffer allocation, csr setup, etc.
.LI \fI(inter)\fR
Each driver has one or two \fIinterrupt\fR entry points from the
vectors (in low.s).
Packets received for the imp at this level are passed to the imp
through his task queue.
.LE
.P
The \fIunit\fR number is an argument in all of these entry points to allow
the driver code to be shared over multiple physical link devices of
the same type.
.H 2 "DMC11 line driver"
The DMC microcomputer does most of the low level work.
Some code is still
required for:
.BL
.LI
sequence number and general consistency checking.
.LI
idle packets.
Idle packets are used to detect the line coming alive.
.LE
.P
Although not serious, a number of minor bugs have been found in the
DMC hardware:
On rare occasions the DMC will loose a buffer with no error status.
Weird traps have occured on one cpu when the cpu at
the other end is halted (perhaps in the middle of a DMC dma).
Defective devices on the unibus which hog the bus for too long
cause problems.
The sequence and timing of commands to first-time initialize the
DMC were found to be critical.
Internal \*u errors which lockout interrupts during /dev/tty8 (console)
error messages make the DMC sick.
(We have improved error message handling to avoid most of these lockouts).
The DMC spurriously issues RDYO interrupts (with RDYO bit clear) when
under heavy load.
.P
Although the above may sound discouraging,
now that we have the DMC up and running
we are very pleased with its performance and reliability.
.H 2 "DA11-B line driver"
The imp-imp links are logically full-duplex channels with no master-slave
relationship.
Since the DA is half-duplex, code had to be provided to give the
physical link this logical appearance.
Both sides must cooperate at the beginning and end of each packet transfer.
.P
Each side is always in one of 4 states:  (1) interrupts off (for about
a second after an INIT pulse is detected, to prevent continuous
interrupts).
(2) idle.  (3) receive.  (4) transmit.
Each side remains in idle state until either:
(1) the local side gets a packet to send from its imp, in which case
the local state goes from idle to transmit while
a dma transmit is started and a \fItransmit go\fR interrupt sent to
the other side.
(2) a \fItransmit go\fR interrupt is received from the other side causing
this side to go from idle to receive and start a receive dma.
.P
Along with the \fItransmit go\fR signal the word count is sent;
this allows the receiver to setup his dma word count properly.
The transmitter actually sets his dma count one word higher (in fact
2 words higher for obscure reasons) so that the receive side always
services his interrupt first upon transfer completion.
Upon completion the receiver decides if he now wants to transmit
a packet (perhaps queued during the receive transfer).
If so the receiver moves directly to transmit state (sending a
\fItransmit go\fR to the previous transmitter), if not receive moves to
idle, sending a completion interrupt to the previous transmitter.
This scheme allows packets to flow alternately in each direction
if transmit queues exist on both sides.
.P
Of course occasionally both sides will decide to transmit at the same
time.
Since the hardware does not contain any arbitration logic for this case,
the software must detect this state and correct it.
At sysgen time one side is declared
\fIsecondary\fR and the other \fIprimary\fR.
In the event of a collision,
the hardware hang is cleared and the \fIprimary\fR
allowed to go first.
.P
Sequence numbers are used to detect missing blocks (from gross hardware
faults), but no checksum is used since the physical distance involved
is less than 100 feet.
Timeouts are used to send null blocks every second for line alive/dead
determination.
Collisions are detected and corrected immediately, but rarely the
link freezes unexplicably (a cryptic comment concerning this was
discovered in a DEC written DA11 driver), so another timeout
trys to thaw things out.
.P
Although the DA11-B is now being used, we think half-duplex links
are generally undesirable and would recommend the DMC over the
DA when buying new equipment.
Only when you happen to have an DA already (as we did) should this link
driver be used.
.H 1 "Virtual terminal protocol (\fRcon\fB program)"
When \fBcon\fR is in use there are actually
4 processes and a \fIpseudo terminal\fR
involved:
.br
.cs R 30
.lg 0
.DS
.S -2
.nf

                          local host | foreign host
                                     |                        _________
keyboard  (fd0) ___> con (send) >____|____> s\(ulcon (recv) ___> |   |   |
                                     |                        |pty|tty|
screen    (fd1) <___ con (recv) <____|____< s\(ulcon (send) <___ |   |   |
                                     |                        _________
(fig E)                              |

.S +2
.fi
.DE
.lg
.cs R
.VL 9
.LI local
The \fBcon\fR at the local host is split into two parts: one reads from the
keyboard (fd0) and writes to the net, while the other reads from
the net and writes to the screen (fd1).
.LI foreign
At the foreign host the \fIcon server\fR (called \fBs\(ulcon\fR)
has also split into
two parts.  One half is reading from the net and writing to a
pseudo terminal, the other half is reading from the same pseudo
terminal and writing to the net.
.LE
.P
A \fIpseudo terminal\fR is a device driver that has two sides.
For a single pseudo terminal the sides might be named /dev/ttyx and
/dev/ptyx.
Anything written on /dev/ptyx looks like it was \fItyped in\fR at /dev/ttyx,
while everything \fIprinted out\fR at /dev/ttyx can be read at /dev/ptyx.
.P
The con server's job is very simple; when a connect arrives on socket 1,
\fBs\(ulcon\fR forks once to generate a child.
The child \fBs\(ulcon\fR then finds a free /dev/ptyx and forks into receiving
and sending parts.
.P
Certain \fIescape\fR or \fIcommand\fR character sequences are generated and
accepted by /dev/ptyx.
For example, when an "\fBstty\fR" function is issued on /dev/ttyx, it is
translated to a command sequence that is then read from /dev/ptyx.
The local \fBcon\fR program and /dev/ptyx
use this \fIvirtual terminal\fR protocol
to propagate \fBstty\fR/\fBgtty\fR's and to perform other operations.
The con server (\fBs\(ulcon\fR) only issues reads and writes, it never
intercepts the command sequences between /dev/ptyx and con.
.P
Each \fIcommand\fR consists of an escape byte called "IAC" (interpret as
command), followed by a command code byte, possibly followed by
data for that command.
The mnemonics for the commands are listed below with their function.
.VL 4
.LI ST
set teletype, followed by 6 bytes from the \fBstty\fR function.
Sent by /dev/ptyx when an \fBstty\fR is issued on /dev/ttyx.
Also sent by \fBcon\fR in response to a GT (below).
.LI GT
get teletype.
Sent by /dev/ptyx when a \fBgtty\fR is issued on /dev/ttyx.
The user process that issued the \fBgtty\fR sleeps until ST
returns from \fBcon\fR.
.LI IN
interrupt signal.
Sent by \fBcon\fR when an actual interrupt (not just rubout) is generated
by the user.
A rubout typed by the user in raw mode thus causes no problems.
.LI QU
quit signal.
.LI EF
end of file.
.LI DM
data mark.
When \fBcon\fR detects local interrupt or quit, and the IAC IN or QU sequence
is sent, a flag is set so that all data characters received by \fBcon\fR
from the net are flushed  --until this command (IAC DM) is received.
/dev/ptyx sends the data mark when IN or QU is received, to flush
out the output stream to the other host.
.LI IAC
an IAC sent as data must be doubled.
.LE
.H 1 "Remote process protocol (\fRcsh\fB program)"
The sequence of events when using csh is as follows:
.AL 1
.LI
The local \fBcsh\fR connects to socket 2 on the foreign host.
It then writes 3 lines (each '\en' terminated but as a single write,
for speed): \fIname, password,\fR and \fIcommand\fR line.
\fIName\fR and \fIpassword\fR are possibly null.
.LI
At the foreign host, \fBs\(ulcsh\fR (csh server) has received the socket 2
connection and forks a child to handle it.
.LI
The child \fBs\(ulcsh\fR has file
descriptors 0,1, and 2 (standard I/O and error)
open as the net file.
\fBs\(ulcsh\fR reads the \fIname, password,\fR and \fIcommand.\fR
If \fIname/password\fR is not null, \fBs\(ulcsh\fR "logs in"
as that user name,
else user name \fIuser\fR is the default.
.LI
\fBs\(ulcsh\fR now \fBexec\fRs in the \fIcommand\fR on top of itself.
The \fIcommand\fR will do its I/O from the net.
.LI
Meanwhile the local \fBcsh\fR has split into two parts, one
reads from standard input and writes to the net, while the
other reads from the net and writes on standard output.
When the half that is reading standard input gets an EOF, it writes
an EOF to the net and exits.
.LI
The command running at the foreign host will eventually exit.
.LI
The local \fBcsh\fR reading the net will get an EOF when the command exits
at the foreign host;  the local \fBcsh\fR then exits.
.LE
.H 1 "Kernel network buffers"
Memory management register 5 (ka5) in our \*u kernel has been freed
for use as a general window into extended disk, network, and misc. buffer
areas.
This allows a healthy supply of \fIdedicated\fR network buffers.
.P
The net buffers have no \fIbuf\fR structure associated with them (as do
system disk buffers) and are large enough to hold a packet complete
with internal, imp-imp and host-host headers.
This means user data can be moved directly to its transmission buffer
in kernel space from user space \fIonce\fR, with no further moves.
.P
The buffers are located on 64 byte (memory management) boundaries so that
they are easily referenced.
Buffers in queues are circularly linked (queues are named by their
tails), and links (which are stored in the first word) can be
plugged directly into *ka5 to access that buffer (links are the
physical memory address shifted right 6 bits).
.H 1 "Current configuration"
Our site currently has 2 11/70s and an 11/45 connected; the /70s by
a DMC11, and the /45 to one /70 with a DA11-B.
Three other /45s in the same building could be connected immediately
if we had 3 pairs of DMC11's.
Currently we are working on a low speed serial (19k baud) link
driver for these machines.
.H 1 "Availability"
The mx software is public domain and free to anyone with a version 6
\*u license.
Currently we only have ours running on split I/D machines:  11/45s,
11/70s, etc.
It is not known whether it could be squeezed into an 11/40 or /60.
Minor mods would also be needed in machines without the
programmable interrupt request hardware to perform this
with software (we were just lazy).
Perhaps more critical are the modifications we have made to the "standard"
version 6 kernel and user space programs.
.BL
.LI
We no longer have "groups".  The group byte in the inode has been
taken over by the high order bits of the 16 bit user id (we have a large
number of users).
.LI
The ka5 window is used for:  most system disk buffers, net buffers,
the proc table, the clist, and high speed (low sys time)
terminal output.
.LI
The \fIspeeds\fR word of the
\fBstty\fR/\fBgtty\fR function has been rearranged to
support various local serial I/O needs:  Full speed 19K baud
input to a circular buffer in user space.  APL overstrikes on
specially modified ADM3As.  Etc.
.LI
Powerfail recovery and retry.  Device drivers are
entered with \fIdev\fR == NODEV
following a powerfail.  99% of our outages are now automatically
recovered.
.LI
A lot of "little" things that improve throughput and system error recovery.
Most of the "PWB diff listing" kernel mods (obtained from UUG) are
installed.
.LE
.P
To run the mx code we really recommend running with our kernel as well.
Many of the mods above can be conditionally compiled out, but some
cannot (such as uid's, we didnt know of #ifdef back then).
Thus installing only "part" of our kernel will require some effort.
.P
The \*u Users Group has a copy of our latest distribution tape.
As soon as they start functioning again, requests for our
software should go though them.
.H 1 "Acknowledgements"
Most, if not all of the ideas in the mx software have been stolen from
other places:
.BL
.LI
The pool of special files controlled by \fBstty\fRs, and the remote
process execution idea are from an early 2 machine network
of Ken Thompson's.
.LI
Imp's, sockets, servers, host-host protocol, and virtual
terminal ideas are from the ARPA-NET.
.LI
Connection-naming approach is from an IFIP network proposal.
.LI
The ka5 buffer pool and various other hacks to improve the throughput
and performance of \*u are the work of George Goble, also of Purdue.
.LI
Of course without \*u, none of this would have happened.
.LE
.H 1 "For further information"
For a copy of this document, the software, or more information contact:
.nf

	Bill Croft
	Purdue University
	Electrical Engineering Department
	W. Lafayette, IN  47907

	(317) 494 7268 or 494 7266
.fi
