This is Info file isect.info, produced by Makeinfo version 1.68 from
the input file isect.texi.

INFO-DIR-SECTION Isect Programs
START-INFO-DIR-ENTRY
* isectd: (isect)
END-INFO-DIR-ENTRY

Isect Middleware How-to Copyright (C) 1999, Isect


File: isect.info,  Node: Top,  Next: Introduction,  Prev: (dir),  Up: (dir)

Isect Middleware How-to
***********************

Isect's middleware daemon's (isectd) APIs for both clients and servers
makes writing distributed programs easy.  It relieves the programmer of
having to understand all the ins and outs of network programming,
allowing them to pay closer to attention to the the application logic.

Isect programs are easy to program, debug, scale, and administer.

* Menu:

* Introduction::    Where isectd hails from
* Inventory::       The programs and what they do
* Setup::           Configuration and Setup
* Startup::         Starting and stopping isectd and isdexecd
* Sample::          A quick running of the test programs
* API::             The isd library functions
* Users::           Users Guide
* Future::          What may be inside future versions


File: isect.info,  Node: Introduction,  Next: Inventory,  Prev: Top,  Up: Top

Introduction
************

Long before the buzzwords "client-server" and "middleware" were ever
coined, programmers used middleware to build client-server systems.
They programmed using character or block IO directly to the network
devices which may have been proprietary, like Data General's MCA cards,
or standard like UARTs for serial communications.

Over TCP/IP networks, regardless whether the network interface is
token-ring, ethernet, or PPP, the socket library developed at the
University of Berkeley became the standard way for programs running on
different computers (or even the same computer) to communicate with
each other.

But writing servers is more complicated than just read()ing requests
from a client.  What if there are multiple clients?  What if the client
dies while you're still processing?  What if the server dies?  How does
the client find the server?  Should or must the server be multitasking?
How should semaphores be used?  These issues all concern network
programming and have little (read: nothing) to do with what the
programmer really needs the server to do.  In fact, for simple server
programs the overhead of the network programming may make it not worth
it to create the server in the first place.

To understand how isectd works, it is important to understand two new
concepts: services and workers.

A "worker" is what most programmers would recognize as a server.  It is
the program that actually does the application's heavy lifting.  For
banking applications it may be the transaction processor accepting
deposit, withdrawal, payment, and advance transactions, and updating
account balances in the database.  It may be also be a receipt printer,
a job scheduler, or an interface into a real-time market data system
like Reuters for price quotes or news.

A "service" is a collection of workers that all do the same thing.
Each worker is single-threaded, it has only to deal with the
transaction it's working on and won't be interrupted by another request
until its done.  By being single-threaded the code is more portable,
but more importantly it is scalable.  Multithreaded, or multitasking,
programs can scale only to the capacity of the machine they're running
on.  They can run no faster than local RAM and CPU allow.
Single-threaded programs that are not tightly bound to each other can
easily be replicated on other machines on the network.  Limited only by
the aggregate capacity of memory and processing power on all the
participating computers.

When a client connects to isectd it specifies the service it is
interested in.  Isectd takes each client requests and sends it to the
next available worker providing the identical service.  Since clients
may make asynchronous requests (not waiting for an answer for one
transaction before sending the next) isectd effectively multiplexes
their transactions across however many workers it has available
providing the service the client's requesting.

The service name is completely arbitrary and functions only as a
commonality between like-workers and the clients requesting the service
those workers provide.  In our banking application above the
transaction processor may be named `tpworker' and isectd may be
configured to start 12 of them on three different computers.
Collectively they may be configured to provide a service called
"TPSERVICE".  When the client programs login to isect they will specify
that the transactions they'll be sending are for "TPSERVICE".  Isectd
will dutifully and equitably dole them out to any available TPSERVICE
worker.


File: isect.info,  Node: Inventory,  Next: Setup,  Prev: Introduction,  Up: Top

Inventory
*********

A mostly-bare isect installation comes with four basic programs,
isdclient, isdecho, isdexecd, and the middleware daemon, isectd.

`isectd' is the process all clients connect to.  The computer it runs
on is the only one clients need to find on the network.  This is
accomplished by aliasing it's hostname in other computers' /etc/hosts
files (or by updating NIS maps) to `isectd'.

`isdexecd' is a remote execution daemon that accepts requests from
`isectd' to start programs on the computer `isdexecd' is running on.
Though `isectd' will typically run on only one computer and there will
usually be only one `isectd' process running, there will be one
`isdexecd' running on every computer that will have a server (a worker).

`isdecho' is an example worker that simply echoes back to the client
whatever it is sent.  Though apparently useless on the surface, it
provides a good sanity check to make sure the plumbing of the system is
working.  It also provides an excellent template for programmers to
write their own workers that can be arbitrarily more complex (not hard
to do).

`isdclient' is a generic isectd client that uses standard input and
output to communicate to any server that accepts and replies text
(ascii) messages.  In most cases this is both sufficient and
preferable.  `isdclient' accepts arguments specifying which host
`isectd' is running on (if it's not the default), which service it
wants to talk to, and even a command to execute.


File: isect.info,  Node: Setup,  Next: Startup,  Prev: Inventory,  Up: Top

Setup
*****

Because isectd and the remote execution deamon isdexecd communicate
with each other and other programs using TCP/IP there are a few files
typically located in the computer's /etc file that should be edited.
One is mostly mandatory and the other is optional.

First, the computer that will host the isectd process should be aliased
in other computers' /etc/hosts files.  When clients try logging on to
isectd the function will first look for a host named `isectd'.

     192.0.0.10	zeus isectd

If your site doesn't use /etc/hosts but NIS, NIS+, or DNS, contact your
network administrator if you want the default login procedure.  For
security reasons you or your administrator may decide its better for
clients to specifically address hostnames rather than making aliases
available.

The other file that must be edited is /etc/services.  This file keeps
track of which ports networking programs listen to.  Port numbers below
1024 are reserved for privileged processes.  It's important to edit
this file (or its NIS equivalent) so that system administrators can
avoid port-number conflicts with other programs.

An example from one site:
     #
     # isect services
     #
     isectd	5501/tcp	# isectd
     isdexec	5502/tcp	# isdexecd

Once the system administration tasks are complete, isectd can be
started.


File: isect.info,  Node: Startup,  Next: Sample,  Prev: Setup,  Up: Top

Startup
*******

After a successful installation the Isect programs are usually
installed in /usr/local/bin.  Hopefully, this directory is in your path.

First, the remote execution daemon, isdexecd, should be started:

     zeus:/var/tmp isdexecd &
     [2] 13822
     Thu Jul 15 11:26:18 1999
     getwd() = /var/tmp
     isdexec assigned port #5502
     isdSocket() assigned 3

isdexecd doesn't disconnect from the controlling process so its advised
to run it in the background.

isdexecd could really be run anywhere, and /var/tmp is a pretty save
place to put it.  By default, all isdexecd's children (the local
workers) will inherit its working directory.  If they abend it is good
to know where the core files are likely to show up.

When isdexecd starts up it reports the date, its working directory, and
the port it was assigned (from /etc/services).  For no really good
reason it also reports the handle of the socket it opened.

isectd is started the same way:
     zeus:/var/tmp isectd &
     [3] 13829
     service->name=op, service->serviceid=134561888
     SERVICE 'op' added.
     SystemError : 1, Operation not permitted, setrlimit()
     Unable to increase open files to 1024
     using 256 descriptors
     UNNAMED assigned port #6215
     isectd assigned port #5501

As isectd starts up it reports various environemental and operational
curiosities.  The first thing it does is install its own reserved
service called 'op'.  This is the service that accepts operator
commands for configuring and administrating isectd while its running.
As a service its unique characteristics are:

  1. The op service has no workers

  2. The op service cannot be stopped and started by the operator

In the output above, isectd complains that it can not increase the
number of file descriptors to 1024 and is using 256 instead.  This is
OK, unless of course you need 1024 or more descriptors.

The UNNAMED port is dynamically created to accept logins from the
workers that get started.  isectd includes the value of UNNAMED and its
hostname in the startup requests it sends isdexecd for workers.  The
workers know to rendezvous with isectd at the specified host and port
number.  Having two separate ports makes it easier for isectd to
distinguish client from worker connection requests.  Also, by passing
the port number to workers through isdexecd the port number doesn't
have to be reserved in /etc/services.

Finally, isectd displays the port it's waiting for client logins on.

Now that both isectd and isdexecd are running, we're ready to configure
a service and a worker.


File: isect.info,  Node: Sample,  Next: API,  Prev: Startup,  Up: Top

Sample
******

isectd has no configuration file that it automatically reads in.  Its
operator service 'op' is capable of creating its configuration and
modifying it while its running.  System administrators can configure it
using isdclient either manually (fun but tedious and error-prone) or as
part of a script.

To configure the default echo service we need to first, create a service
and second, create a worker that provides that service.  This is simple
to do using isdclient:

     zeus:/var/tmp isdclient -c 'config service echo'
     SERVICE 'echo' added.

On the console from where isectd was started, the following messages
may appear:

     login client 1
     service->name=echo, service->serviceid=134566328
     client 1
                      login seconds : 0
                       transactions : 1
              average response time : 0.000 seconds
     isdStartListening() - Have room for 249 clients

At this point all we have is a service defined.  It can't actually do
anything because we haven't configured any workers.  The next two
commands will configure a worker for the 'echo' service then start it.

     zeus:/var/tmp isdclient -c 'config worker echo localhost /usr/local/bin/isdecho'
     WORKER 'localhost:/usr/local/bin/isdecho' added.
     zeus:/var/tmp isdclient -c 'start echo'
     Ok.

On the console isectd may have displayed:

     login client 2
     worker->id=134566472, worker->serviceid=134566328
     client 2
                      login seconds : 0
                       transactions : 1
              average response time : 0.000 seconds
     isdStartListening() - Have room for 248 clients
     worker->id=134566472, worker->hostpid = 13926

What's happend so far is:
  1. We've configured a service called 'echo'

  2. We've configured a worker (the program isdecho) for the 'echo'
     service

  3. We've requested it be started

  4. isectd requested isdexecd to start the worker

  5. the worker has started (notice the worker->hostpid= message above)

Now we can test-out our new server (thought it's admittedly not a very
big test:)

     zeus:/var/tmp isdclient echo
     Now is the time
     for all good men
     to come to the aid of the country.
     go
     Now is the time
     for all good men
     to come to the aid of the country.

In the example above, instead of giving the command to isdclient to
execute immediately (-c) we went into interactive mode.  When isdclient
is invoked this way it logs on to isectd's 'echo' service and waits for
input from the user (stdin).  It buffers everything up until the user
types a 'go' on a line by itself.  Everything preceding the go is sent
to isectd and isdclient waits for a response.  In the example above the
input message was echoed back without modification.


File: isect.info,  Node: API,  Next: Users,  Prev: Sample,  Up: Top

API
***

CLIENT API

Clients connect to isectd, send messages (requests) to it, optionally
read the responses, and ultimately close their connection.

Before using any of the isd*() functions in your source code you must
first `#include "isdio.h"' to define prototypes and the required
`#define's.  In the interest of brevity the `include' statement is
omitted from the examples.

`isdio.h' is normall installed in /usr/local/include on UNIX systems by
`make install'.

* Menu:

* isdio::       isect structures and defines
* isdLogin::    Connecting to `isectd'
* isdLogout::   Disconnecting from `isectd'
* isdSend::     Sending messages
* isdRecv::     Getting messages
* isdConnect::  Connecting workers to `isectd'


File: isect.info,  Node: isdio,  Next: isdLogin,  Up: API

isdio
=====

When calling either `isdSend()' or `isdRecv()', an important parameter
is the address of a structure of type `isdHeader *'.  The definition
appears in the file `isdio.h' which should be included before calling
any of the isd* functions.  Though only two functions require the
header, the file defines the prototypes for all the calls.

isdHeader
---------

     typedef struct {
         long len;
         long sequence;
         short reply;
         short error;
         short command;
         short version;
         unsigned long workerid;
         short more;
         short unused[17];
         double aligner;
     } isdHeader;

`len' is the number of bytes in the message being sent (isdSend()) or
that were received (isdRecv()).  Programmers should not set this value
as it will be overwritten by both isdSend() and isdRecv().

`sequence' is unused by isectd.  It was intended to allow programmers
to assign sequence numbers to the messages.

`reply' is non-zero if the client is going to wait for a reply.  If
`reply' is zero a repsonse to the messages will not be sent to the
client by isectd whether or not the worker sent one.

`error', `command', and `version' are reserved.  The latter being
intended to provide backward compatibility in the event the isdHeader
changes in future revisions.

`workerid' may be set by isectd in the case the worker is waiting for
additional information from the client.  The client should reply
immediately since the worker is unavailable for other clients while
waiting.  This should only occur in circumstances the programmer
intended (never by accident).

Set `more' to a non-zero value if either the worker or the client
intends to send more messages as part of a request.  This is valuable
when either a client or a worker doesn't have all the information
necessary for either a request or a response at one time.  Isectd won't
mark a worker idle until it sends a packet with more == 0.

`unused' is unused-accept that it pads the message to 64 bytes.

`aligner' is used to align the packet to as long a word-boundary as
possible-which normally works for most architectures.  Someone should
let me know when all compilers implement `#assert' directives so I can
`#assert sizeof isdHeader == 64' at compile time and save programmers
the trouble of linking before things blow up.  The Data General C
compiler implemented this and I loved it.


File: isect.info,  Node: isdLogin,  Next: isdLogout,  Prev: isdio,  Up: API

isdLogin
========

     SOCKET isdLogin(char *hostname, char *service, int priority)

`SOCKET' is the file descriptor to use on subsequent calls to
`isdSend()', `isdRecv()', and `isdLogout()'.  `hostname' is an optional
parameter that specifies the name or IP address of a host computer
running isectd.  If hostname is NULL the function will attempt to use
the hostname "isectd" `ISDHOSTNAME' which should be aliased in either
the hosts file or the local DNS maps.

`service' is the arbitrary name used to identify a collection of
workers that understand the same requests.  Service names typically
relate to the actual service provided.  In the example configuration
the service name "echo" is used to send messages to the worker
"isdecho".  The only reserved service name is "op", which isectd
intercepts and handles by itself.  Isectd does not allow competing
definitions of "op" services or workers.

`priority' is an integer used to specify the relative priority of this
client's requests to another client's requests.  isectd uses a simple
heuristic scheduling algorithm which implements priorities by services
lower-valued client connections before higher-valued clients.  For
instance, point-of-sale clients require faster response times than ATM
machines (relatively) and ATM machines require faster response times
than teller terminals.  Posting programs would typically be the lowest
priority.  To implement this ATM clients could connect with a priority
of 0, ATMs with 1, teller programs with 2, and posting programs with 3.
In doing so, a posting program (typically in batch mode) would never
preempt a POS, ATM, or teller transaction (as it probably shouldn't).

Since the priority is relative, any signed integer will do.


File: isect.info,  Node: isdLogout,  Next: isdSend,  Prev: isdLogin,  Up: API

isdLogout
=========

     void isdLogout(SOCKET sock)

`isdLogout()' closes the connection between the client and `isectd' in
an orderly, polite way.  The importance of an orderly disconnect is
dependent on the sophistication of the operating system's TCP/IP stack
and socket library.

`sock' is the file handle returned by `isdLogin()'.

There is no return value.


File: isect.info,  Node: isdSend,  Next: isdRecv,  Prev: isdLogout,  Up: API

isdSend
=======

     int isdSend(SOCKET socket, char *message, int len, isdHeader *header)

Use `isdsend()' to send requests to workers.  `socket' is the socket
value returned from `isdLogin()'.  `message' is the pointer to the
beginning of the buffer to send to the worker.  `len' is the byte
length of `message'.  Setting `len' to -1 translates into `len =
strlen(message)', which is convenient for as often as strings are sent.

`header' is a pointer to an `isdHeader'.


File: isect.info,  Node: isdRecv,  Next: isdConnect,  Prev: isdSend,  Up: API

isdRecv
=======

     char *isdRecv(SOCKET socket, isdHeader *header)

Both clients and workers use `isdRecv()' to receive messages from
`isectd'.  Clients receive messages as responses to requests.  Workers
get requests from clients (via `isectd') using `isdRecv()'.

`socket' is the file handle returned from either `isdLogin()' (clients)
or `isdConnect()' (workers).  After a successful read, `isdRecv()'
returns as its value a pointer to a buffer which is the body of the
message, less the `isdHeader', which is returned separately.  The
length of the buffer is stored in `header->len'.

`header' is a pointer to an `isdHeader'.

If an error occurs reading the message `isdRecv()' returns NULL.


File: isect.info,  Node: isdConnect,  Prev: isdRecv,  Up: API

isdConnect
==========

     SOCKET isdConnect(char **argv)

`isdConnect()' is used by workers to register themselves with `isectd'
after they've started.  `isectd' provides the remote execution daemon,
`isdexecd' with positional starting arguments.

Any additional arguments that may be passed will start after at
ISDARGMAX (from isdio.h).

A successful startup will return a positive (valid) socket for
subsequent calls `isdRecv()' and `isdSend()'.

On failure, `isdConnect()' returns SOCKET_ERROR.


File: isect.info,  Node: Users,  Next: Future,  Prev: API,  Up: Top

Users Guide
***********

CLIENT API

Clients connect to isectd, send messages (requests) to it, optionally
read the responses, and ultimately close their connection.

Before using any of the isd*() functions in your source code you must
first `#include "isdio.h"' to define prototypes and the required
`#define's.  In the interest of brevity the `include' statement is
omitted from the examples.

`isdio.h' is normall installed in /usr/local/include on UNIX systems by
`make install'.

* Menu:

* isdclient::   isect structures and defines
* isectd::      Commands


File: isect.info,  Node: isdclient,  Next: isectd,  Up: Users

isdclient
=========

     isdclient [-d] [-k] [-u] [-h hostname] [-c command] [servicename]

isdclient is a generic TTY interface for sending commands to isectd
workers that accept ASCII input.  isdclient buffers lines from its
standard input until receiving a "go" on a line by itself.  It sends
the buffered data to isectd as input to the OPerator service.  The
message that's returned is written to standard output.


File: isect.info,  Node: isectd,  Prev: isdclient,  Up: Users

isectd
======

     isdclient [-d] [-k] [-u] [-h hostname] [-c command] [servicename]

isdclient is a generic TTY interface for sending commands to isectd
workers that accept ASCII input.  isdclient buffers lines from its
standard input until receiving a "go" on a line by itself.  It sends
the buffered data to isectd as input to the OPerator service.  The
message that's returned is written to standard output.


File: isect.info,  Node: Future,  Prev: Users,  Up: Top

Future
******

LDAP and Username/Passwords

`isectd' should provide a mechanism that implements username/password
security for clients.  Rather than implement its own `isectd' should be
able configurable to use already-existing LDAP-enabled directories.
For sites without directories but the desire to implement security a
template schema file should be provided.

Encryption

The layer should be created on top of the existing API that
automatically encrypts/decrypts messages between clients, workers, and
isectd.  It's reasonable to expect this layer would be compatible with
prevalent RFCs.

Redundancy

The weakest link in a fault-tolerant sense is `isectd'.  If it crashes
all the information currently in-transit between clients and workers is
lost.  This is unlikely because `isectd''s job is fairly
straight-forward but there's no guarantee.

A couple of ways to improve this situation is to log the messages to
disk, or a memory-mapped file so in the case of its inconvenient crash,
some type of state could be recovered.  It doesn't seem to make sense
(at this point at least) to log every message to a file for performance
reasons.

A more interesting approach would be for multiple `isectd's to
cooperate, allowing one to do all the work while informing others of
the sockets it's connected and the messages it's sent.  In the event of
its death a mirror `isectd' could properly dispose or flag the
offending message and continue where its predecessor left off.



Tag Table:
Node: Top239
Node: Introduction1115
Node: Inventory4755
Node: Setup6319
Node: Startup7723
Node: Sample10384
Node: API13237
Node: isdio14029
Node: isdLogin16499
Node: isdLogout18317
Node: isdSend18764
Node: isdRecv19319
Node: isdConnect20100
Node: Users20666
Node: isdclient21303
Node: isectd21788
Node: Future22267

End Tag Table
