BDFAQ -- FAQ for PTCCMAIN replacement   last update 4-oct-94

This is a set of questions that have come up as we describe our new
Planet Connect downlink software to people.   This is not intended to be
a literary work, and unfortunately it is not real well organized, if an
answer does not completely answer one of your questions, keep looking. 
But, hopefully, this FAQ will answer many of your questions.  For
additional information send netmail to 1:105/711.  The current version
of this FAQ is always available at 1:105/711 by freq'ing BDFAQ. 
Additional questions and/or better answers are welcomed.

***  What is the name of the replacement program.

BD.exe  BD is for BirdDroppings.  If the name offends folks we will
change it.

***  Why is this program needed?

PTCCMAIN blows up, it loses data, and omits some operations it should
do.

***  Why does PTCCMAIN blow up?

I don't know, I don't have the source.

***  Why does it lose data?

Again I don't know.  Sloppy coding??

***  If it omits operations it should be performing, how do people get
away with using it?

They use other utilities such as PCFU to clean up the back end of the
data collection procedure.  They tolerate things like missed data.

***  Why can BD collect data without errors when PTCCMAIN loses data?

BD uses a set of serial routines that I've been using  since about 1987
to collect serial data on PC based machines.  These routines, which are
written in assembly language, are well debugged and have been typically
used in process control applications.

***  My PC is not a true IBM compatible, can BD use a fossil? 

BD has been tested with X00 and seems to operate correctly.  However, at
this time, most testing has been done with the built in serial drivers.

***  How do I tell BD to use the fossil instead of the built in
routines?

You can specify it via either a command line parameter or a config file
verb.

***  Does it work with BNU?

I don't know.  There is no reason it should not.  However, fossils are
one of those black magic areas.  X00 works for some folks, BNU works for
others, and both will work for still others.  Generally there should be
no need to use a fossil.

***  How does BD eliminate the need for the back end utilities?

BD does not eliminate the need for all back end utilities.  Utilities
traditionally used such as tick, raid, and areafix are still needed. 
What BD eliminates is the need for some functions performed by programs
such as PCFU.

***  How does BD do that?

PC sends a file called UPLD.LST periodically.  This file has a list of
all the files being uplinked that day.  BD examines UPLD.LST and adds
any new entries to another file called BD.DUP.  BD.DUP has a list of
everything that has been in UPLD.LST in the last 48 hours and anything
else that has been uplinked.  When a file is correctly received BD
checks BD.DUP to see if the file has been received correctly before; if
it has, the file is ignored.  If it has not, the file is placed in the
correct directory.  In either case the date stamp for the file in BD.DUP 
is updated.  Anytime a file has not been transmitted within the previous
48 hours its entry in BD.DUP  is deleted.  If the file is never received
correctly, its name is written into another file called NOTHERE so it
can be freq'ed if the sysop desires.

***  PTCCMAIN requires that all files be placed on the same drive.  Does
BD have the same limitation?

No, files can be on any drive and in any subdirectory.  The drive can be
a network drive.

***  What configurations has BD been tested in?

It was constantly tested during development on both a 286-20 and a
386dx-40.  Both have been tested while writing the files via network to
OS/2 and writing the files locally.  The network interface has been
tested using both eight bit 3Com cards and 16 bit Intel cards.  The
program has a reasonable looking display with both mono and svga
adaptors.  It has been periodically tested with the turbo mode on both
machines turned off and continues to work correctly.   (PTCCMAIN will
not run on either machine if they are not in turbo mode).  My testing
has been done with a 'C' band dish.  I also have Lantastic and intend to
test BD using that network before a real release of the program.  We did
test the program once on a 4.77mhz 8088 (original IBM PC clone), on that
machine the program slowly got behind and started to loose characters
after about three minutes.

***  Is it 16550a aware?  Does it require a 16550?

BD automatically detects a 16550a and enables the fifo if the built in
drivers are used.  In the  early days of program debugging we tested on
the 286-20 with a 16450.  It has not been checked recently.  The 386dx-
40 we test with has a super I/O card with a "do_everything" ASIC on it
that does not have an input fifo.  We have been able to run BD without
data errors when writing to a local hard drive.  With an Intel 16exp
network card it could receive error free data and write over an OS/2
network AS LONG AS there was very little keyboard activity.  


***   How do I tell I need a 16550?

If you have any problems, get a 16550a.  They are not expensive.


***  Will BD work with a 'KU' dish?

It has been beta tested at several KU sites.  Due to the KU uplink
procedure, the Planet Connect KU band data signal has more errors than
the C band data signal.

***  Will BD run under OS/2?

There is an OS/2 version currently in beta testing.  When the OS/2
computer is very busy, it currently gets data errors that the DOS box
doesn't.

***  Will BD run under DOS multitaskers?

It has some "off the shelf" task swapping code built in.  I do not run
any DOS multitaskers so we can not test it.  It has been beta tested
under DesqView and we have not received any major complaints.

***  If Planet Connect increases the speed, will BD continue to operate.

We have tested BD at 115.2k with our 286-20 (using a 16550a), it works
correctly and did not loose any data.  The test consisted of sending
actual PC data that had been saved on disk to the program.

***  PC claims that they transmit at 19.2k bps, but it looks to me like
they are really at 14400 or some other speed.  What are they really
doing?

Planet Connect transmits a bunch of 'filler' between each  data block. 
The filler amounts to about ten percent of the total data.  The filler
is not needed by BD.  There is also a special header and trailer block
on each file and other overhead bytes in the data stream.  It is
reasonable for them to be there.

The third field in the stat window's header bar is the number of
bytes read from the com port since BD was last started.

*** You tested BD at 115.2k, what did that really translate to in useful
information?

We wrote one million bytes to disk (via lan) about every minute and a
half.

***  How are you using Planet Connect for FIDO mail?

We have a 286-20 collecting data from the receiver.  It places the files
into the normal inbound area on our OS/2 BBS system.  We have the
program create a semaphore file after each file is received.  In
addition, we have the batch file that runs Binkley check if any echo
mail files have been received every time it cycles.  If any have, they
are processed.  This is the way that most hubs would run.  By doing
this, the mail will be tossed as it is received and at least for folks
on the West coast, FIDO mail can be on most systems when folks get up in
the morning.  If someone does not want to use the semaphores, it may
require a Binkley event a few minutes after PC starts a new cycle to get
this all rolling.  After that, the mailer calls will trigger the
additional checks.

***  What program are you using to toss mail?

OS/2 squish version 1.10.  Except for the volume of mail received, I
don't know why any tosser should not work.

***  Do you run packet sort or anything like that?

No.  Squish works fine.  We automatically delete dupes and have a
program to automatically add new areas to our areas.bbs file as
passthru.  The first time Planet Connect sends us a message in a new
area the message ends up in our bad area.  We run our add program before
each squish pass and then squish automatically tosses the "previously"
bad messages.

***  So you are not using error exits from the program?

No, we are not.  The program runs continually with the semaphore
control.  Someone who wished to use stay compatible with PTCCMAIN could
use the error levels.

***  What data are you using from the Planet Connect system.

I save FIDO mail, familynet mail, and the filebone files.  I monitor
USENET mail.  Once Planet Connect has a complete USENET feed (and a
return path) I will switch to it from my other feed.  Right now Planet
Connect does not transmit all of the USENET mail so I get it via
landline.  

***  Would BD require changes if Planet Connect started uplinking new
FIDO mail around the clock?  Would it require batch file changes?

No it would not.  We hope that Planet Connect will make this change.

***  What language is BD written in?

A very uninspired style of Borland C.  I say uninspired because no
clever tricks are used to improve performance.  It should be easy to
maintain and should not have weird bugs cropping up.

***  Does it require the Planet Connect dongle?

Not right now.  All beta test versions are time limited.  The purpose of
this program is to overcome deficiencies in PTCCMAIN, NOT to overcome
the dongle.

***  Have you broken the Planet Connect dongle security?

No, and I currently have no intention of doing so.

***  Could you do it if you wanted to?

Yes, but I currently have no intention of doing so.

***  Are you going to put the dongle into the program?

That will require the cooperation of Planet Connect.  I don't like
dongles.  Planet Connect has gotten some bad advice and seems to think
that it is a method of signal protection.  However, if they requested it
I would do it.

***  Does this program process all of the downlinked areas?

Not currently.  We have no way to test the secure areas or the non-fido
e-mail.  For testing, we are running both PTCCMAIN and BD and compare
the files.  The files saved on the disk by the two programs are
identical except when PTCCMAIN makes a mistake.

***  How much more reliable is this program than PTCCMAIN?

Our version of PTCCMAIN crashes at least once or twice each day.  BD is
not crashing at all.

***  Does BD crash with the "high ascii" problem associated with rime?

BD is not crashing at all for any reason.  The high ascii problem does
not have anything to do with the reception of rime data (it may have
something do to with the special handling the program does for the rime
data).  ZIP/ARJ/LHA files all have "high ascii" in them.  PTCCMAIN is
able to downlink those files without crashing.

***  How many data passes does it take to get all of the files?

BD usually gets most of the files on the first pass.  Any files it does
not get on the first pass are also missed by PTCCMAIN.  PTCCMAIN usually
misses files that BD does not.

***  PTCCMAIN seems to write incomplete files.  Does BD have this
problem?

On our system, PTCCMAIN often writes files with one or more 512 byte
blocks missing.  These missing blocks seem to occur at random places in
the file.  We have not seen this problem with BD.

***  What makes you think you can write and support a program like this.

I've been writing operating systems, drivers, and process control
software since 1968.  I typically write code in Forth, 'C', or assembly
language.  I use everything from one chip brain dead cpu's (things like
8051's and 6805's) to high performance 680x0 unix systems.  From my
point of view, BD and PTCCMAIN are rather simple minded data logging
programs.  Except for a five year period when I was the president of
small systems company I have been a self employed consultant most of
that time.  If I did not do a good job, I would have had to get a *real*
job long before now.

***  Are you willing to provide long term support for this program?

I normally work by the hour.<grin>  I might as well work on this as
something else.  However, please note that my answer implies that I want
paid for the work on this program.  I do not intend to continue the
developement for free.  The state considers it to be child abuse if my
children do not get to eat three meals a day.

***  Do you believe that the program will require additional
modifications?

This whole concept is a new one.  I expect numerous changes over the
next few months as Planet Connect and the BBS community figures out what
really should be done.

***  Are you willing to monitor the Planet Connect echo and answer
questions.  The PTCCMAIN author does not do this.  It is frustrating to
have questions and not have any 'real' technical person willing to
answer them.

I'd rather provide support in PC_SITES or some other *sorta* private
echo.  (update.  This echo is called BIRDDROP.  It will soon be
backboned)

***  How were you able to do this without the help of Planet Connect?

Skill, I guess.   Actually, I could improve the program a lot with the
cooperation of Planet Connect.  With a few changes to the uplink
protocol I could eliminate a lot of hard disk rattling.

***  Could you do that without breaking all the BBS's using Planet
Connect?

Yes, we could create a program that has both the current and the new
protocol.  The program would use the existing protocol and then
automatically switch to the new one when the protocol was changed.  The
new program could be uplinked so that all PC subscribers had it, and
then a month later the protocol changed.  I doubt if subscribers would
even notice when the protocol changed.

***  Have you got enough memory to do that?

Sure.  The program takes about 250k  (we could shorten it somewhat by
shortening some buffers).  An additional protocol should only add 10-20k
bytes at most.  The redundant code could be removed during the next set
of changes.

***  How long have you really been working on this program?  I don't
believe it that you started last week.

I am one of a group of people that bought a dish in Portland last
November.  The sysop at the downlink site complained about PTCCMAIN ever
since the system came online.  I've had numerous conversations with him
and with other PC subscribers about PTCCMAIN.  Frankly, I did not
believe things were as bad as everyone let on.  I got my own system
running on the fifth of July.  Six hours after getting PTCCMAIN online,
we started coding, it was obvious we were not going to tolerate PTCCMAIN
long term.  If we had not come up with a replacement, we would have sent
the PC equipment back and switched to pagesat.  So far it is five *hard*
weeks of coding, but a few months of discussions of how the program
should work are behind it.  The experience of writing similar programs
and having useable debugged code certainly made it easier.

***  I did not know pagesat transmitted FIDO echos.

Rumors are that they are getting ready to do it.  A friend of mine with
a pagesat feed has been getting the filebone from it.  Pagesat does send
USENET and that is also useful to me.

***   You said "We" started writing?

One of the fellows who works for me has taken this project on with a
passion.  He has actually done most of the work.

***  What are you going to do with this program?

I'd like to sell it to Planet Connect.  In its current form, it has no
value to anyone except Planet Connect subscribers.  Planet Connect has
to supply a solid downlink program or other folks are also going to
write their own.  Other folks may not care about the ethics of breaking
the Planet Connect security.

If Planet Connect is not interested, I will offer it to Planet Connect
subscribers or pagesat.

Additionally, I do a lot of programming for a company that builds
various kinds of data modems for residential cable TV systems.  It
should be possible to modify the program to receive a Planet Connect
type of transmission via normal cable TV with a special data modem.

***  How can I get a copy of BD?

Even though it is being used, the program is not quite finished and I
don't want to release many more copies until I talk to Planet Connect
about the program.  If I don't hear from them, I'll decide what to do
later.  However, additional suggestions about how the program should
work are welcome and might get you a beta version to try out.

***  How hard is it to configure BD?

It is simple.  Check the example config file or just use your existing
one for PTCCMAIN.


*** I accidentally deleted a file that I wanted, can I get it on the
next pass? 

There are two ways to tell BD to reprocess a file.  One way is from the
stats window; position the cursor over the file and press Alt-f.  This
puts the cursor in edit flags mode.  (A '*' will be displayed at the end
of the line.)  Then press 'G' which sets the GET flag.  Any other
non-flag key will exit the editing flags mode. 

The other way is to create a file named 'RESCAN' in the current
directory.  Put the names of any files you want in there.  This rescan
file will be checked after the next non-bad satellite file.   If the
file does not come, the filename will be written to 'NOTHERE'. 

*** What are the windows on the screen?

The screen is divided into two windows.  The top one contains
printouts of the several logs that are kept.  (pressing the space bar
will cycle between the logs)  The bottom window contains some summary
data.  

*** What kind of log data is displayed?

The first log, which is displayed when you first start BD, is a
binkley style log.  You can limit what items are logged by changing
the config file verb, LogLevel.  

The second log contains one entry for every file that is in the
current upld.lst and secure.lst.  Plannet Connect sends out a new
copy of these files once at the beginning of each transmit day. (7:30
eastern)  

*** What does the data in the second log mean?

Filename   - Name of file attached to entry
Entry Date - Date this entry was created  (probably will be removed)
File Date  - Date this file was created on PC's end
Size	   - Size of the file in bytes
CRC        - Crc we computed on the file
Gr         - Group the file belongs to.  Used to tell what directory
             to put the file in  (see the ACONFIG.SKY file)
dp         - Number of times this file came in as a duplicate
up         - Number of times this file came in as an update
Bd         - Number of times this file came in as bad

Flags - The flags mean as follows:

H - File is here.  File has been correctly received.  
G - Get file on next pass.  
U - This entry was created from a filename that was in the Upld.lst file
S - This entry was created from a filename that was in the Secure.lst file
D - File was dumped.  Usually due to being an encrypted file.  May be due
        to disk full.

*** What does the stat window tell me?

In the title bar:  
-- time of day -- elapsed run time -- bytes read from com port -----

The first line in window deals with current file.
The second line in window deals with sum totals since BD last started.

The third line has miscellanous stuff on it.
  The os entry is the os that we are giving time slices up to.
  The files to get yet is files in todays upld.lst that have
    not been received correctly yet.
  The blinking NO DATA says that there is no data coming in from
    the com port.

*** How does BD detect duplicates?

It compares groups, lengths, and PC's file creation dates. 

*** How does it verify the validity of files?

We examine every block to verify that
(1) that the block has the correct CRC;
(2) that the blocks are sequential; and
(3) that the blocks are in the same file.

*** How can I delete the log?

Just delete it.  BD saves log entries to memory.  Then when it has time,
it opens the log and writes what it has.  Then it closes it. If there is
a conflict, BD waits and tries again later.

*** Does it make any kind of stats?

If the program is run with the '-a' switch, it will display a listing of
the stat file.  The Binkley style log has more data, but it definitely
needs a program written to analyze it.

*** Can I install it over PTCCMAIN?

Yes, and with no changes to your old config files.

BD comes with a generic config file set up for com1 that has all error
level exits disabled.  The config file options are explained in the
config file.  The only file from PTCCMAIN that is needed is ACONFIG.SKY. 
MAKECFG is not used.  Any changes to ACONFIG.SKY will take effect the
next BD time is loaded.

*** PTCCMAIN wont create the directories needed from the ACONFIG.SKY
file.    Will BD?

Yes.  And they can be on different drives.

*** Does BD have any command line options?

Yes.  Type 'BD -h' to see them.

*** How do I report bugs?

Send us email at one of the following addresses:

1:105/711
bird.droppings@f711.n105.z1.fidonet.org




*** My semaphores are not being created.  What's wrong?

Probably the particular semaphore entry in the config file is wrong.  If
the program says it can't create the semaphore, the directories probably
aren't there.

*** How do I know when my downlink site is having problems? 

All errors that the administrator should know about are put in the
'ERROR.LOG' file. 

*** What date and time do the downlinked files have?

I'm not really certain but believe it is the time that the file was
created at Planet Connect's end.

*** I don't want to save the files in one group.  How do I dump them? 

In the 'ACONFIG.SKY' file, change the directory to 'NUL'. OR, append
'NUL'  to the end of the path statement. ('NUL' can be 'NUL' or 'NUL/').

*** How do I set up the semaphores?

There is an explanation of the semaphores in the config file.

*** How do I know what error level the program exited with? 

When the program exits, it writes the error level that it exits with to
the log file and displays it on the screen.

*** If BD fills up one drive, will it continue to save files that should
be written on a different drive?

Yes.  And it won't hang or fill up the screen with complaints like
PTCCMAIN. 

*** Has it been tested with a PS/2 computer.

No.  If you have one and BD does not work correctly use a fossel.


*** Are there any keys that I should know about?

The arrow keys, page up and down, home, and end all move around in
the logs.  Alt-x exits the program.  Spacebar will cycle through
several log displays.

*** What are the files that BD uses or creates?

The only file that BD requires to start is a config file, bd.cfg.

If something deletes the dup info file, BD will not know that it has
already received any files.

Here is a list of the files that BD creates and their default names.

binkley style log    - bd.log
dup info file        - bd.dup
error log            - error.log
files never received - NotHere

