@node GPC source
@chapter The GPC Source Reference
@cindex GPC source
@cindex GPC internals

@quotation
@i{``The Source will be with you.  Always.''}
@end quotation

This section tells you how to look up additional information about the GNU
Pascal compiler from its source code.  It replaces chapters like ``syntax
diagrams'' you probably know from the documentation of other compilers.

Proprietary compilers often come with a lot of technical information about
the internals of the compiler.  This is necessary because their vendors want
to avoid to distribute the source of the compiler---which is always the most
definitive source of this technical information.

With GNU compilers, on the other hand, you are free to get the source code,
look how your compiler works internally, customize it for your own needs,
and to re-distribute it in modified or unmodified form.  You may even take
money for this re-distribution.  (For details, see the GNU General Public
License, @ref{Copying}.)

The following subsections are your guide to the GNU Pascal source code.
If you have further questions, be welcome to ask them at the GNU Pascal
mailing list (see @ref{Support}).

All file paths mentioned in this chapter are relative to the GNU Pascal
source directory, usually a subdirectory @file{p} of the GCC source.
For instance, @file{parse.y} means @file{/usr/local/src/gcc-2.8.1/p/parse.y}
(or whereever you have installed the GCC source on your machine);
@file{../tree.def} means @file{/usr/local/src/gcc-2.8.1/tree.def}, etc.

@menu
* Lexical analyzer::     @file{gpc-lex.c}---How GPC reads your source.
* Language definition::  @file{parse.y}---``Syntax diagrams'' as ``Bison'' source.
* Tree nodes::           @file{../tree.*}---How GPC stores your program internally.
* Parameter passing::    @file{gpc-typeck.c}---How GPC passes parameters.
* GPI files::            @file{module.c}---How GPC's precompiled Module/Unit interfaces work.
* AutoMake::             @file{module.c}---How GPC automatically ``makes'' a large project.
@end menu

(Under construction.)

For more information, see chapters ``Portability'' through ``Fragments'' in
@ref{,the GCC documentation,,gcc,``Using and Porting GNU CC''}.


@c ========================================================================


@node Lexical analyzer
@section GPC's Lexical Analyzer
@cindex lexical analyzer

The source file @file{gpc-lex.c} contains the so-called @emph{lexical
analyzer} of the GNU Pascal compiler.  (For those of you who know
@file{flex}:  This file was @emph{not} created using @file{flex} but
is maintained manually.)  This very-first stage of the compiler is
responsible for reading what you have written and dividing it into
@emph{tokens}, the ``atoms'' of each computer language.  The source
@file{gpc-lex.c} essentially contains one large function, @samp{yylex()}.

Here is, for example, where the real number @samp{3.14} and the subrange
@samp{3..14} are distinguished, and where Borland-style character constants
like @samp{#13} and @samp{^M} are recognized.  This is not always a trivial
task, for example look at the following type declaration:

@smallexample
  Type
    X = ^Y;
    Y = packed array [ ^A..^B ] of Char;
    Z = ^A..^Z;
@end smallexample

If you wish to know how GPC distinguishes the pointer forward declaration
@samp{^Y} and the subrange @samp{^A..^Z}, see @file{gpc-lex.c}, function
@samp{yylex()}, @samp{case '^':} in the big @samp{switch} statement.

@cindex foo
There are several situation where GPC's lexical analzyer becomes
context-sensitive.  One is described above, another example is the token
@samp{protected}, a reserved word in ISO-10206 Extended Pascal, but an
ordinary identifier in ISO-7185 Standard Pascal.  It appears in parameter
lists

@smallexample
  Procedure foo ( protected bar: Integer );
@end smallexample

and says that the parameter @samp{bar} must not be changed inside the body of
the procedure.

OTOH, if you write a valid ISO-7185 Standard Pascal program, you can declare a
parameter @samp{protected}:

@smallexample
  Procedure foo ( protected, bar: Integer );
@end smallexample

Here both standards contradict each other.  GPC solves this problem by
checking explicitly for ``protected'' in the lexical analyzer:  If a comma or
a colon follows, this is an ordinary identifier, otherwise it's a reserved
word.  Having this, GPC even understands

@smallexample
  Procedure foo ( protected protected: Integer );
@end smallexample

without losing the special meaning of @samp{protected} as a reserved word.

The responsible code is in @file{gpc-lex.c}---look out for @samp{PROTECTED}.

If you ever encouter a bug with the lexical analyzer---now you know where to
hunt for it.


@c ========================================================================


@node Language definition
@section Language Definition: GPC's Parser
@cindex language definition
@cindex parser
@cindex grammar
@cindex front-end


The file @file{parse.y} contains the ``bison'' source code of GNU Pascal's
parser.  This stage of the compilation analyzes and checks the syntax of
your Pascal program, and it generates an intermediate, language-independent
code which is then passed to the GNU back-end.

The @emph{bison} language essentially is a machine-readable form of the
Backus-Naur Form, the symbolic notation of grammars of computer languages.
``Syntax diagrams'' are a graphical variant of the Backus-Naur Form.

For details about the ``bison'' language, see
@ref{,the Bison documentation,,bison,the Bison manual}.  A short overview
how to pick up some information you might need for programming follows.

Suppose you have forgotten how a variable is declared in Pascal.  After some
searching in @file{parse.y} you have found the following:

@smallexample
  /* variable declaration part */

  variable_declaration_part:
            VAR variable_declaration_list semi
          | VAR semi
                  @{ error ("missing variable declaration"); @}
          ;

  variable_declaration_list:
            variable_declaration
          | variable_declaration_list semi variable_declaration
                  @{ yyerrok; @}
          | error
          | variable_declaration_list error variable_declaration
                  @{
		    error("missing semicolon");
                    yyerrok;
		  @}
          | variable_declaration_list semi error
          ;
@end smallexample

Translated into English, this means: ``The variable declaration part consists
of the reserved word @samp{Var} followed by a `variable declaration list' and
a semicolon.  A semicolon immediately following @samp{Var} is an error.  A
`variable declaration list' in turn consists of one or more `variable
declarations', separated by semicolons.'' (The latter explanation requires
that you understand the recursive nature of the definition of
@samp{variable_declaration_list}.)

Now we can go on and search for @samp{variable_declaration}.

@smallexample
  variable_declaration:
            id_list
                  @{
                    [...]
                  @}
            enable_caret ':' optional_qualifier_list type_denoter
            absolute_or_value_specification
                  @{
                    [...]
                  @}
          ;
@end smallexample

(The @samp{[...]} are placeholders for some C statements which aren't
important for understanding GPC's grammar.)

From this you can look up that a variable declaration in GNU Pascal consists
of an ``id list'', followed by ``enable_caret'' (whatever that means), a colon,
an ``optional qualifier list'', a ``type denoter'', and an ``absolute or value
specification''.  Some of these parts are easy to understand, the others you
can look up from @file{parse.y}.  Remember that the reserved word @samp{Var}
preceeds all this, and a semicolon follows all this.

Now you know the procedure how to get the exact grammar of the GNU Pascal
language from the source.

The C statements, not shown above, are in some sense the most important part
of the bison source, because they are responsible for the generation of
the intermediate code of the GNU Pascal front-end, the so-called
@emph{tree nodes}.  For instance, the C code in ``type denoter'' juggles
a while with variables of the type @samp{tree}, and finally returns (assigns
to @samp{$$}) a so-called @emph{tree list} which contains the information
about the type.  The ``variable declaration'' gets this tree list (as the
argument @samp{$6}) and passes the type information to the C function
@samp{declare_vars()} (declared in @samp{util.c}).  This function
@samp{declare_vars()} does the real work of compiling a variable
declaration.

This, the parser, is the place where it becomes Pascal.


@c ========================================================================


@node Tree nodes
@section Tree Nodes
@cindex tree nodes
@cindex intermediate code
@cindex front-end

If you want really to understand how the GNU Pascal language front-end works
internally and perhaps want to improve the compiler, it is important that you
understand GPC's internal data structures.

The data structure used by the language front-end to hold all information
about your Pascal program are the so-called ``tree nodes''.  (Well, it needn't
be Pascal source---tree nodes are language independent.)  The tree nodes are
kind of objects, connected to each other via pointers.  Since the GNU compiler
is written in C and was created at a time where nobody really thought about
object-orientated programming languages yet, a lot of effort has been taken to
create these ``objects'' in C.

Here is an extract from the ``object hierarchy''.  Omissions are marked
with ``@dots{}''; nodes in parentheses are ``abstract'':  They are never
instantiated and aren't really defined.  They only appear here to clarify
the structure of the tree node hierarchy.  The complete list is in
@file{../tree.def}; additional information can be found in @file{../tree.h}.

@example
  (tree_node)
  |
  |--- error_mark  (* enables GPC to continue after first error *)
  |
  |--- (identifier)
  |    |
  |    |--- identifier_node
  |    |
  |    \--- op_identifier
  |
  |--- tree_list  (* general-purpose "container object" *)
  |
  |--- tree_vec
  |
  |--- block
  |
  |--- (type)  (* information about types *)
  |    |
  |    |--- void_type
  |    |
  |    |--- integer_type
  |   ...
  |    |
  |    |--- record_type
  |    |
  |    |--- function_type
  |    |
  |    \--- lang_type  (* for language-specific extensions *)
  |
  |--- integer_cst  (* an integer constant *)
  |
  |--- real_cst
  |
  |--- string_cst
  |
  |--- complex_cst
  |
  |--- (declaration)
  |    |
  |    |--- function_decl
  |   ...
  |    |
  |    |--- type_decl
  |    |
  |    \--- var_decl
  |
  |--- (reference)
  |    |
  |    |--- component_ref
  |   ...
  |    |
  |    \--- array_ref
  |
  |--- constructor
  |
  \--- (expression)
       |
       |--- modify_expr  (* assignment *)
       |
       |--- plus_expr  (* addition *)
      ...
       |
       |--- call_expr  (* procedure/function call *)
       |
       |--- goto_expr
       |
       \--- loop_expr  (* for all loops *)
@end example

@cindex foo
Most of these tree nodes---struct variables in fact---contain pointers to
other tree nodes.  A @samp{tree_list} for instance has a @samp{tree_value}
and a @samp{tree_purpose} slot which can contain arbitrary data; a third
pointer @samp{tree_chain} points to the next @samp{tree_list} node and thus
allows us to create linked lists of tree nodes.

One example:  When GPC reads the list of identifiers in a variable
declaration

@smallexample
  Var
    foo, bar, baz: Integer;
@end smallexample

@cindex magic
the parser creates a chain of @samp{tree_list}s whose @samp{tree_value}s hold
@samp{identifier_node}s for the identifiers @samp{foo}, @samp{bar}, and
@samp{baz}.  The function @samp{declare_vars()} (declared in @file{util.c})
gets this tree list as a parameter, does some magic, and finally passes a
chain of @samp{var_decl} nodes to the back-end.

The @samp{var_decl} nodes in turn have a pointer @samp{tree_type} which holds
a @samp{_type} node---an @samp{integer_type} node in the example above.
Having this, GPC can do type-checking when a variable is referenced.

For another example, let's look at the following statement:

@smallexample
  baz:= foo + bar;
@end smallexample

Here the parser creates a @samp{modify_expr} tree node.  This node has two
pointers, @samp{tree_operand[0]} which holds a representation of @samp{baz}, a
@samp{var_decl} node, and @samp{tree_operand[1]} which holds a representation
of the sum @samp{foo + bar}.  The sum in turn is represented as a
@samp{plus_expr} tree node whose @samp{tree_operand[0]} is the @samp{var_decl}
node @samp{foo}, and whose @samp{tree_operand[1]} is the @samp{var_decl} node
@samp{bar}.  Passing this (the @samp{modify_expr} node) to the back-end
results in assembler code for the assignment.

If you want to have a closer look at these tree nodes, write a line
@samp{(*$debug-tree="Foobar"*)} into your program with @samp{FooBar} being
some identifier in your program.  (Note the capitalization of the first
character in the internal representation.)  This tells GPC to output the
@samp{identifier_local_value} tree node---the meaning of this
identifier---to the standard error device in human-readable form.

While hacking and debugging GPC, you will also wish to have a look at these
tree nodes in other cases.  Use the @samp{debug_tree()} function to do so.
(In fact @samp{(*$debug-tree="Foobar"*)} does nothing else than to
@samp{debug_tree()} the @samp{identifier_local_value} of the @samp{Foobar}
identifier node.)


@c ========================================================================


@node Parameter passing
@section Parameter Passing
@cindex parameter passing

GPC supports a lot of funny things in parameter lists:  @samp{protected} and
@samp{const} parameters, strings with specified or unspecified length,
conformant arrays, objects as @samp{Var} parameters, etc.  All this requires
sophisticated type-checking; the responsible function is
@samp{convert_arguments()} in the source file @file{gpc-typeck.c}.  Every
detail can be looked up from there.

Some short notes about the most interesting cases follow.

@table @strong

@cindex conformant arrays
@item Conformant arrays:
First, the array bounds are passed (an even number of parameters of an ordinal
type), then the address of the array itself.

@cindex procedural parameters
@cindex functions as parameters
@item Procedural parameters:
These need special care because a function passed as a parameter can be
confused with a call to the function whose result is then passed as a
parameter.  See also the functions @samp{maybe_call_function()} and
@samp{probably_call_function()} in @file{util.c}.

@item Chars:
According to ISO-10206 Extended Pascal, formal char parameters accept string
values.  GPC does the necessary conversion implicitly.  The empty string
produces a space.

@item Strings and schemata:
Value parameter strings and schemata of known size are really passed by value.
Value parameter strings and schemata of unknown size are passed by reference,
and GPC creates temporary variable to hold a copy of the string.

@item @samp{Const} parameters:
If a constant value is passed to a @samp{Const} parameter, GPC assigns the
value to a temporary variable whose address is passed.

@cindex foo
@item Typeless parameters:
These are denoted by @samp{Var foo} or @samp{Var foo: Void} and are compatible
to C's @samp{void *} parameters; the size of such entities is @emph{not}
passed.  Maybe we will change this in the future and pass the size for
@samp{Var foo} parameters whereas @samp{Var foo: Void} will remain compatible
to C.

@item @samp{CString} parameters:
GPC implicitly converts any string value such that the address of the actual
string data is passed.  However, GPC does not implicitly append a
@samp{chr ( 0 )} terminator, except for string constants.

@end table


@c ========================================================================


@c GPI.DOC, version 1.0 for GNU Pascal 2.0 (corresponding to GCC 2.8.1)
@c 26 Jul 1996, Peter Gerwinski <peter@gerwinski.de>
@c integrated into `gpc-source.texi' 1 Aug 1997 PG

@node GPI files
@section GPI files---GNU Pascal Interfaces
@cindex GPI files

This section documents the mechanism how GPC transfers information
from the exporting Modules and Units to the Program, Module or
Unit which imports (uses) the information.

A GPI file contains a precompiled GNU Pascal Interface.  ``Precompiled''
means in this context that the Interface already has been
parsed (i.e.@: the front-end has done its work), but that no assembler
output has been produced yet.

The GPI file format is an implementation-dependent (but not @emph{too}
implementation-dependent ;-) file format for storing GNU Pascal 
Interfaces to be exported---Extended Pascal and PXSC module 
interfaces as well as interface parts of Borland Pascal Units 
compiled with GNU Pascal.

To see what information is stored in or loaded from a GPI file, run 
GPC with an additional command-line option @samp{--debug-gpi}.  Then, GPC 
will write a human-readable version of what is being stored/loaded 
to the standard error device.  (See also: @ref{Tree nodes}.)

While parsing an Interface, GPC stores the names of exported objects
in tree lists---in @file{gpc-parse.y}, the bison (yacc) source of GPC's
parser, search for @samp{handle_autoexport}.  At the end of the Interface,
everything is stored in one or more GPI files.  This is called in
@file{gpc-parse.y}---search for @samp{create_gpi_files}.  (See also:
@ref{Language definition}, for an introduction to @file{gpc-parse.y})

Everything else is done in gpc-module.c.  Here you can find the source
of @samp{create_gpi_files()} which documents the file format:  First, a header 
of 33 bytes containing the string @samp{GNU Pascal Module/Unit Interface\n}
is stored, then the name of the primary source file of the module as a
string, then the name of the exported interface as a tree node (see
below), after that all exported names in the order as they were stored
while parsing.

The names and the objects (i.e.@: constants, data types, variables and
functions) they refer to are internally represented as so-called
@emph{tree nodes} as defined in the files @file{../tree.h} and
@file{../tree.def} from the GNU compiler back-end.  (See also: @ref{Tree
nodes}.)  The names are stored as @samp{identifier_node}s, their
meanings as @samp{identifier_global_value}s of these nodes.
The main problem when storing tree nodes is that they 
form a complicated tree in memory with a lot of circular references
making it hard to decide which information must be stored and which 
mustn't.

The functions @samp{load_tree()} and @samp{store_tree} load/store
a tree node with the contents of all its contained pointers in a GPI file.

Each tree node has a @samp{tree_code} indicating what kind of information
it contains.  Each different tree node must be stored in a different
way.  See the source of @samp{load_tree()} and @samp{store_tree()} for
details.

Most tree nodes contain pointers to other tree nodes; therefore
@samp{load_tree()} and @samp{store_tree()} are recursive functions.  
The @samp{--debug-gpi} debugging informations contains the recursion 
level in parantheses, e.g.@: @samp{loaded (2):} means that the loaded 
information was requested by a pointer contained in a tree node 
requested by a pointer contained in a tree node representing an 
exported symbol.

Since this recursion can be circular (think of a record containing
a pointer to a record of the same type), we must resolve references
to tree nodes which already have been loaded.  For this reason, all
nodes are recorded in a hash buffer---see @file{gpi-hash.c}.

There are some special tree_nodes (e.g.@: @samp{integer_type_node} or
@samp{NULL_TREE}) which are used very often.  They have been assigned
(normally invalid) unique @samp{tree_code}s, so they can be stored in
a single byte.

That's it.  Now you should be able to ``read'' GPI files using GPC's
@samp{--debug-gpi} option.  If you encounter a case where the loaded 
information differs too much from the stored information, you 
have found a bug---congratulations!  What ``too much'' means, depends
on the object being stored in or loaded from the GPI file.  Remind
that the order things are loaded from a GPI file is the @emph{reversed}
order things are stored when considering @emph{different} recursion levels,
but the @emph{same} order when considering ths @emph{same} recursion level.


@c ========================================================================


@c AUTOMAKE.DOC, version 1.0 for GNU Pascal 2.0 (corresponding to GCC 2.7.2.1)
@c 28 Jul 1996, Peter Gerwinski <peter.gerwinski@uni-essen.de>
@c integrated into `gpc-source.texi' 1 Aug 1997 PG

@node AutoMake
@section GPC's AutoMake Mechanism---How it Works
@cindex AutoMake internals

When a Program/Module/Unit imports (uses) an Interface, GPC
searches for the GPI file (see @ref{GPI files}) derived from the
name of the Interface.

Case 1:  A GPI file was found.

Each GPI file contains the name of the primary source file
(normally a @file{.pas} or @file{.p} file) of the Module/Unit, and the
names of all interfaces imported.  GPC reads this information 
and invokes itself with a command like

@cindex foo
@smallexample
  gpc foo.pas -M -o foo.d
@end smallexample

This means: preprocess the file, and write down the name of
the object file and those of all its source files in @file{foo.d}.
GPC reads @file{foo.d} and looks if the object file exists and if
the source was modified since the creation of the object file
and the gpi file.  If so, GPC calls itself again to compile
the primary source file.  When everything is done, the @file{.d}
file is removed.  If there was no need to recompile, all
interfaces imported by the Module/Unit are processed in the
same way as this one.

Case 2:  No GPI file was found.

In this case, GPC derives the name of the source file from
that of the Interface by trying first @file{interface.p}, then
@file{interface.pas}.  This will almost always work with Borland
Pascal Units, almost never with Extended Pascal Modules.  With
Extended Pascal, compile the Module once manually in order to
produce a GPI file.

All this is done by the function @samp{gpi_open()} which uses some
auxiliary functions such as @samp{module_must_be_recompiled()} and
@samp{compile_module()}.

Each time an object file is compiled or recognized as being
up-to-date, its name is stored in a temporary file with the same base
name as all the other temporary files used by GPC but the extension
@file{.gpc}.  When the top-level @file{gpc} is invoked 
(which calls @file{gpc1} later on), it passes the name of this 
temporary file as an additional command line parameter to
@file{gpc1}.  After compilation has been completed, the top-level 
@file{gpc} reads the temporary file and adds the new object files 
to the arguments passed to the linker.

The additional command @samp{--amtmpfile} (not to be specified
by the user!) is passed to child GPC processes, so all compiles
use the same temporary file.

The source for this is merely in @file{gpc-module.c}, but there are
also some hacks in @file{gpc.c}, additional command line options in
@file{lang-options.h} and @file{gpc-decl.c}, and @file{gpc-defs.h}
contains declarations for the functions and global variables.

