Import /usr/src/bin/sh from FreeBSD 12.1-RELEASE

1 files changed, 301 insertions, 0 deletions

diff --git a/bin/1sh/TOUR b/bin/1sh/TOUR
new file mode 100644
index 00000000..3f196c9b
--- /dev/null
+++ b/bin/1sh/TOUR
@@ -0,0 +1,301 @@
+#	@(#)TOUR	8.1 (Berkeley) 5/31/93
+# $FreeBSD: releng/12.1/bin/sh/TOUR 317882 2017-05-06 13:28:42Z jilles $
+
+NOTE -- This is the original TOUR paper distributed with ash and
+does not represent the current state of the shell.  It is provided anyway
+since it provides helpful information for how the shell is structured,
+but be warned that things have changed -- the current shell is
+still under development.
+
+================================================================
+
+                       A Tour through Ash
+
+               Copyright 1989 by Kenneth Almquist.
+
+
+DIRECTORIES:  The subdirectory bltin contains commands which can
+be compiled stand-alone.  The rest of the source is in the main
+ash directory.
+
+SOURCE CODE GENERATORS:  Files whose names begin with "mk" are
+programs that generate source code.  A complete list of these
+programs is:
+
+        program         input files         generates
+        -------         -----------         ---------
+        mkbuiltins      builtins.def        builtins.h builtins.c
+        mknodes         nodetypes           nodes.h nodes.c
+        mksyntax            -               syntax.h syntax.c
+        mktokens            -               token.h
+
+There are undoubtedly too many of these.
+
+EXCEPTIONS:  Code for dealing with exceptions appears in
+exceptions.c.  The C language doesn't include exception handling,
+so I implement it using setjmp and longjmp.  The global variable
+exception contains the type of exception.  EXERROR is raised by
+calling error.  EXINT is an interrupt.
+
+INTERRUPTS:  In an interactive shell, an interrupt will cause an
+EXINT exception to return to the main command loop.  (Exception:
+EXINT is not raised if the user traps interrupts using the trap
+command.)  The INTOFF and INTON macros (defined in exception.h)
+provide uninterruptible critical sections.  Between the execution
+of INTOFF and the execution of INTON, interrupt signals will be
+held for later delivery.  INTOFF and INTON can be nested.
+
+MEMALLOC.C:  Memalloc.c defines versions of malloc and realloc
+which call error when there is no memory left.  It also defines a
+stack oriented memory allocation scheme.  Allocating off a stack
+is probably more efficient than allocation using malloc, but the
+big advantage is that when an exception occurs all we have to do
+to free up the memory in use at the time of the exception is to
+restore the stack pointer.  The stack is implemented using a
+linked list of blocks.
+
+STPUTC:  If the stack were contiguous, it would be easy to store
+strings on the stack without knowing in advance how long the
+string was going to be:
+        p = stackptr;
+        *p++ = c;       /* repeated as many times as needed */
+        stackptr = p;
+The following three macros (defined in memalloc.h) perform these
+operations, but grow the stack if you run off the end:
+        STARTSTACKSTR(p);
+        STPUTC(c, p);   /* repeated as many times as needed */
+        grabstackstr(p);
+
+We now start a top-down look at the code:
+
+MAIN.C:  The main routine performs some initialization, executes
+the user's profile if necessary, and calls cmdloop.  Cmdloop
+repeatedly parses and executes commands.
+
+OPTIONS.C:  This file contains the option processing code.  It is
+called from main to parse the shell arguments when the shell is
+invoked, and it also contains the set builtin.  The -i and -m op-
+tions (the latter turns on job control) require changes in signal
+handling.  The routines setjobctl (in jobs.c) and setinteractive
+(in trap.c) are called to handle changes to these options.
+
+PARSING:  The parser code is all in parser.c.  A recursive des-
+cent parser is used.  Syntax tables (generated by mksyntax) are
+used to classify characters during lexical analysis.  There are
+four tables:  one for normal use, one for use when inside single
+quotes and dollar single quotes, one for use when inside double
+quotes and one for use in arithmetic.  The tables are machine
+dependent because they are indexed by character variables and
+the range of a char varies from machine to machine.
+
+PARSE OUTPUT:  The output of the parser consists of a tree of
+nodes.  The various types of nodes are defined in the file node-
+types.
+
+Nodes of type NARG are used to represent both words and the con-
+tents of here documents.  An early version of ash kept the con-
+tents of here documents in temporary files, but keeping here do-
+cuments in memory typically results in significantly better per-
+formance.  It would have been nice to make it an option to use
+temporary files for here documents, for the benefit of small
+machines, but the code to keep track of when to delete the tem-
+porary files was complex and I never fixed all the bugs in it.
+(AT&T has been maintaining the Bourne shell for more than ten
+years, and to the best of my knowledge they still haven't gotten
+it to handle temporary files correctly in obscure cases.)
+
+The text field of a NARG structure points to the text of the
+word.  The text consists of ordinary characters and a number of
+special codes defined in parser.h.  The special codes are:
+
+        CTLVAR              Parameter expansion
+        CTLENDVAR           End of parameter expansion
+        CTLBACKQ            Command substitution
+        CTLBACKQ|CTLQUOTE   Command substitution inside double quotes
+        CTLARI              Arithmetic expansion
+        CTLENDARI           End of arithmetic expansion
+        CTLESC              Escape next character
+
+A variable substitution contains the following elements:
+
+        CTLVAR type name '=' [ alternative-text CTLENDVAR ]
+
+The type field is a single character specifying the type of sub-
+stitution.  The possible types are:
+
+        VSNORMAL            $var
+        VSMINUS             ${var-text}
+        VSMINUS|VSNUL       ${var:-text}
+        VSPLUS              ${var+text}
+        VSPLUS|VSNUL        ${var:+text}
+        VSQUESTION          ${var?text}
+        VSQUESTION|VSNUL    ${var:?text}
+        VSASSIGN            ${var=text}
+        VSASSIGN|VSNUL      ${var:=text}
+        VSTRIMLEFT          ${var#text}
+        VSTRIMLEFTMAX       ${var##text}
+        VSTRIMRIGHT         ${var%text}
+        VSTRIMRIGHTMAX      ${var%%text}
+        VSLENGTH            ${#var}
+        VSERROR             delayed error
+
+In addition, the type field will have the VSQUOTE flag set if the
+variable is enclosed in double quotes and the VSLINENO flag if
+LINENO is being expanded (the parameter name is the decimal line
+number).  The parameter's name comes next, terminated by an equals
+sign.  If the type is not VSNORMAL (including when it is VSLENGTH),
+then the text field in the substitution follows, terminated by a
+CTLENDVAR byte.
+
+The type VSERROR is used to allow parsing bad substitutions like
+${var[7]} and generate an error when they are expanded.
+
+Commands in back quotes are parsed and stored in a linked list.
+The locations of these commands in the string are indicated by
+CTLBACKQ and CTLBACKQ+CTLQUOTE characters, depending upon whether
+the back quotes were enclosed in double quotes.
+
+Arithmetic expansion starts with CTLARI and ends with CTLENDARI.
+
+The character CTLESC escapes the next character, so that in case
+any of the CTL characters mentioned above appear in the input,
+they can be passed through transparently.  CTLESC is also used to
+escape '*', '?', '[', and '!' characters which were quoted by the
+user and thus should not be used for file name generation.
+
+CTLESC characters have proved to be particularly tricky to get
+right.  In the case of here documents which are not subject to
+variable and command substitution, the parser doesn't insert any
+CTLESC characters to begin with (so the contents of the text
+field can be written without any processing).  Other here docu-
+ments, and words which are not subject to file name generation,
+have the CTLESC characters removed during the variable and command
+substitution phase.  Words which are subject to file name
+generation have the CTLESC characters removed as part of the file
+name phase.
+
+EXECUTION:  Command execution is handled by the following files:
+        eval.c     The top level routines.
+        redir.c    Code to handle redirection of input and output.
+        jobs.c     Code to handle forking, waiting, and job control.
+        exec.c     Code to do path searches and the actual exec sys call.
+        expand.c   Code to evaluate arguments.
+        var.c      Maintains the variable symbol table.  Called from expand.c.
+
+EVAL.C:  Evaltree recursively executes a parse tree.  The exit
+status is returned in the global variable exitstatus.  The alter-
+native entry evalbackcmd is called to evaluate commands in back
+quotes.  It saves the result in memory if the command is a buil-
+tin; otherwise it forks off a child to execute the command and
+connects the standard output of the child to a pipe.
+
+JOBS.C:  To create a process, you call makejob to return a job
+structure, and then call forkshell (passing the job structure as
+an argument) to create the process.  Waitforjob waits for a job
+to complete.  These routines take care of process groups if job
+control is defined.
+
+REDIR.C:  Ash allows file descriptors to be redirected and then
+restored without forking off a child process.  This is accom-
+plished by duplicating the original file descriptors.  The redir-
+tab structure records where the file descriptors have been dupli-
+cated to.
+
+EXEC.C:  The routine find_command locates a command, and enters
+the command in the hash table if it is not already there.  The
+third argument specifies whether it is to print an error message
+if the command is not found.  (When a pipeline is set up,
+find_command is called for all the commands in the pipeline be-
+fore any forking is done, so to get the commands into the hash
+table of the parent process.  But to make command hashing as
+transparent as possible, we silently ignore errors at that point
+and only print error messages if the command cannot be found
+later.)
+
+The routine shellexec is the interface to the exec system call.
+
+EXPAND.C:  As the routine argstr generates words by parameter
+expansion, command substitution and arithmetic expansion, it
+performs word splitting on the result.  As each word is output,
+the routine expandmeta performs file name generation (if enabled).
+
+VAR.C:  Variables are stored in a hash table.  Probably we should
+switch to extensible hashing.  The variable name is stored in the
+same string as the value (using the format "name=value") so that
+no string copying is needed to create the environment of a com-
+mand.  Variables which the shell references internally are preal-
+located so that the shell can reference the values of these vari-
+ables without doing a lookup.
+
+When a program is run, the code in eval.c sticks any environment
+variables which precede the command (as in "PATH=xxx command") in
+the variable table as the simplest way to strip duplicates, and
+then calls "environment" to get the value of the environment.
+
+BUILTIN COMMANDS:  The procedures for handling these are scat-
+tered throughout the code, depending on which location appears
+most appropriate.  They can be recognized because their names al-
+ways end in "cmd".  The mapping from names to procedures is
+specified in the file builtins.def, which is processed by the
+mkbuiltins command.
+
+A builtin command is invoked with argc and argv set up like a
+normal program.  A builtin command is allowed to overwrite its
+arguments.  Builtin routines can call nextopt to do option pars-
+ing.  This is kind of like getopt, but you don't pass argc and
+argv to it.  Builtin routines can also call error.  This routine
+normally terminates the shell (or returns to the main command
+loop if the shell is interactive), but when called from a non-
+special builtin command it causes the builtin command to
+terminate with an exit status of 2.
+
+The directory bltins contains commands which can be compiled in-
+dependently but can also be built into the shell for efficiency
+reasons.  The header file bltin.h takes care of most of the
+differences between the ash and the stand-alone environment.
+The user should call the main routine "main", and #define main to
+be the name of the routine to use when the program is linked into
+ash.  This #define should appear before bltin.h is included;
+bltin.h will #undef main if the program is to be compiled
+stand-alone. A similar approach is used for a few utilities from
+bin and usr.bin.
+
+CD.C:  This file defines the cd and pwd builtins.
+
+SIGNALS:  Trap.c implements the trap command.  The routine set-
+signal figures out what action should be taken when a signal is
+received and invokes the signal system call to set the signal ac-
+tion appropriately.  When a signal that a user has set a trap for
+is caught, the routine "onsig" sets a flag.  The routine dotrap
+is called at appropriate points to actually handle the signal.
+When an interrupt is caught and no trap has been set for that
+signal, the routine "onint" in error.c is called.
+
+OUTPUT:  Ash uses its own output routines.  There are three out-
+put structures allocated.  "Output" represents the standard out-
+put, "errout" the standard error, and "memout" contains output
+which is to be stored in memory.  This last is used when a buil-
+tin command appears in backquotes, to allow its output to be col-
+lected without doing any I/O through the UNIX operating system.
+The variables out1 and out2 normally point to output and errout,
+respectively, but they are set to point to memout when appropri-
+ate inside backquotes.
+
+INPUT:  The basic input routine is pgetc, which reads from the
+current input file.  There is a stack of input files; the current
+input file is the top file on this stack.  The code allows the
+input to come from a string rather than a file.  (This is for the
+-c option and the "." and eval builtin commands.)  The global
+variable plinno is saved and restored when files are pushed and
+popped from the stack.  The parser routines store the number of
+the current line in this variable.
+
+DEBUGGING:  If DEBUG is defined in shell.h, then the shell will
+write debugging information to the file $HOME/trace.  Most of
+this is done using the TRACE macro, which takes a set of printf
+arguments inside two sets of parenthesis.  Example:
+"TRACE(("n=%d0, n))".  The double parenthesis are necessary be-
+cause the preprocessor can't handle functions with a variable
+number of arguments.  Defining DEBUG also causes the shell to
+generate a core dump if it is sent a quit signal.  The tracing
+code is in show.c.