diff options
Diffstat (limited to 'src/TOUR')
| -rw-r--r-- | src/TOUR | 357 |
1 files changed, 357 insertions, 0 deletions
diff --git a/src/TOUR b/src/TOUR new file mode 100644 index 0000000..f5c00c4 --- /dev/null +++ b/src/TOUR @@ -0,0 +1,357 @@ +# $NetBSD: TOUR,v 1.8 1996/10/16 14:24:56 christos Exp $ +# @(#)TOUR 8.1 (Berkeley) 5/31/93 + +NOTE -- This is the original TOUR paper distributed with ash and +does not represent the current state of the shell. It is provided anyway +since it provides helpful information for how the shell is structured, +but be warned that things have changed -- the current shell is +still under development. + +================================================================ + + A Tour through Ash + + Copyright 1989 by Kenneth Almquist. + + +DIRECTORIES: The subdirectory bltin contains commands which can +be compiled stand-alone. The rest of the source is in the main +ash directory. + +SOURCE CODE GENERATORS: Files whose names begin with "mk" are +programs that generate source code. A complete list of these +programs is: + + program intput files generates + ------- ------------ --------- + mkbuiltins builtins builtins.h builtins.c + mkinit *.c init.c + mknodes nodetypes nodes.h nodes.c + mksignames - signames.h signames.c + mksyntax - syntax.h syntax.c + mktokens - token.h + bltin/mkexpr unary_op binary_op operators.h operators.c + +There are undoubtedly too many of these. Mkinit searches all the +C source files for entries looking like: + + INIT { + x = 1; /* executed during initialization */ + } + + RESET { + x = 2; /* executed when the shell does a longjmp + back to the main command loop */ + } + + SHELLPROC { + x = 3; /* executed when the shell runs a shell procedure */ + } + +It pulls this code out into routines which are when particular +events occur. The intent is to improve modularity by isolating +the information about which modules need to be explicitly +initialized/reset within the modules themselves. + +Mkinit recognizes several constructs for placing declarations in +the init.c file. + INCLUDE "file.h" +includes a file. The storage class MKINIT makes a declaration +available in the init.c file, for example: + MKINIT int funcnest; /* depth of function calls */ +MKINIT alone on a line introduces a structure or union declara- +tion: + MKINIT + struct redirtab { + short renamed[10]; + }; +Preprocessor #define statements are copied to init.c without any +special action to request this. + +INDENTATION: The ash source is indented in multiples of six +spaces. The only study that I have heard of on the subject con- +cluded that the optimal amount to indent is in the range of four +to six spaces. I use six spaces since it is not too big a jump +from the widely used eight spaces. If you really hate six space +indentation, use the adjind (source included) program to change +it to something else. + +EXCEPTIONS: Code for dealing with exceptions appears in +exceptions.c. The C language doesn't include exception handling, +so I implement it using setjmp and longjmp. The global variable +exception contains the type of exception. EXERROR is raised by +calling error. EXINT is an interrupt. EXSHELLPROC is an excep- +tion which is raised when a shell procedure is invoked. The pur- +pose of EXSHELLPROC is to perform the cleanup actions associated +with other exceptions. After these cleanup actions, the shell +can interpret a shell procedure itself without exec'ing a new +copy of the shell. + +INTERRUPTS: In an interactive shell, an interrupt will cause an +EXINT exception to return to the main command loop. (Exception: +EXINT is not raised if the user traps interrupts using the trap +command.) The INTOFF and INTON macros (defined in exception.h) +provide uninterruptable critical sections. Between the execution +of INTOFF and the execution of INTON, interrupt signals will be +held for later delivery. INTOFF and INTON can be nested. + +MEMALLOC.C: Memalloc.c defines versions of malloc and realloc +which call error when there is no memory left. It also defines a +stack oriented memory allocation scheme. Allocating off a stack +is probably more efficient than allocation using malloc, but the +big advantage is that when an exception occurs all we have to do +to free up the memory in use at the time of the exception is to +restore the stack pointer. The stack is implemented using a +linked list of blocks. + +STPUTC: If the stack were contiguous, it would be easy to store +strings on the stack without knowing in advance how long the +string was going to be: + p = stackptr; + *p++ = c; /* repeated as many times as needed */ + stackptr = p; +The folloing three macros (defined in memalloc.h) perform these +operations, but grow the stack if you run off the end: + STARTSTACKSTR(p); + STPUTC(c, p); /* repeated as many times as needed */ + grabstackstr(p); + +We now start a top-down look at the code: + +MAIN.C: The main routine performs some initialization, executes +the user's profile if necessary, and calls cmdloop. Cmdloop is +repeatedly parses and executes commands. + +OPTIONS.C: This file contains the option processing code. It is +called from main to parse the shell arguments when the shell is +invoked, and it also contains the set builtin. The -i and -j op- +tions (the latter turns on job control) require changes in signal +handling. The routines setjobctl (in jobs.c) and setinteractive +(in trap.c) are called to handle changes to these options. + +PARSING: The parser code is all in parser.c. A recursive des- +cent parser is used. Syntax tables (generated by mksyntax) are +used to classify characters during lexical analysis. There are +three tables: one for normal use, one for use when inside single +quotes, and one for use when inside double quotes. The tables +are machine dependent because they are indexed by character vari- +ables and the range of a char varies from machine to machine. + +PARSE OUTPUT: The output of the parser consists of a tree of +nodes. The various types of nodes are defined in the file node- +types. + +Nodes of type NARG are used to represent both words and the con- +tents of here documents. An early version of ash kept the con- +tents of here documents in temporary files, but keeping here do- +cuments in memory typically results in significantly better per- +formance. It would have been nice to make it an option to use +temporary files for here documents, for the benefit of small +machines, but the code to keep track of when to delete the tem- +porary files was complex and I never fixed all the bugs in it. +(AT&T has been maintaining the Bourne shell for more than ten +years, and to the best of my knowledge they still haven't gotten +it to handle temporary files correctly in obscure cases.) + +The text field of a NARG structure points to the text of the +word. The text consists of ordinary characters and a number of +special codes defined in parser.h. The special codes are: + + CTLVAR Variable substitution + CTLENDVAR End of variable substitution + CTLBACKQ Command substitution + CTLBACKQ|CTLQUOTE Command substitution inside double quotes + CTLESC Escape next character + +A variable substitution contains the following elements: + + CTLVAR type name '=' [ alternative-text CTLENDVAR ] + +The type field is a single character specifying the type of sub- +stitution. The possible types are: + + VSNORMAL $var + VSMINUS ${var-text} + VSMINUS|VSNUL ${var:-text} + VSPLUS ${var+text} + VSPLUS|VSNUL ${var:+text} + VSQUESTION ${var?text} + VSQUESTION|VSNUL ${var:?text} + VSASSIGN ${var=text} + VSASSIGN|VSNUL ${var=text} + +In addition, the type field will have the VSQUOTE flag set if the +variable is enclosed in double quotes. The name of the variable +comes next, terminated by an equals sign. If the type is not +VSNORMAL, then the text field in the substitution follows, ter- +minated by a CTLENDVAR byte. + +Commands in back quotes are parsed and stored in a linked list. +The locations of these commands in the string are indicated by +CTLBACKQ and CTLBACKQ+CTLQUOTE characters, depending upon whether +the back quotes were enclosed in double quotes. + +The character CTLESC escapes the next character, so that in case +any of the CTL characters mentioned above appear in the input, +they can be passed through transparently. CTLESC is also used to +escape '*', '?', '[', and '!' characters which were quoted by the +user and thus should not be used for file name generation. + +CTLESC characters have proved to be particularly tricky to get +right. In the case of here documents which are not subject to +variable and command substitution, the parser doesn't insert any +CTLESC characters to begin with (so the contents of the text +field can be written without any processing). Other here docu- +ments, and words which are not subject to splitting and file name +generation, have the CTLESC characters removed during the vari- +able and command substitution phase. Words which are subject +splitting and file name generation have the CTLESC characters re- +moved as part of the file name phase. + +EXECUTION: Command execution is handled by the following files: + eval.c The top level routines. + redir.c Code to handle redirection of input and output. + jobs.c Code to handle forking, waiting, and job control. + exec.c Code to to path searches and the actual exec sys call. + expand.c Code to evaluate arguments. + var.c Maintains the variable symbol table. Called from expand.c. + +EVAL.C: Evaltree recursively executes a parse tree. The exit +status is returned in the global variable exitstatus. The alter- +native entry evalbackcmd is called to evaluate commands in back +quotes. It saves the result in memory if the command is a buil- +tin; otherwise it forks off a child to execute the command and +connects the standard output of the child to a pipe. + +JOBS.C: To create a process, you call makejob to return a job +structure, and then call forkshell (passing the job structure as +an argument) to create the process. Waitforjob waits for a job +to complete. These routines take care of process groups if job +control is defined. + +REDIR.C: Ash allows file descriptors to be redirected and then +restored without forking off a child process. This is accom- +plished by duplicating the original file descriptors. The redir- +tab structure records where the file descriptors have be dupli- +cated to. + +EXEC.C: The routine find_command locates a command, and enters +the command in the hash table if it is not already there. The +third argument specifies whether it is to print an error message +if the command is not found. (When a pipeline is set up, +find_command is called for all the commands in the pipeline be- +fore any forking is done, so to get the commands into the hash +table of the parent process. But to make command hashing as +transparent as possible, we silently ignore errors at that point +and only print error messages if the command cannot be found +later.) + +The routine shellexec is the interface to the exec system call. + +EXPAND.C: Arguments are processed in three passes. The first +(performed by the routine argstr) performs variable and command +substitution. The second (ifsbreakup) performs word splitting +and the third (expandmeta) performs file name generation. If the +"/u" directory is simulated, then when "/u/username" is replaced +by the user's home directory, the flag "didudir" is set. This +tells the cd command that it should print out the directory name, +just as it would if the "/u" directory were implemented using +symbolic links. + +VAR.C: Variables are stored in a hash table. Probably we should +switch to extensible hashing. The variable name is stored in the +same string as the value (using the format "name=value") so that +no string copying is needed to create the environment of a com- +mand. Variables which the shell references internally are preal- +located so that the shell can reference the values of these vari- +ables without doing a lookup. + +When a program is run, the code in eval.c sticks any environment +variables which precede the command (as in "PATH=xxx command") in +the variable table as the simplest way to strip duplicates, and +then calls "environment" to get the value of the environment. +There are two consequences of this. First, if an assignment to +PATH precedes the command, the value of PATH before the assign- +ment must be remembered and passed to shellexec. Second, if the +program turns out to be a shell procedure, the strings from the +environment variables which preceded the command must be pulled +out of the table and replaced with strings obtained from malloc, +since the former will automatically be freed when the stack (see +the entry on memalloc.c) is emptied. + +BUILTIN COMMANDS: The procedures for handling these are scat- +tered throughout the code, depending on which location appears +most appropriate. They can be recognized because their names al- +ways end in "cmd". The mapping from names to procedures is +specified in the file builtins, which is processed by the mkbuil- +tins command. + +A builtin command is invoked with argc and argv set up like a +normal program. A builtin command is allowed to overwrite its +arguments. Builtin routines can call nextopt to do option pars- +ing. This is kind of like getopt, but you don't pass argc and +argv to it. Builtin routines can also call error. This routine +normally terminates the shell (or returns to the main command +loop if the shell is interactive), but when called from a builtin +command it causes the builtin command to terminate with an exit +status of 2. + +The directory bltins contains commands which can be compiled in- +dependently but can also be built into the shell for efficiency +reasons. The makefile in this directory compiles these programs +in the normal fashion (so that they can be run regardless of +whether the invoker is ash), but also creates a library named +bltinlib.a which can be linked with ash. The header file bltin.h +takes care of most of the differences between the ash and the +stand-alone environment. The user should call the main routine +"main", and #define main to be the name of the routine to use +when the program is linked into ash. This #define should appear +before bltin.h is included; bltin.h will #undef main if the pro- +gram is to be compiled stand-alone. + +CD.C: This file defines the cd and pwd builtins. The pwd com- +mand runs /bin/pwd the first time it is invoked (unless the user +has already done a cd to an absolute pathname), but then +remembers the current directory and updates it when the cd com- +mand is run, so subsequent pwd commands run very fast. The main +complication in the cd command is in the docd command, which +resolves symbolic links into actual names and informs the user +where the user ended up if he crossed a symbolic link. + +SIGNALS: Trap.c implements the trap command. The routine set- +signal figures out what action should be taken when a signal is +received and invokes the signal system call to set the signal ac- +tion appropriately. When a signal that a user has set a trap for +is caught, the routine "onsig" sets a flag. The routine dotrap +is called at appropriate points to actually handle the signal. +When an interrupt is caught and no trap has been set for that +signal, the routine "onint" in error.c is called. + +OUTPUT: Ash uses it's own output routines. There are three out- +put structures allocated. "Output" represents the standard out- +put, "errout" the standard error, and "memout" contains output +which is to be stored in memory. This last is used when a buil- +tin command appears in backquotes, to allow its output to be col- +lected without doing any I/O through the UNIX operating system. +The variables out1 and out2 normally point to output and errout, +respectively, but they are set to point to memout when appropri- +ate inside backquotes. + +INPUT: The basic input routine is pgetc, which reads from the +current input file. There is a stack of input files; the current +input file is the top file on this stack. The code allows the +input to come from a string rather than a file. (This is for the +-c option and the "." and eval builtin commands.) The global +variable plinno is saved and restored when files are pushed and +popped from the stack. The parser routines store the number of +the current line in this variable. + +DEBUGGING: If DEBUG is defined in shell.h, then the shell will +write debugging information to the file $HOME/trace. Most of +this is done using the TRACE macro, which takes a set of printf +arguments inside two sets of parenthesis. Example: +"TRACE(("n=%d0, n))". The double parenthesis are necessary be- +cause the preprocessor can't handle functions with a variable +number of arguments. Defining DEBUG also causes the shell to +generate a core dump if it is sent a quit signal. The tracing +code is in show.c. |