1 /* $XConsortium: README-DEVELOPER /main/3 1996/07/15 14:12:13 drk $ */
3 IMPORTANT!!! PLEASE READ ON!!
4 ------------------------------
6 The intention of this file is to outline the procedure to use, anytime
7 a new update of the ksh-93 source is received. When you receive the
8 new source package, you will first need to do the following:
11 1) copy the packing into this directory
12 2) uudecode the package
13 3) uncompress the package
14 4) un-tar the package into ksh93-base (you may need to use
15 cpio, if that is what Korn used to create his package).
17 Once the source ball has been un-tar'ed, go down into the hierarchy,
18 until you see a directory called 'ship' ; don't go down into the 'ship'
19 directory, simply stop at its parent. Dave Korn uses his own mechanism
20 for creating the actual ksh-93 sources. It's not like you can simply
21 look at what is in the 'ship' directory. Instead, you need to run his
22 'shipin' tool, which takes his base source ball, and applies all of his
23 patches, thus generating the source tree. To start this process, which
24 will result in the latest source files being generated, issue the following
27 <First, log onto a 10.0 build machine, such as hpcvusp>
30 nohup /bin/ksh ship/shipin -u -L&
33 When this completes, you will notice that the current directory now
34 contains a series of other subdirectories; the only ones we really care
35 about now are the 'man', 'ship' and the 'src' subdirectories.
36 The trick now is to determine which of the files in
37 these subdirectories have changed, so that you can update the real
38 dtksh/ksh93 source tree. You will want to copy into the dtksh/ksh93/ship
39 directory the newest versions of the following files:
47 Go into the 'shipin' file, and locate the several places where it attempts
48 to mail out information (I look for the word 'mail', until I find what
49 looks like the correct places). Remove this code, since we don't want
50 a ton of mail messages happening each time we build.
52 Within the 'ship' directory, you will notice a series of subdirectories;
53 each one of these corresponds to one of the pieces making up the ksh93
54 client (i.e. there is one for each library: libast, libcmd, libvdelta, etc).
55 Within each of these subdirectories is another subdirectory, whose name
56 represents the date that source depot was created (i.e. 950619). You will
57 need to recreate these dated directories, along with their contents, within
58 your dtksh tree (under dtksh/ksh93/ship). Within each dated directory,
59 copy over (and check in) the items, copyright, UNCRATED and owner files.
60 Check in a dummy 'base' file also. Doing these steps will allow the ksh
61 build to proceed, without it ever again attempting to uncrate new ksh93
62 sources. The 'items' file is particularly important, since it indicates
63 which other libraries the indicated piece is dependent upon.
67 In /clone/fred/DtkshTools I have a collection of scripts which make it easier
68 to update a dtksh tree with new ksh sources. The steps for determining which
69 files have changed are as follows:
71 1) First get a list of all of the files in the new source tree. I do this
72 with the following steps:
74 cd ksh93-base (This is the tree you just untar'ed)
75 find src -print > srcList
77 This gives you a complete list of all of the source files.
79 2) Use this list to determine which source files are new or have changed:
81 /clone/fred/DtkshTools/CheckKshSrc dtksh ksh93-base srcList > diffList
83 The output generated by the above script will tell you which files have
84 changed, which have not changed, and which have been added.
86 3) You now need to take the diffList, and modify it so that it only
87 contains the name of new or changed files within the ksh93 source
90 /clone/fred/DtkshTools/CreateTrimList diffList > trimList
92 4) Now, we will delete the files in the dtksh tree which we are going
93 to replace; we do this before we copy over the new versions:
95 /clone/fred/DtkshTools/TrimOldSrcs /clone/fred/dtksh trimList
97 5) We now need to check out (Rcs -l) the files we are going to replace:
99 /clone/fred/DtkshTools/CheckOutSrcs /clone/fred/dtksh trimList
101 If at any time you want to get a complete list of checked out files
102 in your dtksh tree, issue the following commands:
104 find /clone/fred/dtksh -type d > dirList
105 /clone/fred/DtkshTools/RunRcslocks dirList
107 6) We are now ready to copy the new source files out of the ksh93-base
108 directory, and into the dtksh directory:
111 /clone/fred/DtkshTools/CopyNewSrcs ksh93-base dtksh trimList
115 You're not done yet, however. We have found it necessary to modify
116 several of the stock ksh sources, Makefiles and Mamfiles, to accomodate
117 problems we have encountered, or to enable options not turned on by
118 default, such as multibyte support. You will need to make sure that
119 these changes get rolled back in, before checking in the new source.
120 The following source files need to have changes added:
122 src/cmd/ksh93/Makefile (*** THIS STEP IS OBSOLETE ***)
123 ----------------------
128 src/cmd/ksh93/Mamfile (*** THIS STEP IS OBSOLETE ***)
129 ---------------------
131 Change: setv MULTIBYTE -UMULTIBYTE
132 To: setv MULTIBYTE -DMULTIBYTE
134 src/cmd/ksh93/OPTIONS (*** THIS STEP IS OBSOLETE ***)
135 ---------------------
142 The following two files must change, to prevent the ksh source from linking
143 in its own malloc package. This creates problems on the IBM platform, due
144 to the way their libraries are built. As it turns out, the IBM Xt library
145 already has all of its malloc/free/etc calls resolved to their standard
146 libc malloc functions. If ksh defines its own malloc package, then dtksh
147 ends of effectively linked simultaneously to two different malloc packages.
148 This rapidly leads to a core dump, due to cross contamination of the
149 memory pools (i.e. A value obtained by XtMalloc() passed to free() ). As
150 it turns out, this also creates problems on the HP platform, as was
151 recently discovered (Note that it appears that the problem which was
152 seen on the HP platform was due to a defect in the ksh93 memory allocation
153 code; this defect had not yet been fixed in the ksh93-d release. Until
154 the time that this is fixed, we must continue to be sure to not link
155 in ksh93's malloc package).
157 src/lib/libast/Makefile
158 -----------------------
160 Remove: calloc.c malloc.c from the list of sources
162 src/lib/libast/Mamfile
163 ----------------------
164 Remove: calloc.o malloc.o from the archive line; DO NOT REMOVE
165 ANY OF THE OTHER REFERENCES TO malloc OR calloc, AS
166 THIS WILL CAUSE THE BUILD TO BREAK.
170 Another file which needs special handling is builtins.c, which is in
171 the base dtksh directory. This file was first created by taking the
172 file 'dtksh/ksh93/src/cmd/ksh93/data/builtins.c', and adding into its
173 list of builtin commands, the list we added for X, Xt, Xm and Dt. The
174 best way to update this file is to first goto 'dtksh/ksh93/src/cmd/ksh93/data'
175 and do an Rcsdiff, to find out what changes Dave Korn made. You will
176 then make those same changes to the dtksh/builtins.c file.
181 Fold in any changes made to dtksh/ksh93/src/cmd/ksh93/data/builtins.c
185 Another file which needs special handling is aliases.c, which is in
186 the base dtksh directory. This file was first created by taking the
187 file 'dtksh/ksh93/src/cmd/ksh93/data/aliases.c', and adding into its
188 list of commands aliases, the list we added for X, Xt, Xm and Dt. The
189 best way to update this file is to first goto 'dtksh/ksh93/src/cmd/ksh93/data'
190 and do an Rcsdiff, to find out what changes Dave Korn made. You will
191 then make those same changes to the dtksh/aliases.c file.
196 Fold in any changes made to dtksh/ksh93/src/cmd/ksh93/data/aliases.c
199 HP-SPECIFIC Ksh-93 SOURCE MODIFICATIONS:
200 ========================================
202 The dtksh code requires a handful of changes to be made to the ksh93
203 source itself. Most of these changes relate to either localization
204 customizations, additional multibyte support, or fixes for environment
205 variable mirroring. For each of the files listed below, two versions
206 have been checked into the RCS vaults: the original (unaltered) version,
207 and the version with our customizations. This will make it easy to
208 determine the changes we have made, so that the next time we receive
209 new code from Korn, it is a relatively simple task to again merge in the
210 changes. Note that all of the changes relating to multibyte support
211 have been passed onto David Korn, and that hopefully they will show up
212 in the official source sometime in the future (after the ksh93-d release).
217 The ksh (and dtksh) executables use a special client (suid_exec) when
218 trying to execute an suid script; this special client makes sure that
219 you really have permission to run the suid script. The client, as
220 supplied by David Korn, has hardcoded assumptions about where it lives
221 (/etc) and where the valid (and secure) places are where shells can
222 live. We need to change this so that it knows the client lives in
223 the cde tree, and that in addition to the normal 'safe' shell locations,
224 /usr/dt/bin is also a safe location. To do this, the following changes
225 must be made, each time new ksh source is received:
227 dtksh/ksh93/src/cmd/ksh93/data/msg.c
228 ------------------------------------
232 e_sysprofile = "/etc/profile";
233 e_suidprofile = "/etc/suid_profile";
234 e_suidexec = "/etc/suid_exec";
238 e_sysprofile = PROFILEPATH;
239 e_suidprofile = SUIDPROFILEPATH;
240 e_suidexec = SUIDEXECPATH;
243 dtksh/ksh93/src/cmd/ksh93/sh/suid_exec.c
244 ----------------------------------------
248 #define THISPROG "/etc/suid_exec"
252 #define THISPROG SUIDEXECPATH
254 Also, search for the block of code like the following (around line 256):
256 if (shell == 0 || !endsh(shell) || (
257 !in_dir("/bin",shell) &&
258 !in_dir("/usr/bin",shell) &&
260 And change it as follows:
262 if (shell == 0 || !endsh(shell) || (
263 !in_dir(CDE_INSTALLATION_TOP"/bin",shell) &&
264 !in_dir("/bin",shell) &&
265 !in_dir("/usr/bin",shell) &&
270 During CDE development, it was discovered that some shared library
271 implementations (i.e. at least IBM) bound a shared library to all
272 of its dependencies at the time that the library was build. This
273 created problems for libraries which made getenv() or putenv()
274 calls, because we expected those calls to be resolved against the
275 getenv() and putenv() calls provided in the ksh93 libraries, not
276 against the standard libc versions. This was a problem because
277 ksh keeps track of its own environment, and does not pass the
278 information on through to the standard libc environment. The result
279 was that certain key pieces of information (i.e LANG) might be set
280 by a shell script, but when calls where made to shared library functions,
281 they would not see this setting. The fix is to modify the ksh93 code
282 to *also* pass along any environment settings onto the libc functions;
283 we call this 'environment variable mirroring'. We won't describe the
284 exact changes here; you can find them by diff'ing the last two revisions
285 for the following files:
287 src/cmd/ksh93/include/nval.h (New field added to Namval structure)
288 src/cmd/ksh93/sh/name.c (Mirroring code implemented here)
293 There are a handful of multibyte and localization changes which we
294 must make. Some of them relate to adding message catalog support
295 to ksh, while others relate to making changes so that Japanese SJIS character
296 will work. The files which must be modified are:
298 src/cmd/ksh93/sh/init.c (Message cat and file desc locking/unlocking)
299 src/cmd/ksh93/sh/lex.c (Japanese SJIS processing fixes)
300 src/cmd/ksh93/sh/macro.c (Multibyte fixes)
301 src/cmd/ksh93/bltins/print.c (SJIS multibyte fix to echo/print command)
302 src/cmd/ksh93/sh/userinit.c (Add call to setlocale() in sh_userinit())
307 During CDE development, USL needed to add an 'ifndef __USLC__' around
308 a certain define. The file which needs the change is:
310 src/lib/libast/sfio/sfhdr.h
315 There appears to be a bug in the ksh93 build process, which shows up
316 on the HP platform. David Korn was unable to find the cause, so we
317 need to modify one of his build scripts:
319 ksh93/src/cmd/ksh93/features/options.sh
320 ---------------------------------------
322 Towards the top of the script, there is the following set of
325 : get the option settings from the options file
328 You need to modify this so that it is like the following:
330 : get the option settings from the options file
331 if [ -z "$OPTIONS" ] ; then
337 =======================
339 Each time a new release is received from Dave Korn, we also need to copy
340 his latest ksh93 man page from src/cmd/ksh93/sh.1 into
341 /x/cdesrc/doc/cde1/man/man1/ksh93. You will need to make one change to
342 the man page, so that it prints out the ksh information, instead of sh
343 information. The change is one approximately line 7.
348 .nr Z 0 \" set to 1 when command name is ksh rather than sh
353 .nr Z 1 \" set to 1 when command name is ksh rather than sh
357 One last note. There is a shell script called 'MakeClean' in the root
358 directory of the dtksh tree. This script cleans out files in the ksh93
359 portion of the tree, which will force a complete build to occur. Since
360 the ksh93 portion does not use Imake (it uses Mam or something bazarre
361 like that), we needed to jury rig something which would act like our
362 normal 'make clean' . If you discover any other ksh93 files which need
363 to be removed during a 'make clean' operation, simply add another line
364 to the MakeClean script. A good indication that a file needs to be
365 added is if you make a clone from one of the source trees, do a 'make clean',
366 start a 'make', and somewhere during the building of the ksh93 sources,
367 you see an error message telling you that you did not have permission to
368 overwrite something within your clone tree. The ksh93 build process builds
369 alot of files on the fly, and moves things around to other directories, so
370 these file typically need to be cleared out before doing a complete make.