1 Design document for the unified scheme data
2 ===========================================
4 How are things connected?
5 -------------------------
7 The unified scheme takes all its data from the build.info files seen
8 throughout the source tree. These files hold the minimum information
9 needed to build end product files from diverse sources. See the
10 section on build.info files below.
12 From the information in build.info files, Configure builds up an
13 information database as a hash table called %unified_info, which is
14 stored in configdata.pm, found at the top of the build tree (which may
15 or may not be the same as the source tree).
17 Configurations/common.tmpl uses the data from %unified_info to
18 generate the rules for building end product files as well as
19 intermediary files with the help of a few functions found in the
20 build-file templates. See the section on build-file templates further
21 down for more information.
26 As mentioned earlier, build.info files are meant to hold the minimum
27 information needed to build output files, and therefore only (with a
28 few possible exceptions [1]) have information about end products (such
29 as scripts, library files and programs) and source files (such as C
30 files, C header files, assembler files, etc). Intermediate files such
31 as object files are rarely directly referred to in build.info files (and
32 when they are, it's always with the file name extension .o), they are
33 inferred by Configure. By the same rule of minimalism, end product
34 file name extensions (such as .so, .a, .exe, etc) are never mentioned
35 in build.info. Their file name extensions will be inferred by the
36 build-file templates, adapted for the platform they are meant for (see
37 sections on %unified_info and build-file templates further down).
39 The variables PROGRAMS, LIBS, MODULES and SCRIPTS are used to declare
40 end products. There are variants for them with '_NO_INST' as suffix
41 (PROGRAM_NO_INST etc) to specify end products that shouldn't get
44 The variables SOURCE, DEPEND, INCLUDE and DEFINE are indexed by a
45 produced file, and their values are the source used to produce that
46 particular produced file, extra dependencies, include directories
47 needed, or C macros to be defined.
49 All their values in all the build.info throughout the source tree are
50 collected together and form a set of programs, libraries, modules and
51 scripts to be produced, source files, dependencies, etc etc etc.
53 Let's have a pretend example, a very limited contraption of OpenSSL,
54 composed of the program 'apps/openssl', the libraries 'libssl' and
55 'libcrypto', an module 'engines/ossltest' and their sources and
60 INCLUDE[libcrypto]=include
61 INCLUDE[libssl]=include
62 DEPEND[libssl]=libcrypto
64 This is the top directory build.info file, and it tells us that two
65 libraries are to be built, the include directory 'include/' shall be
66 used throughout when building anything that will end up in each
67 library, and that the library 'libssl' depend on the library
68 'libcrypto' to function properly.
72 SOURCE[openssl]=openssl.c
73 INCLUDE[openssl]=.. ../include
74 DEPEND[openssl]=../libssl
76 This is the build.info file in 'apps/', one may notice that all file
77 paths mentioned are relative to the directory the build.info file is
78 located in. This one tells us that there's a program to be built
79 called 'apps/openssl' (the file name extension will depend on the
80 platform and is therefore not mentioned in the build.info file). It's
81 built from one source file, 'apps/openssl.c', and building it requires
82 the use of '.' and 'include' include directories (both are declared
83 from the point of view of the 'apps/' directory), and that the program
84 depends on the library 'libssl' to function properly.
88 SOURCE[../libcrypto]=aes.c evp.c cversion.c
89 DEPEND[cversion.o]=buildinf.h
91 GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
92 DEPEND[buildinf.h]=../Makefile
93 DEPEND[../util/mkbuildinf.pl]=../util/Foo.pm
95 This is the build.info file in 'crypto', and it tells us a little more
96 about what's needed to produce 'libcrypto'. LIBS is used again to
97 declare that 'libcrypto' is to be produced. This declaration is
98 really unnecessary as it's already mentioned in the top build.info
99 file, but can make the info file easier to understand. This is to
100 show that duplicate information isn't an issue.
102 This build.info file informs us that 'libcrypto' is built from a few
103 source files, 'crypto/aes.c', 'crypto/evp.c' and 'crypto/cversion.c'.
104 It also shows us that building the object file inferred from
105 'crypto/cversion.c' depends on 'crypto/buildinf.h'. Finally, it
106 also shows the possibility to declare how some files are generated
107 using some script, in this case a perl script, and how such scripts
108 can be declared to depend on other files, in this case a perl module.
110 Two things are worth an extra note:
112 'DEPEND[cversion.o]' mentions an object file. DEPEND indexes is the
113 only location where it's valid to mention them
117 SOURCE[../libssl]=tls.c
119 This is the build.info file in 'ssl/', and it tells us that the
120 library 'libssl' is built from the source file 'ssl/tls.c'.
124 SOURCE[dasync]=e_dasync.c
125 DEPEND[dasync]=../libcrypto
126 INCLUDE[dasync]=../include
128 MODULES_NO_INST=ossltest
129 SOURCE[ossltest]=e_ossltest.c
130 DEPEND[ossltest]=../libcrypto.a
131 INCLUDE[ossltest]=../include
133 This is the build.info file in 'engines/', telling us that two modules
134 called 'engines/dasync' and 'engines/ossltest' shall be built, that
135 dasync's source is 'engines/e_dasync.c' and ossltest's source is
136 'engines/e_ossltest.c' and that the include directory 'include/' may
137 be used when building anything that will be part of these modules.
138 Also, both modules depend on the library 'libcrypto' to function
139 properly. ossltest is explicitly linked with the static variant of
140 the library 'libcrypto'. Finally, only dasync is being installed, as
141 ossltest is only for internal testing.
143 When Configure digests these build.info files, the accumulated
144 information comes down to this:
146 LIBS=libcrypto libssl
147 SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c
148 DEPEND[crypto/cversion.o]=crypto/buildinf.h
149 INCLUDE[libcrypto]=include
150 SOURCE[libssl]=ssl/tls.c
151 INCLUDE[libssl]=include
152 DEPEND[libssl]=libcrypto
154 PROGRAMS=apps/openssl
155 SOURCE[apps/openssl]=apps/openssl.c
156 INCLUDE[apps/openssl]=. include
157 DEPEND[apps/openssl]=libssl
159 MODULES=engines/dasync
160 SOURCE[engines/dasync]=engines/e_dasync.c
161 DEPEND[engines/dasync]=libcrypto
162 INCLUDE[engines/dasync]=include
164 MODULES_NO_INST=engines/ossltest
165 SOURCE[engines/ossltest]=engines/e_ossltest.c
166 DEPEND[engines/ossltest]=libcrypto.a
167 INCLUDE[engines/ossltest]=include
169 GENERATE[crypto/buildinf.h]=util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
170 DEPEND[crypto/buildinf.h]=Makefile
171 DEPEND[util/mkbuildinf.pl]=util/Foo.pm
174 A few notes worth mentioning:
176 LIBS may be used to declare routine libraries only.
178 PROGRAMS may be used to declare programs only.
180 MODULES may be used to declare modules only.
182 The indexes for SOURCE must only be end product files, such as
183 libraries, programs or modules. The values of SOURCE variables must
184 only be source files (possibly generated).
186 INCLUDE and DEPEND shows a relationship between different files
187 (usually produced files) or between files and directories, such as a
188 program depending on a library, or between an object file and some
191 When Configure processes the build.info files, it will take it as
192 truth without question, and will therefore perform very few checks.
193 If the build tree is separate from the source tree, it will assume
194 that all built files and up in the build directory and that all source
195 files are to be found in the source tree, if they can be found there.
196 Configure will assume that source files that can't be found in the
197 source tree (such as 'crypto/bildinf.h' in the example above) are
198 generated and will be found in the build tree.
201 The %unified_info database
202 --------------------------
204 The information in all the build.info get digested by Configure and
205 collected into the %unified_info database, divided into the following
208 depends => a hash table containing 'file' => [ 'dependency' ... ]
209 pairs. These are directly inferred from the DEPEND
210 variables in build.info files.
212 modules => a list of modules. These are directly inferred from
213 the MODULES variable in build.info files.
215 generate => a hash table containing 'file' => [ 'generator' ... ]
216 pairs. These are directly inferred from the GENERATE
217 variables in build.info files.
219 includes => a hash table containing 'file' => [ 'include' ... ]
220 pairs. These are directly inferred from the INCLUDE
221 variables in build.info files.
223 install => a hash table containing 'type' => [ 'file' ... ] pairs.
224 The types are 'programs', 'libraries', 'modules' and
225 'scripts', and the array of files list the files of
226 that type that should be installed.
228 libraries => a list of libraries. These are directly inferred from
229 the LIBS variable in build.info files.
231 programs => a list of programs. These are directly inferred from
232 the PROGRAMS variable in build.info files.
234 scripts => a list of scripts. There are directly inferred from
235 the SCRIPTS variable in build.info files.
237 sources => a hash table containing 'file' => [ 'sourcefile' ... ]
238 pairs. These are indirectly inferred from the SOURCE
239 variables in build.info files. Object files are
240 mentioned in this hash table, with source files from
241 SOURCE variables, and AS source files for programs and
245 a hash table just like 'sources', but only as source
246 files (object files) for building shared libraries.
248 As an example, here is how the build.info files example from the
249 section above would be digested into a %unified_info table:
251 our %unified_info = (
258 "crypto/buildinf.h" =>
262 "crypto/cversion.o" =>
270 "engines/ossltest" =>
278 "util/mkbuildinf.pl" =>
290 "crypto/buildinf.h" =>
292 "util/mkbuildinf.pl",
305 "engines/ossltest" =>
317 "util/mkbuildinf.pl" =>
361 "crypto/cversion.o" =>
369 "engines/e_dasync.o" =>
371 "engines/e_dasync.c",
375 "engines/e_dasync.o",
377 "engines/e_ossltest.o" =>
379 "engines/e_ossltest.c",
381 "engines/ossltest" =>
383 "engines/e_ossltest.o",
402 As can be seen, everything in %unified_info is fairly simple suggest
403 of information. Still, it tells us that to build all programs, we
404 must build 'apps/openssl', and to build the latter, we will need to
405 build all its sources ('apps/openssl.o' in this case) and all the
406 other things it depends on (such as 'libssl'). All those dependencies
407 need to be built as well, using the same logic, so to build 'libssl',
408 we need to build 'ssl/tls.o' as well as 'libcrypto', and to build the
415 Build-file templates are essentially build-files (such as Makefile on
416 Unix) with perl code fragments mixed in. Those perl code fragment
417 will generate all the configuration dependent data, including all the
418 rules needed to build end product files and intermediary files alike.
419 At a minimum, there must be a perl code fragment that defines a set of
420 functions that are used to generates specific build-file rules, to
421 build static libraries from object files, to build shared libraries
422 from static libraries, to programs from object files and libraries,
425 generatesrc - function that produces build file lines to generate
426 a source file from some input.
428 It's called like this:
430 generatesrc(src => "PATH/TO/tobegenerated",
431 generator => [ "generatingfile", ... ]
432 generator_incs => [ "INCL/PATH", ... ]
433 generator_deps => [ "dep1", ... ]
434 incs => [ "INCL/PATH", ... ],
435 deps => [ "dep1", ... ],
436 intent => one of "libs", "dso", "bin" );
438 'src' has the name of the file to be generated.
439 'generator' is the command or part of command to
440 generate the file, of which the first item is
441 expected to be the file to generate from.
442 generatesrc() is expected to analyse and figure out
443 exactly how to apply that file and how to capture
444 the result. 'generator_incs' and 'generator_deps'
445 are include directories and files that the generator
446 file itself depends on. 'incs' and 'deps' are
447 include directories and files that are used if $(CC)
448 is used as an intermediary step when generating the
449 end product (the file indicated by 'src'). 'intent'
450 indicates what the generated file is going to be
453 src2obj - function that produces build file lines to build an
454 object file from source files and associated data.
456 It's called like this:
458 src2obj(obj => "PATH/TO/objectfile",
459 srcs => [ "PATH/TO/sourcefile", ... ],
460 deps => [ "dep1", ... ],
461 incs => [ "INCL/PATH", ... ]
462 intent => one of "lib", "dso", "bin" );
464 'obj' has the intended object file with '.o'
465 extension, src2obj() is expected to change it to
466 something more suitable for the platform.
467 'srcs' has the list of source files to build the
468 object file, with the first item being the source
469 file that directly corresponds to the object file.
470 'deps' is a list of explicit dependencies. 'incs'
471 is a list of include file directories. Finally,
472 'intent' indicates what this object file is going
475 obj2lib - function that produces build file lines to build a
476 static library file ("libfoo.a" in Unix terms) from
481 obj2lib(lib => "PATH/TO/libfile",
482 objs => [ "PATH/TO/objectfile", ... ]);
484 'lib' has the intended library file name *without*
485 extension, obj2lib is expected to add that. 'objs'
486 has the list of object files to build this library.
488 libobj2shlib - backward compatibility function that's used the
489 same way as obj2shlib (described next), and was
490 expected to build the shared library from the
491 corresponding static library when that was suitable.
492 NOTE: building a shared library from a static
493 library is now DEPRECATED, as they no longer share
494 object files. Attempting to do this will fail.
496 obj2shlib - function that produces build file lines to build a
497 shareable object library file ("libfoo.so" in Unix
498 terms) from the corresponding object files.
502 obj2shlib(shlib => "PATH/TO/shlibfile",
503 lib => "PATH/TO/libfile",
504 objs => [ "PATH/TO/objectfile", ... ],
505 deps => [ "PATH/TO/otherlibfile", ... ]);
507 'lib' has the base (static) library file name
508 *without* extension. This is useful in case
509 supporting files are needed (such as import
510 libraries on Windows).
511 'shlib' has the corresponding shared library name
512 *without* extension. 'deps' has the list of other
513 libraries (also *without* extension) this library
514 needs to be linked with. 'objs' has the list of
515 object files to build this library.
517 obj2dso - function that produces build file lines to build a
518 dynamic shared object file from object files.
522 obj2dso(lib => "PATH/TO/libfile",
523 objs => [ "PATH/TO/objectfile", ... ],
524 deps => [ "PATH/TO/otherlibfile",
527 This is almost the same as obj2shlib, but the
528 intent is to build a shareable library that can be
529 loaded in runtime (a "plugin"...).
531 obj2bin - function that produces build file lines to build an
532 executable file from object files.
536 obj2bin(bin => "PATH/TO/binfile",
537 objs => [ "PATH/TO/objectfile", ... ],
538 deps => [ "PATH/TO/libfile", ... ]);
540 'bin' has the intended executable file name
541 *without* extension, obj2bin is expected to add
542 that. 'objs' has the list of object files to build
543 this library. 'deps' has the list of library files
544 (also *without* extension) that the programs needs
547 in2script - function that produces build file lines to build a
548 script file from some input.
552 in2script(script => "PATH/TO/scriptfile",
553 sources => [ "PATH/TO/infile", ... ]);
555 'script' has the intended script file name.
556 'sources' has the list of source files to build the
557 resulting script from.
559 Along with the build-file templates is the driving template
560 Configurations/common.tmpl, which looks through all the information in
561 %unified_info and generates all the rulesets to build libraries,
562 programs and all intermediate files, using the rule generating
563 functions defined in the build-file template.
565 As an example with the smaller build.info set we've seen as an
566 example, producing the rules to build 'libcrypto' would result in the
569 # Note: obj2shlib will only be called if shared libraries are
571 # Note 2: obj2shlib must convert the '.o' extension to whatever
572 # is suitable on the local platform.
573 obj2shlib(shlib => "libcrypto",
574 objs => [ "crypto/aes.o", "crypto/evp.o", "crypto/cversion.o" ],
577 obj2lib(lib => "libcrypto"
578 objs => [ "crypto/aes.o", "crypto/evp.o", "crypto/cversion.o" ]);
580 src2obj(obj => "crypto/aes.o"
581 srcs => [ "crypto/aes.c" ],
583 incs => [ "include" ],
586 src2obj(obj => "crypto/evp.o"
587 srcs => [ "crypto/evp.c" ],
589 incs => [ "include" ],
592 src2obj(obj => "crypto/cversion.o"
593 srcs => [ "crypto/cversion.c" ],
594 deps => [ "crypto/buildinf.h" ],
595 incs => [ "include" ],
598 generatesrc(src => "crypto/buildinf.h",
599 generator => [ "util/mkbuildinf.pl", "\"$(CC)",
600 "$(CFLAGS)\"", "\"$(PLATFORM)\"" ],
601 generator_incs => [ "util" ],
602 generator_deps => [ "util/Foo.pm" ],
607 The returned strings from all those calls are then concatenated
608 together and written to the resulting build-file.