1 Design document for the unified scheme data
2 ===========================================
4 How are things connected?
5 -------------------------
7 The unified scheme takes all its data from the build.info files seen
8 throughout the source tree. These files hold the minimum information
9 needed to build end product files from diverse sources. See the
10 section on build.info files below.
12 From the information in build.info files, Configure builds up an
13 information database as a hash table called %unified_info, which is
14 stored in configdata.pm, found at the top of the build tree (which may
15 or may not be the same as the source tree).
17 Configurations/common.tmpl uses the data from %unified_info to
18 generate the rules for building end product files as well as
19 intermediary files with the help of a few functions found in the
20 build-file templates. See the section on build-file templates further
21 down for more information.
26 As mentioned earlier, build.info files are meant to hold the minimum
27 information needed to build output files, and therefore only (with a
28 few possible exceptions [1]) have information about end products (such
29 as scripts, library files and programs) and source files (such as C
30 files, C header files, assembler files, etc). Intermediate files such
31 as object files are rarely directly referred to in build.info files (and
32 when they are, it's always with the file name extension .o), they are
33 inferred by Configure. By the same rule of minimalism, end product
34 file name extensions (such as .so, .a, .exe, etc) are never mentioned
35 in build.info. Their file name extensions will be inferred by the
36 build-file templates, adapted for the platform they are meant for (see
37 sections on %unified_info and build-file templates further down).
39 The variables PROGRAMS, LIBS, ENGINES and SCRIPTS are used to declare
42 The variables SOURCE, DEPEND, INCLUDE and ORDINALS are indexed by a
43 produced file, and their values are the source used to produce that
44 particular produced file, extra dependencies, include directories
45 needed, and ordinal files (explained further below.
47 All their values in all the build.info throughout the source tree are
48 collected together and form a set of programs, libraries, engines and
49 scripts to be produced, source files, dependencies, etc etc etc.
51 Let's have a pretend example, a very limited contraption of OpenSSL,
52 composed of the program 'apps/openssl', the libraries 'libssl' and
53 'libcrypto', an engine 'engines/ossltest' and their sources and
58 ORDINALS[libcrypto]=crypto
60 INCLUDE[libcrypto]=include
61 INCLUDE[libssl]=include
62 DEPEND[libssl]=libcrypto
64 This is the top directory build.info file, and it tells us that two
65 libraries are to be built, there are some ordinals to be used to
66 declare what symbols in those libraries are seen as public, the
67 include directory 'include/' shall be used throughout when building
68 anything that will end up in each library, and that the library
69 'libssl' depend on the library 'libcrypto' to function properly.
73 SOURCE[openssl]=openssl.c
74 INCLUDE[openssl]=.. ../include
75 DEPEND[openssl]=../libssl
77 This is the build.info file in 'apps/', one may notice that all file
78 paths mentioned are relative to the directory the build.info file is
79 located in. This one tells us that there's a program to be built
80 called 'apps/openssl' (the file name extension will depend on the
81 platform and is therefore not mentioned in the build.info file). It's
82 built from one source file, 'apps/openssl.c', and building it requires
83 the use of '.' and 'include' include directories (both are declared
84 from the point of view of the 'apps/' directory), and that the program
85 depends on the library 'libssl' to function properly.
89 SOURCE[../libcrypto]=aes.c evp.c cversion.c
90 DEPEND[cversion.o]=buildinf.h
92 GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
93 DEPEND[buildinf.h]=../Makefile
95 This is the build.info file in 'crypto', and it tells us a little more
96 about what's needed to produce 'libcrypto'. LIBS is used again to
97 declare that 'libcrypto' is to be produced. This declaration is
98 really unnecessary as it's already mentioned in the top build.info
99 file, but can make the info file easier to understand. This is to
100 show that duplicate information isn't an issue.
102 This build.info file informs us that 'libcrypto' is built from a few
103 source files, 'crypto/aes.c', 'crypto/evp.c' and 'crypto/cversion.c'.
104 It also shows us that building the object file inferred from
105 'crypto/cversion.c' depends on 'crypto/buildinf.h'. Finally, it
106 also shows the possibility to declare how some files are generated
107 using some script, in this case a perl script.
109 Two things are worth an extra note:
111 'DEPEND[cversion.o]' mentions an object file. DEPEND indexes is the
112 only location where it's valid to mention them
114 Lines in 'BEGINRAW'..'ENDRAW' sections must always mention files as
115 seen from the top directory, no exception.
119 SOURCE[../libssl]=tls.c
121 This is the build.info file in 'ssl/', and it tells us that the
122 library 'libssl' is built from the source file 'ssl/tls.c'.
126 SOURCE[libossltest]=e_ossltest.c
127 DEPEND[libossltest]=../libcrypto
128 INCLUDE[libossltest]=../include
130 This is the build.info file in 'engines/', telling us that an engine
131 called 'engines/libossltest' shall be built, that it's source is
132 'engines/e_ossltest.c' and that the include directory 'include/' may
133 be used when building anything that will be part of this engine.
134 Finally, the engine 'engines/libossltest' depends on the library
135 'libcrypto' to function properly.
137 When Configure digests these build.info files, the accumulated
138 information comes down to this:
140 LIBS=libcrypto libssl
141 ORDINALS[libcrypto]=crypto
142 SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c
143 DEPEND[crypto/cversion.o]=crypto/buildinf.h
144 INCLUDE[libcrypto]=include
146 SOURCE[libssl]=ssl/tls.c
147 INCLUDE[libssl]=include
148 DEPEND[libssl]=libcrypto
150 PROGRAMS=apps/openssl
151 SOURCE[apps/openssl]=apps/openssl.c
152 INCLUDE[apps/openssl]=. include
153 DEPEND[apps/openssl]=libssl
155 ENGINES=engines/ossltest
156 SOURCE[engines/ossltest]=engines/e_ossltest.c
157 DEPEND[engines/ossltest]=libcrypto
158 INCLUDE[engines/ossltest]=include
160 GENERATE[crypto/buildinf.h]=util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
161 DEPEND[crypto/buildinf.h]=Makefile
164 A few notes worth mentioning:
166 LIBS may be used to declare routine libraries only.
168 PROGRAMS may be used to declare programs only.
170 ENGINES may be used to declare engines only.
172 The indexes for SOURCE, INCLUDE and ORDINALS must only be end product
173 files, such as libraries, programs or engines. The values of SOURCE
174 variables must only be source files (possibly generated)
176 DEPEND shows a relationship between different produced files, such
177 as a program depending on a library, or between an object file and
178 some extra source file.
180 When Configure processes the build.info files, it will take it as
181 truth without question, and will therefore perform very few checks.
182 If the build tree is separate from the source tree, it will assume
183 that all built files and up in the build directory and that all source
184 files are to be found in the source tree, if they can be found there.
185 Configure will assume that source files that can't be found in the
186 source tree (such as 'crypto/bildinf.h' in the example above) are
187 generated and will be found in the build tree.
190 The %unified_info database
191 --------------------------
193 The information in all the build.info get digested by Configure and
194 collected into the %unified_info database, divided into the following
197 depends => a hash table containing 'file' => [ 'dependency' ... ]
198 pairs. These are directly inferred from the DEPEND
199 variables in build.info files.
201 engines => a list of engines. These are directly inferred from
202 the ENGINES variable in build.info files.
204 generate => a hash table containing 'file' => [ 'generator' ... ]
205 pairs. These are directly inferred from the GENERATE
206 variables in build.info files.
208 includes => a hash table containing 'file' => [ 'include' ... ]
209 pairs. These are directly inferred from the INCLUDE
210 variables in build.info files.
212 libraries => a list of libraries. These are directly inferred from
213 the LIBS variable in build.info files.
215 ordinals => a hash table containing 'file' => [ 'word', 'ordfile' ]
216 pairs. 'file' and 'word' are directly inferred from
217 the ORDINALS variables in build.info files, while the
218 file 'ofile' comes from internal knowledge in
221 programs => a list of programs. These are directly inferred from
222 the PROGRAMS variable in build.info files.
224 rawlines => a list of build-file lines. These are a direct copy of
225 the BEGINRAW..ENDRAW lines in build.info files. Note:
226 only the BEGINRAW..ENDRAW section for the current
227 platform are copied, the rest are ignored.
229 scripts => a list of scripts. There are directly inferred from
230 the SCRIPTS variable in build.info files.
232 sources => a hash table containing 'file' => [ 'sourcefile' ... ]
233 pairs. These are indirectly inferred from the SOURCE
234 variables in build.info files. Object files are
235 mentioned in this hash table, with source files from
236 SOURCE variables, and AS source files for programs and
240 a hash table just like 'sources', but only as source
241 files (object files) for building shared libraries.
243 As an example, here is how the build.info files example from the
244 section above would be digested into a %unified_info table:
246 our %unified_info = (
253 "crypto/buildinf.h" =>
257 "crypto/cversion.o" =>
261 "engines/ossltest" =>
276 "crypto/buildinf.h" =>
278 "util/mkbuildinf.pl",
291 "engines/ossltest" =>
314 "util/libcrypto.num",
343 "crypto/cversion.o" =>
351 "engines/e_ossltest.o" =>
353 "engines/e_ossltest.c",
355 "engines/ossltest" =>
357 "engines/e_ossltest.o",
376 As can be seen, everything in %unified_info is fairly simple suggest
377 of information. Still, it tells us that to build all programs, we
378 must build 'apps/openssl', and to build the latter, we will need to
379 build all its sources ('apps/openssl.o' in this case) and all the
380 other things it depends on (such as 'libssl'). All those dependencies
381 need to be built as well, using the same logic, so to build 'libssl',
382 we need to build 'ssl/tls.o' as well as 'libcrypto', and to build the
389 Build-file templates are essentially build-files (such as Makefile on
390 Unix) with perl code fragments mixed in. Those perl code fragment
391 will generate all the configuration dependent data, including all the
392 rules needed to build end product files and intermediary files alike.
393 At a minimum, there must be a perl code fragment that defines a set of
394 functions that are used to generates specific build-file rules, to
395 build static libraries from object files, to build shared libraries
396 from static libraries, to programs from object files and libraries,
399 generatesrc - function that produces build file lines to generate
400 a source file from some input.
402 It's called like this:
404 generatesrc(src => "PATH/TO/tobegenerated",
405 generator => [ "generatingfile", ... ]
406 incs => [ "INCL/PATH", ... ],
407 deps => [ "dep1", ... ],
408 intent => one of "libs", "dso", "bin" );
410 'src' has the name of the file to be generated.
411 'generator' is the command or part of command to
412 generate the file, of which the first item is
413 expected to be the file to generate from.
414 generatesrc() is expected to analyse and figure out
415 exactly how to apply that file and how to capture
416 the result. 'incs' and 'deps' are include
417 directories and files that are used if $(CC) used as
418 an intermediary step when generating the end product
419 (the file indicated by 'src'). 'intent' indicates
420 what the generated file is going to be used for.
422 src2obj - function that produces build file lines to build an
423 object file from source files and associated data.
425 It's called like this:
427 src2obj(obj => "PATH/TO/objectfile",
428 srcs => [ "PATH/TO/sourcefile", ... ],
429 deps => [ "dep1", ... ],
430 incs => [ "INCL/PATH", ... ]
431 intent => one of "lib", "dso", "bin" );
433 'obj' has the intended object file *without*
434 extension, src2obj() is expected to add that.
435 'srcs' has the list of source files to build the
436 object file, with the first item being the source
437 file that directly corresponds to the object file.
438 'deps' is a list of explicit dependencies. 'incs'
439 is a list of include file directories. Finally,
440 'intent' indicates what this object file is going
443 obj2lib - function that produces build file lines to build a
444 static library file ("libfoo.a" in Unix terms) from
449 obj2lib(lib => "PATH/TO/libfile",
450 objs => [ "PATH/TO/objectfile", ... ]);
452 'lib' has the intended library file name *without*
453 extension, obj2lib is expected to add that. 'objs'
454 has the list of object files (also *without*
455 extension) to build this library.
457 libobj2shlib - function that produces build file lines to build a
458 shareable object library file ("libfoo.so" in Unix
459 terms) from the corresponding static library file
464 libobj2shlib(shlib => "PATH/TO/shlibfile",
465 lib => "PATH/TO/libfile",
466 objs => [ "PATH/TO/objectfile", ... ],
467 deps => [ "PATH/TO/otherlibfile", ... ],
468 ordinals => [ "word", "/PATH/TO/ordfile" ]);
470 'lib' has the intended library file name *without*
471 extension, libobj2shlib is expected to add that.
472 'shlib' has the corresponding shared library name
473 *without* extension. 'deps' has the list of other
474 libraries (also *without* extension) this library
475 needs to be linked with. 'objs' has the list of
476 object files (also *without* extension) to build
477 this library. 'ordinals' MAY be present, and when
478 it is, its value is an array where the word is
479 "crypto" or "ssl" and the file is one of the ordinal
480 files util/libcrypto.num or util/libssl.num in the
483 This function has a choice; it can use the
484 corresponding static library as input to make the
485 shared library, or the list of object files.
487 obj2dynlib - function that produces build file lines to build a
488 dynamically loadable library file ("libfoo.so" on
489 Unix) from object files.
493 obj2dynlib(lib => "PATH/TO/libfile",
494 objs => [ "PATH/TO/objectfile", ... ],
495 deps => [ "PATH/TO/otherlibfile",
498 This is almost the same as libobj2shlib, but the
499 intent is to build a shareable library that can be
500 loaded in runtime (a "plugin"...). The differences
501 are subtle, one of the most visible ones is that the
502 resulting shareable library is produced from object
505 obj2bin - function that produces build file lines to build an
506 executable file from object files.
510 obj2bin(bin => "PATH/TO/binfile",
511 objs => [ "PATH/TO/objectfile", ... ],
512 deps => [ "PATH/TO/libfile", ... ]);
514 'bin' has the intended executable file name
515 *without* extension, obj2bin is expected to add
516 that. 'objs' has the list of object files (also
517 *without* extension) to build this library. 'deps'
518 has the list of library files (also *without*
519 extension) that the programs needs to be linked
522 in2script - function that produces build file lines to build a
523 script file from some input.
527 in2script(script => "PATH/TO/scriptfile",
528 sources => [ "PATH/TO/infile", ... ]);
530 'script' has the intended script file name.
531 'sources' has the list of source files to build the
532 resulting script from.
534 Along with the build-file templates is the driving engine
535 Configurations/common.tmpl, which looks through all the information in
536 %unified_info and generates all the rulesets to build libraries,
537 programs and all intermediate files, using the rule generating
538 functions defined in the build-file template.
540 As an example with the smaller build.info set we've seen as an
541 example, producing the rules to build 'libssl' would result in the
544 # Note: libobj2shlib will only be called if shared libraries are
546 # Note 2: libobj2shlib gets both the name of the static library
547 # and the names of all the object files that go into it. It's up
548 # to the implementation to decide which to use as input.
549 # Note 3: common.tmpl peals off the ".o" extension from all object
550 # files, as the platform at hand may have a different one.
551 libobj2shlib(shlib => "libssl",
553 objs => [ "ssl/tls" ],
554 deps => [ "libcrypto" ]
555 ordinals => [ "ssl", "util/libssl.num" ]);
557 obj2lib(lib => "libssl"
558 objs => [ "ssl/tls" ]);
560 src2obj(obj => "ssl/tls"
561 srcs => [ "ssl/tls.c" ],
563 incs => [ "include" ],
566 The returned strings from all those calls are then concatenated
567 together and written to the resulting build-file.