1 Design document for the unified scheme data
2 ===========================================
4 How are things connected?
5 -------------------------
7 The unified scheme takes all its data from the build.info files seen
8 throughout the source tree. These files hold the minimum information
9 needed to build end product files from diverse sources. See the
10 section on build.info files below.
12 From the information in build.info files, Configure builds up an
13 information database as a hash table called %unified_info, which is
14 stored in configdata.pm, found at the top of the build tree (which may
15 or may not be the same as the source tree).
17 Configurations/common.tmpl uses the data from %unified_info to
18 generate the rules for building end product files as well as
19 intermediary files with the help of a few functions found in the
20 build-file templates. See the section on build-file templates further
21 down for more information.
26 As mentioned earlier, build.info files are meant to hold the minimum
27 information needed to build output files, and therefore only (with a
28 few possible exceptions [1]) have information about end products (such
29 as scripts, library files and programs) and source files (such as C
30 files, C header files, assembler files, etc). Intermediate files such
31 as object files are rarely directly referred to in build.info files (and
32 when they are, it's always with the file name extension .o), they are
33 inferred by Configure. By the same rule of minimalism, end product
34 file name extensions (such as .so, .a, .exe, etc) are never mentioned
35 in build.info. Their file name extensions will be inferred by the
36 build-file templates, adapted for the platform they are meant for (see
37 sections on %unified_info and build-file templates further down).
39 The variables PROGRAMS, LIBS, ENGINES and SCRIPTS are used to declare
42 The variables SOURCE, DEPEND, INCLUDE and ORDINALS are indexed by a
43 produced file, and their values are the source used to produce that
44 particular produced file, extra dependencies, include directories
45 needed, and ordinal files (explained further below.
47 All their values in all the build.info throughout the source tree are
48 collected together and form a set of programs, libraries, engines and
49 scripts to be produced, source files, dependencies, etc etc etc.
51 Let's have a pretend example, a very limited contraption of OpenSSL,
52 composed of the program 'apps/openssl', the libraries 'libssl' and
53 'libcrypto', an engine 'engines/ossltest' and their sources and
58 ORDINALS[libcrypto]=crypto
60 INCLUDE[libcrypto]=include
61 INCLUDE[libssl]=include
62 DEPEND[libssl]=libcrypto
64 This is the top directory build.info file, and it tells us that two
65 libraries are to be built, there are some ordinals to be used to
66 declare what symbols in those libraries are seen as public, the
67 include directory 'include/' shall be used throughout when building
68 anything that will end up in each library, and that the library
69 'libssl' depend on the library 'libcrypto' to function properly.
73 SOURCE[openssl]=openssl.c
74 INCLUDE[openssl]=.. ../include
75 DEPEND[openssl]=../libssl
77 This is the build.info file in 'apps/', one may notice that all file
78 paths mentioned are relative to the directory the build.info file is
79 located in. This one tells us that there's a program to be built
80 called 'apps/openssl' (the file name extension will depend on the
81 platform and is therefore not mentioned in the build.info file). It's
82 built from one source file, 'apps/openssl.c', and building it requires
83 the use of '.' and 'include' include directories (both are declared
84 from the point of view of the 'apps/' directory), and that the program
85 depends on the library 'libssl' to function properly.
89 SOURCE[../libcrypto]=aes.c evp.c cversion.c
90 DEPEND[cversion.o]=buildinf.h
92 GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
93 DEPEND[buildinf.h]=../Makefile
94 DEPEND[../util/mkbuildinf.pl]=../util/Foo.pm
96 This is the build.info file in 'crypto', and it tells us a little more
97 about what's needed to produce 'libcrypto'. LIBS is used again to
98 declare that 'libcrypto' is to be produced. This declaration is
99 really unnecessary as it's already mentioned in the top build.info
100 file, but can make the info file easier to understand. This is to
101 show that duplicate information isn't an issue.
103 This build.info file informs us that 'libcrypto' is built from a few
104 source files, 'crypto/aes.c', 'crypto/evp.c' and 'crypto/cversion.c'.
105 It also shows us that building the object file inferred from
106 'crypto/cversion.c' depends on 'crypto/buildinf.h'. Finally, it
107 also shows the possibility to declare how some files are generated
108 using some script, in this case a perl script, and how such scripts
109 can be declared to depend on other files, in this case a perl module.
111 Two things are worth an extra note:
113 'DEPEND[cversion.o]' mentions an object file. DEPEND indexes is the
114 only location where it's valid to mention them
116 Lines in 'BEGINRAW'..'ENDRAW' sections must always mention files as
117 seen from the top directory, no exception.
121 SOURCE[../libssl]=tls.c
123 This is the build.info file in 'ssl/', and it tells us that the
124 library 'libssl' is built from the source file 'ssl/tls.c'.
128 SOURCE[libossltest]=e_ossltest.c
129 DEPEND[libossltest]=../libcrypto
130 INCLUDE[libossltest]=../include
132 This is the build.info file in 'engines/', telling us that an engine
133 called 'engines/libossltest' shall be built, that it's source is
134 'engines/e_ossltest.c' and that the include directory 'include/' may
135 be used when building anything that will be part of this engine.
136 Finally, the engine 'engines/libossltest' depends on the library
137 'libcrypto' to function properly.
139 When Configure digests these build.info files, the accumulated
140 information comes down to this:
142 LIBS=libcrypto libssl
143 ORDINALS[libcrypto]=crypto
144 SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c
145 DEPEND[crypto/cversion.o]=crypto/buildinf.h
146 INCLUDE[libcrypto]=include
148 SOURCE[libssl]=ssl/tls.c
149 INCLUDE[libssl]=include
150 DEPEND[libssl]=libcrypto
152 PROGRAMS=apps/openssl
153 SOURCE[apps/openssl]=apps/openssl.c
154 INCLUDE[apps/openssl]=. include
155 DEPEND[apps/openssl]=libssl
157 ENGINES=engines/ossltest
158 SOURCE[engines/ossltest]=engines/e_ossltest.c
159 DEPEND[engines/ossltest]=libcrypto
160 INCLUDE[engines/ossltest]=include
162 GENERATE[crypto/buildinf.h]=util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)"
163 DEPEND[crypto/buildinf.h]=Makefile
164 DEPEND[util/mkbuildinf.pl]=util/Foo.pm
167 A few notes worth mentioning:
169 LIBS may be used to declare routine libraries only.
171 PROGRAMS may be used to declare programs only.
173 ENGINES may be used to declare engines only.
175 The indexes for SOURCE and ORDINALS must only be end product files,
176 such as libraries, programs or engines. The values of SOURCE
177 variables must only be source files (possibly generated)
179 INCLUDE and DEPEND shows a relationship between different files
180 (usually produced files) or between files and directories, such as a
181 program depending on a library, or between an object file and some
184 When Configure processes the build.info files, it will take it as
185 truth without question, and will therefore perform very few checks.
186 If the build tree is separate from the source tree, it will assume
187 that all built files and up in the build directory and that all source
188 files are to be found in the source tree, if they can be found there.
189 Configure will assume that source files that can't be found in the
190 source tree (such as 'crypto/bildinf.h' in the example above) are
191 generated and will be found in the build tree.
194 The %unified_info database
195 --------------------------
197 The information in all the build.info get digested by Configure and
198 collected into the %unified_info database, divided into the following
201 depends => a hash table containing 'file' => [ 'dependency' ... ]
202 pairs. These are directly inferred from the DEPEND
203 variables in build.info files.
205 engines => a list of engines. These are directly inferred from
206 the ENGINES variable in build.info files.
208 generate => a hash table containing 'file' => [ 'generator' ... ]
209 pairs. These are directly inferred from the GENERATE
210 variables in build.info files.
212 includes => a hash table containing 'file' => [ 'include' ... ]
213 pairs. These are directly inferred from the INCLUDE
214 variables in build.info files.
216 libraries => a list of libraries. These are directly inferred from
217 the LIBS variable in build.info files.
219 ordinals => a hash table containing 'file' => [ 'word', 'ordfile' ]
220 pairs. 'file' and 'word' are directly inferred from
221 the ORDINALS variables in build.info files, while the
222 file 'ofile' comes from internal knowledge in
225 programs => a list of programs. These are directly inferred from
226 the PROGRAMS variable in build.info files.
228 rawlines => a list of build-file lines. These are a direct copy of
229 the BEGINRAW..ENDRAW lines in build.info files. Note:
230 only the BEGINRAW..ENDRAW section for the current
231 platform are copied, the rest are ignored.
233 scripts => a list of scripts. There are directly inferred from
234 the SCRIPTS variable in build.info files.
236 sources => a hash table containing 'file' => [ 'sourcefile' ... ]
237 pairs. These are indirectly inferred from the SOURCE
238 variables in build.info files. Object files are
239 mentioned in this hash table, with source files from
240 SOURCE variables, and AS source files for programs and
244 a hash table just like 'sources', but only as source
245 files (object files) for building shared libraries.
247 As an example, here is how the build.info files example from the
248 section above would be digested into a %unified_info table:
250 our %unified_info = (
257 "crypto/buildinf.h" =>
261 "crypto/cversion.o" =>
265 "engines/ossltest" =>
273 "util/mkbuildinf.pl" =>
284 "crypto/buildinf.h" =>
286 "util/mkbuildinf.pl",
299 "engines/ossltest" =>
311 "util/mkbuildinf.pl" =>
326 "util/libcrypto.num",
355 "crypto/cversion.o" =>
363 "engines/e_ossltest.o" =>
365 "engines/e_ossltest.c",
367 "engines/ossltest" =>
369 "engines/e_ossltest.o",
388 As can be seen, everything in %unified_info is fairly simple suggest
389 of information. Still, it tells us that to build all programs, we
390 must build 'apps/openssl', and to build the latter, we will need to
391 build all its sources ('apps/openssl.o' in this case) and all the
392 other things it depends on (such as 'libssl'). All those dependencies
393 need to be built as well, using the same logic, so to build 'libssl',
394 we need to build 'ssl/tls.o' as well as 'libcrypto', and to build the
401 Build-file templates are essentially build-files (such as Makefile on
402 Unix) with perl code fragments mixed in. Those perl code fragment
403 will generate all the configuration dependent data, including all the
404 rules needed to build end product files and intermediary files alike.
405 At a minimum, there must be a perl code fragment that defines a set of
406 functions that are used to generates specific build-file rules, to
407 build static libraries from object files, to build shared libraries
408 from static libraries, to programs from object files and libraries,
411 generatesrc - function that produces build file lines to generate
412 a source file from some input.
414 It's called like this:
416 generatesrc(src => "PATH/TO/tobegenerated",
417 generator => [ "generatingfile", ... ]
418 generator_incs => [ "INCL/PATH", ... ]
419 generator_deps => [ "dep1", ... ]
420 incs => [ "INCL/PATH", ... ],
421 deps => [ "dep1", ... ],
422 intent => one of "libs", "dso", "bin" );
424 'src' has the name of the file to be generated.
425 'generator' is the command or part of command to
426 generate the file, of which the first item is
427 expected to be the file to generate from.
428 generatesrc() is expected to analyse and figure out
429 exactly how to apply that file and how to capture
430 the result. 'generator_incs' and 'generator_deps'
431 are include directories and files that the generator
432 file itself depends on. 'incs' and 'deps' are
433 include directories and files that are used if $(CC)
434 is used as an intermediary step when generating the
435 end product (the file indicated by 'src'). 'intent'
436 indicates what the generated file is going to be
439 src2obj - function that produces build file lines to build an
440 object file from source files and associated data.
442 It's called like this:
444 src2obj(obj => "PATH/TO/objectfile",
445 srcs => [ "PATH/TO/sourcefile", ... ],
446 deps => [ "dep1", ... ],
447 incs => [ "INCL/PATH", ... ]
448 intent => one of "lib", "dso", "bin" );
450 'obj' has the intended object file *without*
451 extension, src2obj() is expected to add that.
452 'srcs' has the list of source files to build the
453 object file, with the first item being the source
454 file that directly corresponds to the object file.
455 'deps' is a list of explicit dependencies. 'incs'
456 is a list of include file directories. Finally,
457 'intent' indicates what this object file is going
460 obj2lib - function that produces build file lines to build a
461 static library file ("libfoo.a" in Unix terms) from
466 obj2lib(lib => "PATH/TO/libfile",
467 objs => [ "PATH/TO/objectfile", ... ]);
469 'lib' has the intended library file name *without*
470 extension, obj2lib is expected to add that. 'objs'
471 has the list of object files (also *without*
472 extension) to build this library.
474 libobj2shlib - function that produces build file lines to build a
475 shareable object library file ("libfoo.so" in Unix
476 terms) from the corresponding static library file
481 libobj2shlib(shlib => "PATH/TO/shlibfile",
482 lib => "PATH/TO/libfile",
483 objs => [ "PATH/TO/objectfile", ... ],
484 deps => [ "PATH/TO/otherlibfile", ... ],
485 ordinals => [ "word", "/PATH/TO/ordfile" ]);
487 'lib' has the intended library file name *without*
488 extension, libobj2shlib is expected to add that.
489 'shlib' has the corresponding shared library name
490 *without* extension. 'deps' has the list of other
491 libraries (also *without* extension) this library
492 needs to be linked with. 'objs' has the list of
493 object files (also *without* extension) to build
494 this library. 'ordinals' MAY be present, and when
495 it is, its value is an array where the word is
496 "crypto" or "ssl" and the file is one of the ordinal
497 files util/libcrypto.num or util/libssl.num in the
500 This function has a choice; it can use the
501 corresponding static library as input to make the
502 shared library, or the list of object files.
504 obj2dynlib - function that produces build file lines to build a
505 dynamically loadable library file ("libfoo.so" on
506 Unix) from object files.
510 obj2dynlib(lib => "PATH/TO/libfile",
511 objs => [ "PATH/TO/objectfile", ... ],
512 deps => [ "PATH/TO/otherlibfile",
515 This is almost the same as libobj2shlib, but the
516 intent is to build a shareable library that can be
517 loaded in runtime (a "plugin"...). The differences
518 are subtle, one of the most visible ones is that the
519 resulting shareable library is produced from object
522 obj2bin - function that produces build file lines to build an
523 executable file from object files.
527 obj2bin(bin => "PATH/TO/binfile",
528 objs => [ "PATH/TO/objectfile", ... ],
529 deps => [ "PATH/TO/libfile", ... ]);
531 'bin' has the intended executable file name
532 *without* extension, obj2bin is expected to add
533 that. 'objs' has the list of object files (also
534 *without* extension) to build this library. 'deps'
535 has the list of library files (also *without*
536 extension) that the programs needs to be linked
539 in2script - function that produces build file lines to build a
540 script file from some input.
544 in2script(script => "PATH/TO/scriptfile",
545 sources => [ "PATH/TO/infile", ... ]);
547 'script' has the intended script file name.
548 'sources' has the list of source files to build the
549 resulting script from.
551 Along with the build-file templates is the driving engine
552 Configurations/common.tmpl, which looks through all the information in
553 %unified_info and generates all the rulesets to build libraries,
554 programs and all intermediate files, using the rule generating
555 functions defined in the build-file template.
557 As an example with the smaller build.info set we've seen as an
558 example, producing the rules to build 'libcrypto' would result in the
561 # Note: libobj2shlib will only be called if shared libraries are
563 # Note 2: libobj2shlib gets both the name of the static library
564 # and the names of all the object files that go into it. It's up
565 # to the implementation to decide which to use as input.
566 # Note 3: common.tmpl peals off the ".o" extension from all object
567 # files, as the platform at hand may have a different one.
568 libobj2shlib(shlib => "libcrypto",
570 objs => [ "crypto/aes", "crypto/evp", "crypto/cversion" ],
572 ordinals => [ "crypto", "util/libcrypto.num" ]);
574 obj2lib(lib => "libcrypto"
575 objs => [ "crypto/aes", "crypto/evp", "crypto/cversion" ]);
577 src2obj(obj => "crypto/aes"
578 srcs => [ "crypto/aes.c" ],
580 incs => [ "include" ],
583 src2obj(obj => "crypto/evp"
584 srcs => [ "crypto/evp.c" ],
586 incs => [ "include" ],
589 src2obj(obj => "crypto/cversion"
590 srcs => [ "crypto/cversion.c" ],
591 deps => [ "crypto/buildinf.h" ],
592 incs => [ "include" ],
595 generatesrc(src => "crypto/buildinf.h",
596 generator => [ "util/mkbuildinf.pl", "\"$(CC)",
597 "$(CFLAGS)\"", "\"$(PLATFORM)\"" ],
598 generator_incs => [ "util" ],
599 generator_deps => [ "util/Foo.pm" ],
604 The returned strings from all those calls are then concatenated
605 together and written to the resulting build-file.