1 @c ***********************************************************************
2 @node GNUnet Developer Handbook
3 @chapter GNUnet Developer Handbook
5 This book is intended to be an introduction for programmers that want to
6 extend the GNUnet framework. GNUnet is more than a simple peer-to-peer
9 For developers, GNUnet is:
12 @item developed by a community that believes in the GNU philosophy
13 @item Free Software (Free as in Freedom), licensed under the
14 GNU Affero General Public License
15 (@uref{https://www.gnu.org/licenses/licenses.html#AGPL})
16 @item A set of standards, including coding conventions and
18 @item A set of layered protocols, both specifying the communication
19 between peers as well as the communication between components
21 @item A set of libraries with well-defined APIs suitable for
25 In particular, the architecture specifies that a peer consists of many
26 processes communicating via protocols. Processes can be written in almost
28 @code{C}, @code{Java} and @code{Guile} APIs exist for accessing existing
29 services and for writing extensions.
30 It is possible to write extensions in other languages by
31 implementing the necessary IPC protocols.
33 GNUnet can be extended and improved along many possible dimensions, and
34 anyone interested in Free Software and Freedom-enhancing Networking is
35 welcome to join the effort. This Developer Handbook attempts to provide
36 an initial introduction to some of the key design choices and central
37 components of the system.
38 This part of the GNUNet documentation is far from complete,
39 and we welcome informed contributions, be it in the form of
40 new chapters, sections or insightful comments.
43 * Developer Introduction::
44 * Internal dependencies::
46 * System Architecture::
47 * Subsystem stability::
48 * Naming conventions and coding style guide::
50 * Developing extensions for GNUnet using the gnunet-ext template::
52 * Building GNUnet and its dependencies::
54 * Performance regression analysis with Gauger::
57 * Automatic Restart Manager (ARM)::
58 * TRANSPORT Subsystem::
60 * Distance-Vector plugin::
68 * HOSTLIST Subsystem::
69 * IDENTITY Subsystem::
70 * NAMESTORE Subsystem::
71 * PEERINFO Subsystem::
72 * PEERSTORE Subsystem::
74 * STATISTICS Subsystem::
75 * Distributed Hash Table (DHT)::
76 * GNU Name System (GNS)::
78 * REVOCATION Subsystem::
79 * File-sharing (FS) Subsystem::
85 @node Developer Introduction
86 @section Developer Introduction
88 This Developer Handbook is intended as first introduction to GNUnet for
89 new developers that want to extend the GNUnet framework. After the
90 introduction, each of the GNUnet subsystems (directories in the
91 @file{src/} tree) is (supposed to be) covered in its own chapter. In
92 addition to this documentation, GNUnet developers should be aware of the
93 services available on the GNUnet server to them.
95 New developers can have a look a the GNUnet tutorials for C and java
96 available in the @file{src/} directory of the repository or under the
99 @c ** FIXME: Link to files in source, not online.
100 @c ** FIXME: Where is the Java tutorial?
102 @item @xref{Top, Introduction,, gnunet-c-tutorial, The GNUnet C Tutorial}.
103 @item @uref{https://tutorial.gnunet.org/, GNUnet C tutorial}
104 @item GNUnet Java tutorial
107 In addition to the GNUnet Reference Documentation you are reading,
108 the GNUnet server at @uref{https://gnunet.org} contains
109 various resources for GNUnet developers and those
110 who aspire to become regular contributors.
111 They are all conveniently reachable via the "Developer"
112 entry in the navigation menu. Some additional tools (such as static
113 analysis reports) require a special developer access to perform certain
114 operations. If you want (or require) access, you should contact
115 @uref{http://grothoff.org/christian/, Christian Grothoff},
118 @c FIXME: A good part of this belongs on the website or should be
119 @c extended in subsections explaining usage of this. A simple list
120 @c is just taking space people have to read.
121 The public subsystems on the GNUnet server that help developers are:
125 @item The version control system (git) keeps our code and enables
126 distributed development.
127 It is publicly accessible at @uref{https://git.gnunet.org/}.
128 Only developers with write access can commit code, everyone else is
129 encouraged to submit patches to the GNUnet-developers mailinglist:
130 @uref{https://lists.gnu.org/mailman/listinfo/gnunet-developers, https://lists.gnu.org/mailman/listinfo/gnunet-developers}
132 @item The bugtracking system (Mantis).
133 We use it to track feature requests, open bug reports and their
135 It can be accessed at
136 @uref{https://bugs.gnunet.org/, https://bugs.gnunet.org/}.
137 Anyone can report bugs.
139 @item Our site installation of the
140 Continuous Integration (CI) system @code{Buildbot} is used
141 to check GNUnet builds automatically on a range of platforms.
142 The web interface of this CI is exposed at
143 @uref{https://old.gnunet.org/buildbot/, https://old.gnunet.org/buildbot/}.
144 Builds are triggered automatically 30 minutes after the last commit to
145 our repository was made.
147 @item The current quality of our automated test suite is assessed using
148 Code coverage analysis. This analysis is run daily; however the webpage
149 is only updated if all automated tests pass at that time. Testcases that
150 improve our code coverage are always welcome.
152 @item We try to automatically find bugs using a static analysis scan.
153 This scan is run daily; however the webpage is only updated if all
154 automated tests pass at the time. Note that not everything that is
155 flagged by the analysis is a bug, sometimes even good code can be marked
156 as possibly problematic. Nevertheless, developers are encouraged to at
157 least be aware of all issues in their code that are listed.
159 @item We use Gauger for automatic performance regression visualization.
161 Details on how to use Gauger are here.
163 @item We use @uref{http://junit.org/, junit} to automatically test
164 @command{gnunet-java}.
165 Automatically generated, current reports on the test suite are here.
168 @item We use Cobertura to generate test coverage reports for gnunet-java.
169 Current reports on test coverage are here.
176 @c ***********************************************************************
181 @node Project overview
182 @subsection Project overview
184 The GNUnet project consists at this point of several sub-projects. This
185 section is supposed to give an initial overview about the various
186 sub-projects. Note that this description also lists projects that are far
187 from complete, including even those that have literally not a single line
190 GNUnet sub-projects in order of likely relevance are currently:
194 @item @command{gnunet}
195 Core of the P2P framework, including file-sharing, VPN and
196 chat applications; this is what the Developer Handbook covers mostly
197 @item @command{gnunet-gtk}
198 Gtk+-based user interfaces, including:
201 @item @command{gnunet-fs-gtk} (file-sharing),
202 @item @command{gnunet-statistics-gtk} (statistics over time),
203 @item @command{gnunet-peerinfo-gtk}
204 (information about current connections and known peers),
205 @item @command{gnunet-namestore-gtk} (GNS record editor),
206 @item @command{gnunet-conversation-gtk} (voice chat GUI) and
207 @item @command{gnunet-setup} (setup tool for "everything")
210 @item @command{gnunet-fuse}
211 Mounting directories shared via GNUnet's file-sharing
212 on GNU/Linux distributions
213 @item @command{gnunet-update}
214 Installation and update tool
215 @item @command{gnunet-ext}
216 Template for starting 'external' GNUnet projects
217 @item @command{gnunet-java}
218 Java APIs for writing GNUnet services and applications
219 @item @command{gnunet-java-ext}
220 @item @command{eclectic}
221 Code to run GNUnet nodes on testbeds for research, development,
222 testing and evaluation
223 @c ** FIXME: Solve the status and location of gnunet-qt
224 @item @command{gnunet-qt}
225 Qt-based GNUnet GUI (is it deprecated?)
226 @item @command{gnunet-cocoa}
227 cocoa-based GNUnet GUI (is it deprecated?)
228 @item @command{gnunet-guile}
229 Guile bindings for GNUnet
230 @item @command{gnunet-python}
231 Python bindings for GNUnet
235 We are also working on various supporting libraries and tools:
236 @c ** FIXME: What about gauger, and what about libmwmodem?
239 @item @command{libextractor}
240 GNU libextractor (meta data extraction)
241 @item @command{libmicrohttpd}
242 GNU libmicrohttpd (embedded HTTP(S) server library)
243 @item @command{gauger}
244 Tool for performance regression analysis
245 @item @command{monkey}
246 Tool for automated debugging of distributed systems
247 @item @command{libmwmodem}
248 Library for accessing satellite connection quality reports
249 @item @command{libgnurl}
250 gnURL (feature-restricted variant of cURL/libcurl)
252 work in progress of the new gnunet.org website (Jinja2 framework based to
253 replace our current Drupal website)
254 @item @command{bibliography}
255 Our collected bibliography, papers, references, and so forth
256 @item @command{gnunet-videos-}
257 Videos about and around gnunet activities
260 Finally, there are various external projects (see links for a list of
261 those that have a public website) which build on top of the GNUnet
264 @c ***********************************************************************
265 @node Internal dependencies
266 @section Internal dependencies
268 This section tries to give an overview of what processes a typical GNUnet
269 peer running a particular application would consist of. All of the
270 processes listed here should be automatically started by
271 @command{gnunet-arm -s}.
272 The list is given as a rough first guide to users for failure diagnostics.
273 Ideally, end-users should never have to worry about these internal
276 In terms of internal dependencies, a minimum file-sharing system consists
277 of the following GNUnet processes (in order of dependency):
280 @item gnunet-service-arm
281 @item gnunet-service-resolver (required by all)
282 @item gnunet-service-statistics (required by all)
283 @item gnunet-service-peerinfo
284 @item gnunet-service-transport (requires peerinfo)
285 @item gnunet-service-core (requires transport)
286 @item gnunet-daemon-hostlist (requires core)
287 @item gnunet-daemon-topology (requires hostlist, peerinfo)
288 @item gnunet-service-datastore
289 @item gnunet-service-dht (requires core)
290 @item gnunet-service-identity
291 @item gnunet-service-fs (requires identity, mesh, dht, datastore, core)
295 A minimum VPN system consists of the following GNUnet processes (in
296 order of dependency):
299 @item gnunet-service-arm
300 @item gnunet-service-resolver (required by all)
301 @item gnunet-service-statistics (required by all)
302 @item gnunet-service-peerinfo
303 @item gnunet-service-transport (requires peerinfo)
304 @item gnunet-service-core (requires transport)
305 @item gnunet-daemon-hostlist (requires core)
306 @item gnunet-service-dht (requires core)
307 @item gnunet-service-mesh (requires dht, core)
308 @item gnunet-service-dns (requires dht)
309 @item gnunet-service-regex (requires dht)
310 @item gnunet-service-vpn (requires regex, dns, mesh, dht)
314 A minimum GNS system consists of the following GNUnet processes (in
315 order of dependency):
318 @item gnunet-service-arm
319 @item gnunet-service-resolver (required by all)
320 @item gnunet-service-statistics (required by all)
321 @item gnunet-service-peerinfo
322 @item gnunet-service-transport (requires peerinfo)
323 @item gnunet-service-core (requires transport)
324 @item gnunet-daemon-hostlist (requires core)
325 @item gnunet-service-dht (requires core)
326 @item gnunet-service-mesh (requires dht, core)
327 @item gnunet-service-dns (requires dht)
328 @item gnunet-service-regex (requires dht)
329 @item gnunet-service-vpn (requires regex, dns, mesh, dht)
330 @item gnunet-service-identity
331 @item gnunet-service-namestore (requires identity)
332 @item gnunet-service-gns (requires vpn, dns, dht, namestore, identity)
335 @c ***********************************************************************
337 @section Code overview
339 This section gives a brief overview of the GNUnet source code.
340 Specifically, we sketch the function of each of the subdirectories in
341 the @file{gnunet/src/} directory. The order given is roughly bottom-up
342 (in terms of the layers of the system).
345 @item @file{util/} --- libgnunetutil
346 Library with general utility functions, all
347 GNUnet binaries link against this library. Anything from memory
348 allocation and data structures to cryptography and inter-process
349 communication. The goal is to provide an OS-independent interface and
350 more 'secure' or convenient implementations of commonly used primitives.
351 The API is spread over more than a dozen headers, developers should study
352 those closely to avoid duplicating existing functions.
353 @pxref{libgnunetutil}.
354 @item @file{hello/} --- libgnunethello
355 HELLO messages are used to
356 describe under which addresses a peer can be reached (for example,
357 protocol, IP, port). This library manages parsing and generating of HELLO
359 @item @file{block/} --- libgnunetblock
360 The DHT and other components of GNUnet
361 store information in units called 'blocks'. Each block has a type and the
362 type defines a particular format and how that binary format is to be
363 linked to a hash code (the key for the DHT and for databases). The block
364 library is a wrapper around block plugins which provide the necessary
365 functions for each block type.
366 @item @file{statistics/} --- statistics service
367 The statistics service enables associating
368 values (of type uint64_t) with a component name and a string. The main
369 uses is debugging (counting events), performance tracking and user
370 entertainment (what did my peer do today?).
371 @item @file{arm/} --- Automatic Restart Manager (ARM)
372 The automatic-restart-manager (ARM) service
373 is the GNUnet master service. Its role is to start gnunet-services, to
374 re-start them when they crashed and finally to shut down the system when
376 @item @file{peerinfo/} --- peerinfo service
377 The peerinfo service keeps track of which peers are known
378 to the local peer and also tracks the validated addresses for each peer
379 (in the form of a HELLO message) for each of those peers. The peer is not
380 necessarily connected to all peers known to the peerinfo service.
381 Peerinfo provides persistent storage for peer identities --- peers are
382 not forgotten just because of a system restart.
383 @item @file{datacache/} --- libgnunetdatacache
384 The datacache library provides (temporary) block storage for the DHT.
385 Existing plugins can store blocks in Sqlite, Postgres or MySQL databases.
386 All data stored in the cache is lost when the peer is stopped or
387 restarted (datacache uses temporary tables).
388 @item @file{datastore/} --- datastore service
389 The datastore service stores file-sharing blocks in
390 databases for extended periods of time. In contrast to the datacache, data
391 is not lost when peers restart. However, quota restrictions may still
392 cause old, expired or low-priority data to be eventually discarded.
393 Existing plugins can store blocks in Sqlite, Postgres or MySQL databases.
394 @item @file{template/} --- service template
395 Template for writing a new service. Does nothing.
396 @item @file{ats/} --- Automatic Transport Selection
397 The automatic transport selection (ATS) service
398 is responsible for deciding which address (i.e.
399 which transport plugin) should be used for communication with other peers,
400 and at what bandwidth.
401 @item @file{nat/} --- libgnunetnat
402 Library that provides basic functions for NAT traversal.
403 The library supports NAT traversal with
404 manual hole-punching by the user, UPnP and ICMP-based autonomous NAT
405 traversal. The library also includes an API for testing if the current
406 configuration works and the @code{gnunet-nat-server} which provides an
407 external service to test the local configuration.
408 @item @file{fragmentation/} --- libgnunetfragmentation
409 Some transports (UDP and WLAN, mostly) have restrictions on the maximum
410 transfer unit (MTU) for packets. The fragmentation library can be used to
411 break larger packets into chunks of at most 1k and transmit the resulting
412 fragments reliably (with acknowledgment, retransmission, timeouts,
414 @item @file{transport/} --- transport service
415 The transport service is responsible for managing the
416 basic P2P communication. It uses plugins to support P2P communication
417 over TCP, UDP, HTTP, HTTPS and other protocols.The transport service
418 validates peer addresses, enforces bandwidth restrictions, limits the
419 total number of connections and enforces connectivity restrictions (i.e.
421 @item @file{peerinfo-tool/} --- gnunet-peerinfo
422 This directory contains the gnunet-peerinfo binary which can be used to
423 inspect the peers and HELLOs known to the peerinfo service.
425 The core service is responsible for establishing encrypted, authenticated
426 connections with other peers, encrypting and decrypting messages and
427 forwarding messages to higher-level services that are interested in them.
428 @item @file{testing/} --- libgnunettesting
429 The testing library allows starting (and stopping) peers
430 for writing testcases.
431 It also supports automatic generation of configurations for peers
432 ensuring that the ports and paths are disjoint. libgnunettesting is also
433 the foundation for the testbed service
434 @item @file{testbed/} --- testbed service
435 The testbed service is used for creating small or large scale deployments
436 of GNUnet peers for evaluation of protocols.
437 It facilitates peer deployments on multiple
438 hosts (for example, in a cluster) and establishing various network
439 topologies (both underlay and overlay).
440 @item @file{nse/} --- Network Size Estimation
441 The network size estimation (NSE) service
442 implements a protocol for (securely) estimating the current size of the
444 @item @file{dht/} --- distributed hash table
445 The distributed hash table (DHT) service provides a
446 distributed implementation of a hash table to store blocks under hash
447 keys in the P2P network.
448 @item @file{hostlist/} --- hostlist service
449 The hostlist service allows learning about
450 other peers in the network by downloading HELLO messages from an HTTP
451 server, can be configured to run such an HTTP server and also implements
452 a P2P protocol to advertise and automatically learn about other peers
453 that offer a public hostlist server.
454 @item @file{topology/} --- topology service
455 The topology service is responsible for
456 maintaining the mesh topology. It tries to maintain connections to friends
457 (depending on the configuration) and also tries to ensure that the peer
458 has a decent number of active connections at all times. If necessary, new
459 connections are added. All peers should run the topology service,
460 otherwise they may end up not being connected to any other peer (unless
461 some other service ensures that core establishes the required
462 connections). The topology service also tells the transport service which
463 connections are permitted (for friend-to-friend networking)
464 @item @file{fs/} --- file-sharing
465 The file-sharing (FS) service implements GNUnet's
466 file-sharing application. Both anonymous file-sharing (using gap) and
467 non-anonymous file-sharing (using dht) are supported.
468 @item @file{cadet/} --- cadet service
469 The CADET service provides a general-purpose routing abstraction to create
470 end-to-end encrypted tunnels in mesh networks. We wrote a paper
471 documenting key aspects of the design.
472 @item @file{tun/} --- libgnunettun
473 Library for building IPv4, IPv6 packets and creating
474 checksums for UDP, TCP and ICMP packets. The header
475 defines C structs for common Internet packet formats and in particular
476 structs for interacting with TUN (virtual network) interfaces.
477 @item @file{mysql/} --- libgnunetmysql
478 Library for creating and executing prepared MySQL
479 statements and to manage the connection to the MySQL database.
480 Essentially a lightweight wrapper for the interaction between GNUnet
481 components and libmysqlclient.
483 Service that allows intercepting and modifying DNS requests of
484 the local machine. Currently used for IPv4-IPv6 protocol translation
485 (DNS-ALG) as implemented by "pt/" and for the GNUnet naming system. The
486 service can also be configured to offer an exit service for DNS traffic.
487 @item @file{vpn/} --- VPN service
488 The virtual public network (VPN) service provides a virtual
489 tunnel interface (VTUN) for IP routing over GNUnet.
490 Needs some other peers to run an "exit" service to work.
491 Can be activated using the "gnunet-vpn" tool or integrated with DNS using
494 Daemon to allow traffic from the VPN to exit this
495 peer to the Internet or to specific IP-based services of the local peer.
496 Currently, an exit service can only be restricted to IPv4 or IPv6, not to
497 specific ports and or IP address ranges. If this is not acceptable,
498 additional firewall rules must be added manually. exit currently only
499 works for normal UDP, TCP and ICMP traffic; DNS queries need to leave the
500 system via a DNS service.
502 protocol translation daemon. This daemon enables 4-to-6,
503 6-to-4, 4-over-6 or 6-over-4 transitions for the local system. It
504 essentially uses "DNS" to intercept DNS replies and then maps results to
505 those offered by the VPN, which then sends them using mesh to some daemon
506 offering an appropriate exit service.
507 @item @file{identity/}
508 Management of egos (alter egos) of a user; identities are
509 essentially named ECC private keys and used for zones in the GNU name
510 system and for namespaces in file-sharing, but might find other uses later
511 @item @file{revocation/}
512 Key revocation service, can be used to revoke the
513 private key of an identity if it has been compromised
514 @item @file{namecache/}
515 Cache for resolution results for the GNU name system;
516 data is encrypted and can be shared among users,
517 loss of the data should ideally only result in a
518 performance degradation (persistence not required)
519 @item @file{namestore/}
520 Database for the GNU name system with per-user private information,
523 GNU name system, a GNU approach to DNS and PKI.
525 A plugin for distance-vector (DV)-based routing.
526 DV consists of a service and a transport plugin to provide peers
527 with the illusion of a direct P2P connection for connections
528 that use multiple (typically up to 3) hops in the actual underlay network.
530 Service for the (distributed) evaluation of regular expressions.
531 @item @file{scalarproduct/}
532 The scalar product service offers an API to perform a secure multiparty
533 computation which calculates a scalar product between two peers
534 without exposing the private input vectors of the peers to each other.
535 @item @file{consensus/}
536 The consensus service will allow a set of peers to agree
537 on a set of values via a distributed set union computation.
539 The rest API allows access to GNUnet services using RESTful interaction.
540 The services provide plugins that can exposed by the rest server.
541 @c FIXME: Where did this disappear to?
542 @c @item @file{experimentation/}
543 @c The experimentation daemon coordinates distributed
544 @c experimentation to evaluate transport and ATS properties.
547 @c ***********************************************************************
548 @node System Architecture
549 @section System Architecture
551 @c FIXME: For those irritated by the textflow, we are missing images here,
552 @c in the short term we should add them back, in the long term this should
553 @c work without images or have images with alt-text.
555 GNUnet developers like LEGOs. The blocks are indestructible, can be
556 stacked together to construct complex buildings and it is generally easy
557 to swap one block for a different one that has the same shape. GNUnet's
558 architecture is based on LEGOs:
560 @image{images/service_lego_block,5in,,picture of a LEGO block stack - 3 APIs upon IPC/network protocol provided by a service}
562 This chapter documents the GNUnet LEGO system, also known as GNUnet's
565 The most common GNUnet component is a service. Services offer an API (or
566 several, depending on what you count as "an API") which is implemented as
567 a library. The library communicates with the main process of the service
568 using a service-specific network protocol. The main process of the service
569 typically doesn't fully provide everything that is needed --- it has holes
570 to be filled by APIs to other services.
572 A special kind of component in GNUnet are user interfaces and daemons.
573 Like services, they have holes to be filled by APIs of other services.
574 Unlike services, daemons do not implement their own network protocol and
577 @image{images/daemon_lego_block,5in,,A daemon in GNUnet is a component that does not offer an API for others to build upon}
579 The GNUnet system provides a range of services, daemons and user
580 interfaces, which are then combined into a layered GNUnet instance (also
583 @image{images/service_stack,5in,,A GNUnet peer consists of many layers of services}
585 Note that while it is generally possible to swap one service for another
586 compatible service, there is often only one implementation. However,
587 during development we often have a "new" version of a service in parallel
588 with an "old" version. While the "new" version is not working, developers
589 working on other parts of the service can continue their development by
590 simply using the "old" service. Alternative design ideas can also be
591 easily investigated by swapping out individual components. This is
592 typically achieved by simply changing the name of the "BINARY" in the
593 respective configuration section.
595 Key properties of GNUnet services are that they must be separate
596 processes and that they must protect themselves by applying tight error
597 checking against the network protocol they implement (thereby achieving a
598 certain degree of robustness).
600 On the other hand, the APIs are implemented to tolerate failures of the
601 service, isolating their host process from errors by the service. If the
602 service process crashes, other services and daemons around it should not
603 also fail, but instead wait for the service process to be restarted by
607 @c ***********************************************************************
608 @node Subsystem stability
609 @section Subsystem stability
611 This section documents the current stability of the various GNUnet
612 subsystems. Stability here describes the expected degree of compatibility
613 with future versions of GNUnet. For each subsystem we distinguish between
614 compatibility on the P2P network level (communication protocol between
615 peers), the IPC level (communication between the service and the service
616 library) and the API level (stability of the API). P2P compatibility is
617 relevant in terms of which applications are likely going to be able to
618 communicate with future versions of the network. IPC communication is
619 relevant for the implementation of language bindings that re-implement the
620 IPC messages. Finally, API compatibility is relevant to developers that
621 hope to be able to avoid changes to applications build on top of the APIs
624 The following table summarizes our current view of the stability of the
625 respective protocols or APIs:
627 @multitable @columnfractions .20 .20 .20 .20
628 @headitem Subsystem @tab P2P @tab IPC @tab C API
629 @item util @tab n/a @tab n/a @tab stable
630 @item arm @tab n/a @tab stable @tab stable
631 @item ats @tab n/a @tab unstable @tab testing
632 @item block @tab n/a @tab n/a @tab stable
633 @item cadet @tab testing @tab testing @tab testing
634 @item consensus @tab experimental @tab experimental @tab experimental
635 @item core @tab stable @tab stable @tab stable
636 @item datacache @tab n/a @tab n/a @tab stable
637 @item datastore @tab n/a @tab stable @tab stable
638 @item dht @tab stable @tab stable @tab stable
639 @item dns @tab stable @tab stable @tab stable
640 @item dv @tab testing @tab testing @tab n/a
641 @item exit @tab testing @tab n/a @tab n/a
642 @item fragmentation @tab stable @tab n/a @tab stable
643 @item fs @tab stable @tab stable @tab stable
644 @item gns @tab stable @tab stable @tab stable
645 @item hello @tab n/a @tab n/a @tab testing
646 @item hostlist @tab stable @tab stable @tab n/a
647 @item identity @tab stable @tab stable @tab n/a
648 @item multicast @tab experimental @tab experimental @tab experimental
649 @item mysql @tab stable @tab n/a @tab stable
650 @item namestore @tab n/a @tab stable @tab stable
651 @item nat @tab n/a @tab n/a @tab stable
652 @item nse @tab stable @tab stable @tab stable
653 @item peerinfo @tab n/a @tab stable @tab stable
654 @item psyc @tab experimental @tab experimental @tab experimental
655 @item pt @tab n/a @tab n/a @tab n/a
656 @item regex @tab stable @tab stable @tab stable
657 @item revocation @tab stable @tab stable @tab stable
658 @item social @tab experimental @tab experimental @tab experimental
659 @item statistics @tab n/a @tab stable @tab stable
660 @item testbed @tab n/a @tab testing @tab testing
661 @item testing @tab n/a @tab n/a @tab testing
662 @item topology @tab n/a @tab n/a @tab n/a
663 @item transport @tab stable @tab stable @tab stable
664 @item tun @tab n/a @tab n/a @tab stable
665 @item vpn @tab testing @tab n/a @tab n/a
668 Here is a rough explanation of the values:
672 No incompatible changes are planned at this time; for IPC/APIs, if
673 there are incompatible changes, they will be minor and might only require
674 minimal changes to existing code; for P2P, changes will be avoided if at
675 all possible for the 0.10.x-series
678 No incompatible changes are
679 planned at this time, but the code is still known to be in flux; so while
680 we have no concrete plans, our expectation is that there will still be
681 minor modifications; for P2P, changes will likely be extensions that
682 should not break existing code
685 Changes are planned and will happen; however, they
686 will not be totally radical and the result should still resemble what is
687 there now; nevertheless, anticipated changes will break protocol/API
691 Changes are planned and the result may look nothing like
692 what the API/protocol looks like today
695 Someone should think about where this subsystem headed
698 This subsystem does not have an API/IPC-protocol/P2P-protocol
701 @c ***********************************************************************
702 @node Naming conventions and coding style guide
703 @section Naming conventions and coding style guide
705 Here you can find some rules to help you write code for GNUnet.
707 @c ***********************************************************************
709 * Naming conventions::
713 @node Naming conventions
714 @subsection Naming conventions
717 @c ***********************************************************************
724 * private (library-internal) symbols (including structs and macros)::
726 * performance tests::
731 @subsubsection include files
734 @item _lib: library without need for a process
735 @item _service: library that needs a service process
736 @item _plugin: plugin definition
737 @item _protocol: structs used in network protocol
740 @item gnunet_config.h --- generated
741 @item platform.h --- first included
742 @item plibc.h --- external library
743 @item gnunet_common.h --- fundamental routines
744 @item gnunet_directories.h --- generated
745 @item gettext.h --- external library
749 @c ***********************************************************************
751 @subsubsection binaries
754 @item gnunet-service-xxx: service process (has listen socket)
755 @item gnunet-daemon-xxx: daemon process (no listen socket)
756 @item gnunet-helper-xxx[-yyy]: SUID helper for module xxx
757 @item gnunet-yyy: command-line tool for end-users
758 @item libgnunet_plugin_xxx_yyy.so: plugin for API xxx
759 @item libgnunetxxx.so: library for API xxx
762 @c ***********************************************************************
764 @subsubsection logging
767 @item services and daemons use their directory name in
768 @code{GNUNET_log_setup} (i.e. 'core') and log using
770 @item command-line tools use their full name in
771 @code{GNUNET_log_setup} (i.e. 'gnunet-publish') and log using
773 @item service access libraries log using
774 '@code{GNUNET_log_from}' and use '@code{DIRNAME-api}' for the
775 component (i.e. 'core-api')
776 @item pure libraries (without associated service) use
777 '@code{GNUNET_log_from}' with the component set to their
778 library name (without lib or '@file{.so}'),
779 which should also be their directory name (i.e. '@file{nat}')
780 @item plugins should use '@code{GNUNET_log_from}'
781 with the directory name and the plugin name combined to produce
782 the component name (i.e. 'transport-tcp').
783 @item logging should be unified per-file by defining a
784 @code{LOG} macro with the appropriate arguments,
788 #define LOG(kind,...)
789 GNUNET_log_from (kind, "example-api",__VA_ARGS__)
794 @c ***********************************************************************
796 @subsubsection configuration
799 @item paths (that are substituted in all filenames) are in PATHS
800 (have as few as possible)
801 @item all options for a particular module (@file{src/MODULE})
802 are under @code{[MODULE]}
803 @item options for a plugin of a module
804 are under @code{[MODULE-PLUGINNAME]}
807 @c ***********************************************************************
808 @node exported symbols
809 @subsubsection exported symbols
812 @item must start with @code{GNUNET_modulename_} and be defined in
814 @item exceptions: those defined in @file{gnunet_common.h}
817 @c ***********************************************************************
818 @node private (library-internal) symbols (including structs and macros)
819 @subsubsection private (library-internal) symbols (including structs and macros)
822 @item must NOT start with any prefix
823 @item must not be exported in a way that linkers could use them or@ other
824 libraries might see them via headers; they must be either
825 declared/defined in C source files or in headers that are in the
826 respective directory under @file{src/modulename/} and NEVER be declared
827 in @file{src/include/}.
831 @subsubsection testcases
834 @item must be called @file{test_module-under-test_case-description.c}
835 @item "case-description" maybe omitted if there is only one test
838 @c ***********************************************************************
839 @node performance tests
840 @subsubsection performance tests
843 @item must be called @file{perf_module-under-test_case-description.c}
844 @item "case-description" maybe omitted if there is only one performance
846 @item Must only be run if @code{HAVE_BENCHMARKS} is satisfied
849 @c ***********************************************************************
850 @node src/ directories
851 @subsubsection src/ directories
854 @item gnunet-NAME: end-user applications (i.e., gnunet-search, gnunet-arm)
855 @item gnunet-service-NAME: service processes with accessor library (i.e.,
857 @item libgnunetNAME: accessor library (_service.h-header) or standalone
858 library (_lib.h-header)
859 @item gnunet-daemon-NAME: daemon process without accessor library (i.e.,
860 gnunet-daemon-hostlist) and no GNUnet management port
861 @item libgnunet_plugin_DIR_NAME: loadable plugins (i.e.,
862 libgnunet_plugin_transport_tcp)
867 @subsection Coding style
869 @c XXX: Adjust examples to GNU Standards!
871 @item We follow the GNU Coding Standards (@pxref{Top, The GNU Coding Standards,, standards, The GNU Coding Standards});
872 @item Indentation is done with spaces, two per level, no tabs;
873 @item C99 struct initialization is fine;
874 @item declare only one variable per line, for example:
891 @c TODO: include actual example from a file in source
894 This helps keep diffs small and forces developers to think precisely about
895 the type of every variable.
896 Note that @code{char *} is different from @code{const char*} and
897 @code{int} is different from @code{unsigned int} or @code{uint32_t}.
898 Each variable type should be chosen with care.
900 @item While @code{goto} should generally be avoided, having a
901 @code{goto} to the end of a function to a block of clean up
902 statements (free, close, etc.) can be acceptable.
904 @item Conditions should be written with constants on the left (to avoid
905 accidental assignment) and with the @code{true} target being either the
906 @code{error} case or the significantly simpler continuation. For example:
909 if (0 != stat ("filename,"
916 /* handle normal case here */
924 if (stat ("filename," &sbuf) == 0) @{
925 /* handle normal case here */
932 If possible, the error clause should be terminated with a @code{return} (or
933 @code{goto} to some cleanup routine) and in this case, the @code{else} clause
937 if (0 != stat ("filename",
943 /* handle normal case here */
946 This serves to avoid deep nesting. The 'constants on the left' rule
947 applies to all constants (including. @code{GNUNET_SCHEDULER_NO_TASK}),
948 NULL, and enums). With the two above rules (constants on left, errors in
949 'true' branch), there is only one way to write most branches correctly.
951 @item Combined assignments and tests are allowed if they do not hinder
952 code clarity. For example, one can write:
955 if (NULL == (value = lookup_function()))
962 @item Use @code{break} and @code{continue} wherever possible to avoid
963 deep(er) nesting. Thus, we would write:
967 while (NULL != (pos = next))
970 if (! should_free (pos))
972 GNUNET_CONTAINER_DLL_remove (head,
982 next = head; while (NULL != (pos = next)) @{
984 if (should_free (pos)) @{
985 /* unnecessary nesting! */
986 GNUNET_CONTAINER_DLL_remove (head, tail, pos);
992 @item We primarily use @code{for} and @code{while} loops.
993 A @code{while} loop is used if the method for advancing in the loop is
994 not a straightforward increment operation. In particular, we use:
998 while (NULL != (pos = next))
1001 if (! should_free (pos))
1003 GNUNET_CONTAINER_DLL_remove (head,
1010 to free entries in a list (as the iteration changes the structure of the
1011 list due to the free; the equivalent @code{for} loop does no longer
1012 follow the simple @code{for} paradigm of @code{for(INIT;TEST;INC)}).
1013 However, for loops that do follow the simple @code{for} paradigm we do
1014 use @code{for}, even if it involves linked lists:
1017 /* simple iteration over a linked list */
1027 @item The first argument to all higher-order functions in GNUnet must be
1028 declared to be of type @code{void *} and is reserved for a closure. We do
1029 not use inner functions, as trampolines would conflict with setups that
1030 use non-executable stacks.
1031 The first statement in a higher-order function, which unusually should
1032 be part of the variable declarations, should assign the
1033 @code{cls} argument to the precise expected type. For example:
1037 callback (void *cls,
1040 struct Foo *foo = cls;
1041 int other_variables;
1043 /* rest of function */
1047 @item As shown in the example above, after the return type of a
1048 function there should be a break. Each parameter should
1051 @item It is good practice to write complex @code{if} expressions instead
1052 of using deeply nested @code{if} statements. However, except for addition
1053 and multiplication, all operators should use parens. This is fine:
1063 However, this is not:
1068 if (0 == bar && x != y)
1073 Note that splitting the @code{if} statement above is debatable as the
1074 @code{return x} is a very trivial statement. However, once the logic after
1075 the branch becomes more complicated (and is still identical), the "or"
1076 formulation should be used for sure.
1078 @item There should be two empty lines between the end of the function and
1079 the comments describing the following function. There should be a single
1080 empty line after the initial variable declarations of a function. If a
1081 function has no local variables, there should be no initial empty line. If
1082 a long function consists of several complex steps, those steps might be
1083 separated by an empty line (possibly followed by a comment describing the
1084 following step). The code should not contain empty lines in arbitrary
1085 places; if in doubt, it is likely better to NOT have an empty line (this
1086 way, more code will fit on the screen).
1089 @c ***********************************************************************
1091 @section Build-system
1093 If you have code that is likely not to compile or build rules you might
1094 want to not trigger for most developers, use @code{if HAVE_EXPERIMENTAL}
1095 in your @file{Makefile.am}.
1096 Then it is OK to (temporarily) add non-compiling (or known-to-not-port)
1099 If you want to compile all testcases but NOT run them, run configure with
1100 the @code{--enable-test-suppression} option.
1102 If you want to run all testcases, including those that take a while, run
1103 configure with the @code{--enable-expensive-testcases} option.
1105 If you want to compile and run benchmarks, run configure with the
1106 @code{--enable-benchmarks} option.
1108 If you want to obtain code coverage results, run configure with the
1109 @code{--enable-coverage} option and run the @file{coverage.sh} script in
1110 the @file{contrib/} directory.
1113 @node Developing extensions for GNUnet using the gnunet-ext template
1114 @section Developing extensions for GNUnet using the gnunet-ext template
1116 For developers who want to write extensions for GNUnet we provide the
1117 gnunet-ext template to provide an easy to use skeleton.
1119 gnunet-ext contains the build environment and template files for the
1120 development of GNUnet services, command line tools, APIs and tests.
1122 First of all you have to obtain gnunet-ext from git:
1125 git clone https://git.gnunet.org/gnunet-ext.git
1128 The next step is to bootstrap and configure it. For configure you have to
1129 provide the path containing GNUnet with
1130 @code{--with-gnunet=/path/to/gnunet} and the prefix where you want the
1131 install the extension using @code{--prefix=/path/to/install}:
1135 ./configure --prefix=/path/to/install --with-gnunet=/path/to/gnunet
1138 When your GNUnet installation is not included in the default linker search
1139 path, you have to add @code{/path/to/gnunet} to the file
1140 @file{/etc/ld.so.conf} and run @code{ldconfig} or your add it to the
1141 environmental variable @code{LD_LIBRARY_PATH} by using
1144 export LD_LIBRARY_PATH=/path/to/gnunet/lib
1147 @cindex writing testcases
1148 @node Writing testcases
1149 @section Writing testcases
1151 Ideally, any non-trivial GNUnet code should be covered by automated
1152 testcases. Testcases should reside in the same place as the code that is
1153 being tested. The name of source files implementing tests should begin
1154 with @code{test_} followed by the name of the file that contains
1155 the code that is being tested.
1157 Testcases in GNUnet should be integrated with the autotools build system.
1158 This way, developers and anyone building binary packages will be able to
1159 run all testcases simply by running @code{make check}. The final
1160 testcases shipped with the distribution should output at most some brief
1161 progress information and not display debug messages by default. The
1162 success or failure of a testcase must be indicated by returning zero
1163 (success) or non-zero (failure) from the main method of the testcase.
1164 The integration with the autotools is relatively straightforward and only
1165 requires modifications to the @file{Makefile.am} in the directory
1166 containing the testcase. For a testcase testing the code in @file{foo.c}
1167 the @file{Makefile.am} would contain the following lines:
1170 check_PROGRAMS = test_foo
1171 TESTS = $(check_PROGRAMS)
1172 test_foo_SOURCES = test_foo.c
1173 test_foo_LDADD = $(top_builddir)/src/util/libgnunetutil.la
1176 Naturally, other libraries used by the testcase may be specified in the
1177 @code{LDADD} directive as necessary.
1179 Often testcases depend on additional input files, such as a configuration
1180 file. These support files have to be listed using the @code{EXTRA_DIST}
1181 directive in order to ensure that they are included in the distribution.
1186 EXTRA_DIST = test_foo_data.conf
1189 Executing @code{make check} will run all testcases in the current
1190 directory and all subdirectories. Testcases can be compiled individually
1191 by running @code{make test_foo} and then invoked directly using
1192 @code{./test_foo}. Note that due to the use of plugins in GNUnet, it is
1193 typically necessary to run @code{make install} before running any
1194 testcases. Thus the canonical command @code{make check install} has to be
1195 changed to @code{make install check} for GNUnet.
1197 @c ***********************************************************************
1198 @cindex Building GNUnet
1199 @node Building GNUnet and its dependencies
1200 @section Building GNUnet and its dependencies
1202 In the following section we will outline how to build GNUnet and
1203 some of its dependencies. We will assume a fair amount of knowledge
1204 for building applications under UNIX-like systems. Furthermore we
1205 assume that the build environment is sane and that you are aware of
1206 any implications actions in this process could have.
1207 Instructions here can be seen as notes for developers (an extension to
1208 the 'HACKING' section in README) as well as package maintainers.
1209 @b{Users should rely on the available binary packages.}
1210 We will use Debian as an example Operating System environment. Substitute
1211 accordingly with your own Operating System environment.
1213 For the full list of dependencies, consult the appropriate, up-to-date
1214 section in the @file{README} file.
1216 First, we need to build or install (depending on your OS) the following
1217 packages. If you build them from source, build them in this exact order:
1220 libgpgerror, libgcrypt, libnettle, libunbound, GnuTLS (with libunbound
1224 After we have build and installed those packages, we continue with
1225 packages closer to GNUnet in this step: libgnurl (our libcurl fork),
1226 GNU libmicrohttpd, and GNU libextractor. Again, if your package manager
1227 provides one of these packages, use the packages provided from it
1228 unless you have good reasons (package version too old, conflicts, etc).
1229 We advise against compiling widely used packages such as GnuTLS
1230 yourself if your OS provides a variant already unless you take care
1231 of maintenance of the packages then.
1233 In the optimistic case, this command will give you all the dependencies:
1236 sudo apt-get install libgnurl libmicrohttpd libextractor
1239 From experience we know that at the very least libgnurl is not
1240 available in some environments. You could substitute libgnurl
1241 with libcurl, but we recommend to install libgnurl, as it gives
1242 you a predefined libcurl with the small set GNUnet requires. In
1243 the past namespaces of libcurl and libgnurl were shared, which
1244 caused problems when you wanted to integrate both of them in one
1245 Operating System. This has been resolved, and they can be installed
1249 @cindex compiling libgnurl
1250 GNUnet and some of its function depend on a limited subset of cURL/libcurl.
1251 Rather than trying to enforce a certain configuration on the world, we
1252 opted to maintain a microfork of it that ensures we can link against the
1253 right set of features. We called this specialized set of libcurl
1254 ``libgnurl''. It is fully ABI compatible with libcurl and currently used
1255 by GNUnet and some of its dependencies.
1257 We download libgnurl and its digital signature from the GNU fileserver,
1258 assuming @env{TMPDIR} exists.
1260 Note: TMPDIR might be @file{/tmp}, @env{TMPDIR}, @env{TMP} or any other
1261 location. For consistency we assume @env{TMPDIR} points to @file{/tmp}
1262 for the remainder of this section.
1266 wget https://ftp.gnu.org/gnu/gnunet/gnurl-7.60.0.tar.Z
1267 wget https://ftp.gnu.org/gnu/gnunet/gnurl-7.60.0.tar.Z.sig
1270 Next, verify the digital signature of the file:
1273 gpg --verify gnurl-7.60.0.tar.Z.sig
1276 If gpg fails, you might try with @command{gpg2} on your OS. If the error
1277 states that ``the key can not be found'' or it is unknown, you have to
1278 retrieve the key (A88C8ADD129828D7EAC02E52E22F9BBFEE348588) from a
1282 gpg --keyserver pgp.mit.edu --recv-keys A88C8ADD129828D7EAC02E52E22F9BBFEE348588
1285 and rerun the verification command.
1287 libgnurl will require the following packages to be present at runtime:
1288 gnutls (with DANE support / libunbound), libidn, zlib and at compile time:
1289 libtool, groff, perl, pkg-config, and python 2.7.
1291 Once you have verified that all the required packages are present on your
1292 system, we can proceed to compile libgnurl:
1295 tar -xvf gnurl-7.60.0.tar.Z
1297 sh configure --disable-ntlm-wb
1303 After you've compiled and installed libgnurl, we can proceed to building
1309 First, in addition to the GNUnet sources you might require downloading the
1310 latest version of various dependencies, depending on how recent the
1311 software versions in your distribution of GNU/Linux are.
1312 Most distributions do not include sufficiently recent versions of these
1314 Thus, a typically installation on a "modern" GNU/Linux distribution
1315 requires you to install the following dependencies (ideally in this
1319 @item libgpgerror and libgcrypt
1320 @item libnettle and libunbound (possibly from distribution), GnuTLS
1321 @item libgnurl (read the README)
1322 @item GNU libmicrohttpd
1323 @item GNU libextractor
1326 Make sure to first install the various mandatory and optional
1327 dependencies including development headers from your distribution.
1329 Other dependencies that you should strongly consider to install is a
1330 database (MySQL, sqlite or Postgres).
1331 The following instructions will assume that you installed at least sqlite.
1332 For most distributions you should be able to find pre-build packages for
1333 the database. Again, make sure to install the client libraries @b{and} the
1334 respective development headers (if they are packaged separately) as well.
1336 You can find specific, detailed instructions for installing of the
1337 dependencies (and possibly the rest of the GNUnet installation) in the
1338 platform-specific descriptions, which can be found in the Index.
1339 Please consult them now.
1340 If your distribution is not listed, please study the build
1341 instructions for Debian stable, carefully as you try to install the
1342 dependencies for your own distribution.
1343 Contributing additional instructions for further platforms is always
1345 Please take in mind that operating system development tends to move at
1346 a rather fast speed. Due to this you should be aware that some of
1347 the instructions could be outdated by the time you are reading this.
1348 If you find a mistake, please tell us about it (or even better: send
1349 a patch to the documentation to fix it!).
1351 Before proceeding further, please double-check the dependency list.
1352 Note that in addition to satisfying the dependencies, you might have to
1353 make sure that development headers for the various libraries are also
1355 There maybe files for other distributions, or you might be able to find
1356 equivalent packages for your distribution.
1358 While it is possible to build and install GNUnet without having root
1359 access, we will assume that you have full control over your system in
1361 First, you should create a system user @emph{gnunet} and an additional
1362 group @emph{gnunetdns}. On the GNU/Linux distributions Debian and Ubuntu,
1366 sudo adduser --system --home /var/lib/gnunet --group \
1367 --disabled-password gnunet
1368 sudo addgroup --system gnunetdns
1372 On other Unixes and GNU systems, this should have the same effect:
1375 sudo useradd --system --groups gnunet --home-dir /var/lib/gnunet
1376 sudo addgroup --system gnunetdns
1379 Now compile and install GNUnet using:
1382 tar xvf gnunet-@value{VERSION}.tar.gz
1383 cd gnunet-@value{VERSION}
1384 ./configure --with-sudo=sudo --with-nssdir=/lib
1389 If you want to be able to enable DEBUG-level log messages, add
1390 @code{--enable-logging=verbose} to the end of the
1391 @command{./configure} command.
1392 @code{DEBUG}-level log messages are in English only and
1393 should only be useful for developers (or for filing
1394 really detailed bug reports).
1397 Next, edit the file @file{/etc/gnunet.conf} to contain the following:
1401 START_SYSTEM_SERVICES = YES
1402 START_USER_SERVICES = NO
1406 You may need to update your @code{ld.so} cache to include
1407 files installed in @file{/usr/local/lib}:
1414 Then, switch from user @code{root} to user @code{gnunet} to start
1418 # su -s /bin/sh - gnunet
1419 $ gnunet-arm -c /etc/gnunet.conf -s
1422 You may also want to add the last line in the gnunet user's @file{crontab}
1423 prefixed with @code{@@reboot} so that it is executed whenever the system
1427 @@reboot /usr/local/bin/gnunet-arm -c /etc/gnunet.conf -s
1431 This will only start the system-wide GNUnet services.
1432 Type @command{exit} to get back your root shell.
1433 Now, you need to configure the per-user part. For each
1434 user that should get access to GNUnet on the system, run
1435 (replace alice with your username):
1438 sudo adduser alice gnunet
1442 to allow them to access the system-wide GNUnet services. Then, each
1443 user should create a configuration file @file{~/.config/gnunet.conf}
1448 START_SYSTEM_SERVICES = NO
1449 START_USER_SERVICES = YES
1450 DEFAULTSERVICES = gns
1454 and start the per-user services using
1457 $ gnunet-arm -c ~/.config/gnunet.conf -s
1461 Again, adding a @code{crontab} entry to autostart the peer is advised:
1464 @@reboot /usr/local/bin/gnunet-arm -c $HOME/.config/gnunet.conf -s
1468 Note that some GNUnet services (such as SOCKS5 proxies) may need a
1469 system-wide TCP port for each user.
1470 For those services, systems with more than one user may require each user
1471 to specify a different port number in their personal configuration file.
1473 Finally, the user should perform the basic initial setup for the GNU Name
1474 System (GNS) certificate authority. This is done by running:
1477 $ gnunet-gns-proxy-setup-ca
1481 The first generates the default zones, whereas the second setups the GNS
1482 Certificate Authority with the user's browser. Now, to activate GNS in the
1483 normal DNS resolution process, you need to edit your
1484 @file{/etc/nsswitch.conf} where you should find a line like this:
1487 hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4
1491 The exact details may differ a bit, which is fine. Add the text
1492 @emph{"gns [NOTFOUND=return]"} after @emph{"files"}.
1493 Keep in mind that we included a backslash ("\") here just for
1494 markup reasons. You should write the text below on @b{one line}
1495 and @b{without} the "\":
1498 hosts: files gns [NOTFOUND=return] mdns4_minimal \
1499 [NOTFOUND=return] dns mdns4
1502 @c FIXME: Document new behavior.
1503 You might want to make sure that @file{/lib/libnss_gns.so.2} exists on
1504 your system, it should have been created during the installation.
1507 @c **********************************************************************
1508 @cindex TESTING library
1509 @node TESTING library
1510 @section TESTING library
1512 The TESTING library is used for writing testcases which involve starting a
1513 single or multiple peers. While peers can also be started by testcases
1514 using the ARM subsystem, using TESTING library provides an elegant way to
1515 do this. The configurations of the peers are auto-generated from a given
1516 template to have non-conflicting port numbers ensuring that peers'
1517 services do not run into bind errors. This is achieved by testing ports'
1518 availability by binding a listening socket to them before allocating them
1519 to services in the generated configurations.
1521 An another advantage while using TESTING is that it shortens the testcase
1522 startup time as the hostkeys for peers are copied from a pre-computed set
1523 of hostkeys instead of generating them at peer startup which may take a
1524 considerable amount of time when starting multiple peers or on an embedded
1527 TESTING also allows for certain services to be shared among peers. This
1528 feature is invaluable when testing with multiple peers as it helps to
1529 reduce the number of services run per each peer and hence the total
1530 number of processes run per testcase.
1532 TESTING library only handles creating, starting and stopping peers.
1533 Features useful for testcases such as connecting peers in a topology are
1534 not available in TESTING but are available in the TESTBED subsystem.
1535 Furthermore, TESTING only creates peers on the localhost, however by
1536 using TESTBED testcases can benefit from creating peers across multiple
1541 * Finer control over peer stop::
1542 * Helper functions::
1543 * Testing with multiple processes::
1550 TESTING abstracts a group of peers as a TESTING system. All peers in a
1551 system have common hostname and no two services of these peers have a
1552 same port or a UNIX domain socket path.
1554 TESTING system can be created with the function
1555 @code{GNUNET_TESTING_system_create()} which returns a handle to the
1556 system. This function takes a directory path which is used for generating
1557 the configurations of peers, an IP address from which connections to the
1558 peers' services should be allowed, the hostname to be used in peers'
1559 configuration, and an array of shared service specifications of type
1560 @code{struct GNUNET_TESTING_SharedService}.
1562 The shared service specification must specify the name of the service to
1563 share, the configuration pertaining to that shared service and the
1564 maximum number of peers that are allowed to share a single instance of
1567 TESTING system created with @code{GNUNET_TESTING_system_create()} chooses
1568 ports from the default range @code{12000} - @code{56000} while
1569 auto-generating configurations for peers.
1570 This range can be customised with the function
1571 @code{GNUNET_TESTING_system_create_with_portrange()}. This function is
1572 similar to @code{GNUNET_TESTING_system_create()} except that it take 2
1573 additional parameters --- the start and end of the port range to use.
1575 A TESTING system is destroyed with the function
1576 @code{GNUNET_TESTING_system_destory()}. This function takes the handle of
1577 the system and a flag to remove the files created in the directory used
1578 to generate configurations.
1580 A peer is created with the function
1581 @code{GNUNET_TESTING_peer_configure()}. This functions takes the system
1582 handle, a configuration template from which the configuration for the peer
1583 is auto-generated and the index from where the hostkey for the peer has to
1584 be copied from. When successful, this function returns a handle to the
1585 peer which can be used to start and stop it and to obtain the identity of
1586 the peer. If unsuccessful, a NULL pointer is returned with an error
1587 message. This function handles the generated configuration to have
1588 non-conflicting ports and paths.
1590 Peers can be started and stopped by calling the functions
1591 @code{GNUNET_TESTING_peer_start()} and @code{GNUNET_TESTING_peer_stop()}
1592 respectively. A peer can be destroyed by calling the function
1593 @code{GNUNET_TESTING_peer_destroy}. When a peer is destroyed, the ports
1594 and paths in allocated in its configuration are reclaimed for usage in new
1597 @c ***********************************************************************
1598 @node Finer control over peer stop
1599 @subsection Finer control over peer stop
1601 Using @code{GNUNET_TESTING_peer_stop()} is normally fine for testcases.
1602 However, calling this function for each peer is inefficient when trying to
1603 shutdown multiple peers as this function sends the termination signal to
1604 the given peer process and waits for it to terminate. It would be faster
1605 in this case to send the termination signals to the peers first and then
1606 wait on them. This is accomplished by the functions
1607 @code{GNUNET_TESTING_peer_kill()} which sends a termination signal to the
1608 peer, and the function @code{GNUNET_TESTING_peer_wait()} which waits on
1611 Further finer control can be achieved by choosing to stop a peer
1612 asynchronously with the function @code{GNUNET_TESTING_peer_stop_async()}.
1613 This function takes a callback parameter and a closure for it in addition
1614 to the handle to the peer to stop. The callback function is called with
1615 the given closure when the peer is stopped. Using this function
1616 eliminates blocking while waiting for the peer to terminate.
1618 An asynchronous peer stop can be canceled by calling the function
1619 @code{GNUNET_TESTING_peer_stop_async_cancel()}. Note that calling this
1620 function does not prevent the peer from terminating if the termination
1621 signal has already been sent to it. It does, however, cancels the
1622 callback to be called when the peer is stopped.
1624 @c ***********************************************************************
1625 @node Helper functions
1626 @subsection Helper functions
1628 Most of the testcases can benefit from an abstraction which configures a
1629 peer and starts it. This is provided by the function
1630 @code{GNUNET_TESTING_peer_run()}. This function takes the testing
1631 directory pathname, a configuration template, a callback and its closure.
1632 This function creates a peer in the given testing directory by using the
1633 configuration template, starts the peer and calls the given callback with
1636 The function @code{GNUNET_TESTING_peer_run()} starts the ARM service of
1637 the peer which starts the rest of the configured services. A similar
1638 function @code{GNUNET_TESTING_service_run} can be used to just start a
1639 single service of a peer. In this case, the peer's ARM service is not
1640 started; instead, only the given service is run.
1642 @c ***********************************************************************
1643 @node Testing with multiple processes
1644 @subsection Testing with multiple processes
1646 When testing GNUnet, the splitting of the code into a services and clients
1647 often complicates testing. The solution to this is to have the testcase
1648 fork @code{gnunet-service-arm}, ask it to start the required server and
1649 daemon processes and then execute appropriate client actions (to test the
1650 client APIs or the core module or both). If necessary, multiple ARM
1651 services can be forked using different ports (!) to simulate a network.
1652 However, most of the time only one ARM process is needed. Note that on
1653 exit, the testcase should shutdown ARM with a @code{TERM} signal (to give
1654 it the chance to cleanly stop its child processes).
1656 The following code illustrates spawning and killing an ARM process from a
1660 static void run (void *cls,
1662 const char *cfgfile,
1663 const struct GNUNET_CONFIGURATION_Handle *cfg) @{
1664 struct GNUNET_OS_Process *arm_pid;
1665 arm_pid = GNUNET_OS_start_process (NULL,
1667 "gnunet-service-arm",
1668 "gnunet-service-arm",
1672 /* do real test work here */
1673 if (0 != GNUNET_OS_process_kill (arm_pid, SIGTERM))
1675 (GNUNET_ERROR_TYPE_WARNING, "kill");
1676 GNUNET_assert (GNUNET_OK == GNUNET_OS_process_wait (arm_pid));
1677 GNUNET_OS_process_close (arm_pid); @}
1679 GNUNET_PROGRAM_run (argc, argv,
1688 An alternative way that works well to test plugins is to implement a
1689 mock-version of the environment that the plugin expects and then to
1690 simply load the plugin directly.
1692 @c ***********************************************************************
1693 @node Performance regression analysis with Gauger
1694 @section Performance regression analysis with Gauger
1696 To help avoid performance regressions, GNUnet uses Gauger. Gauger is a
1697 simple logging tool that allows remote hosts to send performance data to
1698 a central server, where this data can be analyzed and visualized. Gauger
1699 shows graphs of the repository revisions and the performance data recorded
1700 for each revision, so sudden performance peaks or drops can be identified
1701 and linked to a specific revision number.
1703 In the case of GNUnet, the buildbots log the performance data obtained
1704 during the tests after each build. The data can be accessed on GNUnet's
1707 The menu on the left allows to select either the results of just one
1708 build bot (under "Hosts") or review the data from all hosts for a given
1709 test result (under "Metrics"). In case of very different absolute value
1710 of the results, for instance arm vs. amd64 machines, the option
1711 "Normalize" on a metric view can help to get an idea about the
1712 performance evolution across all hosts.
1714 Using Gauger in GNUnet and having the performance of a module tracked over
1715 time is very easy. First of course, the testcase must generate some
1716 consistent metric, which makes sense to have logged. Highly volatile or
1717 random dependent metrics probably are not ideal candidates for meaningful
1718 regression detection.
1720 To start logging any value, just include @code{gauger.h} in your testcase
1721 code. Then, use the macro @code{GAUGER()} to make the Buildbots log
1722 whatever value is of interest for you to @code{gnunet.org}'s Gauger
1723 server. No setup is necessary as most Buildbots have already everything
1724 in place and new metrics are created on demand. To delete a metric, you
1725 need to contact a member of the GNUnet development team (a file will need
1726 to be removed manually from the respective directory).
1728 The code in the test should look like this:
1734 int main (int argc, char *argv[]) @{
1736 [run test, generate data]
1737 GAUGER("YOUR_MODULE",
1747 @item @strong{YOUR_MODULE} is a category in the gauger page and should be
1748 the name of the module or subsystem like "Core" or "DHT"
1749 @item @strong{METRIC} is
1750 the name of the metric being collected and should be concise and
1751 descriptive, like "PUT operations in sqlite-datastore".
1752 @item @strong{value} is the value
1753 of the metric that is logged for this run.
1754 @item @strong{UNIT} is the unit in
1755 which the value is measured, for instance "kb/s" or "kb of RAM/node".
1758 If you wish to use Gauger for your own project, you can grab a copy of the
1759 latest stable release or check out Gauger's Subversion repository.
1761 @cindex TESTBED Subsystem
1762 @node TESTBED Subsystem
1763 @section TESTBED Subsystem
1765 The TESTBED subsystem facilitates testing and measuring of multi-peer
1766 deployments on a single host or over multiple hosts.
1768 The architecture of the testbed module is divided into the following:
1771 @item Testbed API: An API which is used by the testing driver programs. It
1772 provides with functions for creating, destroying, starting, stopping
1775 @item Testbed service (controller): A service which is started through the
1776 Testbed API. This service handles operations to create, destroy, start,
1777 stop peers, connect them, modify their configurations.
1779 @item Testbed helper: When a controller has to be started on a host, the
1780 testbed API starts the testbed helper on that host which in turn starts
1781 the controller. The testbed helper receives a configuration for the
1782 controller through its stdin and changes it to ensure the controller
1783 doesn't run into any port conflict on that host.
1787 The testbed service (controller) is different from the other GNUnet
1788 services in that it is not started by ARM and is not supposed to be run
1789 as a daemon. It is started by the testbed API through a testbed helper.
1790 In a typical scenario involving multiple hosts, a controller is started
1791 on each host. Controllers take up the actual task of creating peers,
1792 starting and stopping them on the hosts they run.
1794 While running deployments on a single localhost the testbed API starts the
1795 testbed helper directly as a child process. When running deployments on
1796 remote hosts the testbed API starts Testbed Helpers on each remote host
1797 through remote shell. By default testbed API uses SSH as a remote shell.
1798 This can be changed by setting the environmental variable
1799 GNUNET_TESTBED_RSH_CMD to the required remote shell program. This
1800 variable can also contain parameters which are to be passed to the remote
1801 shell program. For e.g:
1804 export GNUNET_TESTBED_RSH_CMD="ssh -o BatchMode=yes \
1805 -o NoHostAuthenticationForLocalhost=yes %h"
1808 Substitutions are allowed in the command string above,
1809 this allows for substitutions through placemarks which begin with a `%'.
1810 At present the following substitutions are supported
1818 Note that the substitution placemark is replaced only when the
1819 corresponding field is available and only once. Specifying
1825 doesn't work either. If you want to user username substitutions for
1826 @command{SSH}, use the argument @code{-l} before the
1827 username substitution.
1834 The testbed API and the helper communicate through the helpers stdin and
1835 stdout. As the helper is started through a remote shell on remote hosts
1836 any output messages from the remote shell interfere with the communication
1837 and results in a failure while starting the helper. For this reason, it is
1838 suggested to use flags to make the remote shells produce no output
1839 messages and to have password-less logins. The default remote shell, SSH,
1840 the default options are:
1843 -o BatchMode=yes -o NoHostBasedAuthenticationForLocalhost=yes"
1846 Password-less logins should be ensured by using SSH keys.
1848 Since the testbed API executes the remote shell as a non-interactive
1849 shell, certain scripts like .bashrc, .profiler may not be executed. If
1850 this is the case testbed API can be forced to execute an interactive
1851 shell by setting up the environmental variable
1852 @code{GNUNET_TESTBED_RSH_CMD_SUFFIX} to a shell program.
1854 An example could be:
1857 export GNUNET_TESTBED_RSH_CMD_SUFFIX="sh -lc"
1860 The testbed API will then execute the remote shell program as:
1863 $GNUNET_TESTBED_RSH_CMD -p $port $dest $GNUNET_TESTBED_RSH_CMD_SUFFIX \
1864 gnunet-helper-testbed
1867 On some systems, problems may arise while starting testbed helpers if
1868 GNUnet is installed into a custom location since the helper may not be
1869 found in the standard path. This can be addressed by setting the variable
1870 `@code{HELPER_BINARY_PATH}' to the path of the testbed helper.
1871 Testbed API will then use this path to start helper binaries both
1872 locally and remotely.
1874 Testbed API can accessed by including the
1875 @file{gnunet_testbed_service.h} file and linking with
1876 @code{-lgnunettestbed}.
1878 @c ***********************************************************************
1880 * Supported Topologies::
1881 * Hosts file format::
1882 * Topology file format::
1883 * Testbed Barriers::
1887 @node Supported Topologies
1888 @subsection Supported Topologies
1890 While testing multi-peer deployments, it is often needed that the peers
1891 are connected in some topology. This requirement is addressed by the
1892 function @code{GNUNET_TESTBED_overlay_connect()} which connects any given
1893 two peers in the testbed.
1895 The API also provides a helper function
1896 @code{GNUNET_TESTBED_overlay_configure_topology()} to connect a given set
1897 of peers in any of the following supported topologies:
1901 @item @code{GNUNET_TESTBED_TOPOLOGY_CLIQUE}: All peers are connected with
1904 @item @code{GNUNET_TESTBED_TOPOLOGY_LINE}: Peers are connected to form a
1907 @item @code{GNUNET_TESTBED_TOPOLOGY_RING}: Peers are connected to form a
1910 @item @code{GNUNET_TESTBED_TOPOLOGY_2D_TORUS}: Peers are connected to
1911 form a 2 dimensional torus topology. The number of peers may not be a
1912 perfect square, in that case the resulting torus may not have the uniform
1913 poloidal and toroidal lengths
1915 @item @code{GNUNET_TESTBED_TOPOLOGY_ERDOS_RENYI}: Topology is generated
1916 to form a random graph. The number of links to be present should be given
1918 @item @code{GNUNET_TESTBED_TOPOLOGY_SMALL_WORLD}: Peers are connected to
1919 form a 2D Torus with some random links among them. The number of random
1920 links are to be given
1922 @item @code{GNUNET_TESTBED_TOPOLOGY_SMALL_WORLD_RING}: Peers are
1923 connected to form a ring with some random links among them. The number of
1924 random links are to be given
1926 @item @code{GNUNET_TESTBED_TOPOLOGY_SCALE_FREE}: Connects peers in a
1927 topology where peer connectivity follows power law - new peers are
1928 connected with high probability to well connected peers.
1929 (See Emergence of Scaling in Random Networks. Science 286,
1931 (@uref{https://git.gnunet.org/bibliography.git/plain/docs/emergence_of_scaling_in_random_networks__barabasi_albert_science_286__1999.pdf, pdf}))
1933 @item @code{GNUNET_TESTBED_TOPOLOGY_FROM_FILE}: The topology information
1934 is loaded from a file. The path to the file has to be given.
1935 @xref{Topology file format}, for the format of this file.
1937 @item @code{GNUNET_TESTBED_TOPOLOGY_NONE}: No topology
1941 The above supported topologies can be specified respectively by setting
1942 the variable @code{OVERLAY_TOPOLOGY} to the following values in the
1943 configuration passed to Testbed API functions
1944 @code{GNUNET_TESTBED_test_run()} and
1945 @code{GNUNET_TESTBED_run()}:
1951 @item @code{2D_TORUS}
1953 @item @code{SMALL_WORLD}
1954 @item @code{SMALL_WORLD_RING}
1955 @item @code{SCALE_FREE}
1956 @item @code{FROM_FILE}
1961 Topologies @code{RANDOM}, @code{SMALL_WORLD} and @code{SMALL_WORLD_RING}
1962 require the option @code{OVERLAY_RANDOM_LINKS} to be set to the number of
1963 random links to be generated in the configuration. The option will be
1964 ignored for the rest of the topologies.
1966 Topology @code{SCALE_FREE} requires the options
1967 @code{SCALE_FREE_TOPOLOGY_CAP} to be set to the maximum number of peers
1968 which can connect to a peer and @code{SCALE_FREE_TOPOLOGY_M} to be set to
1969 how many peers a peer should be at least connected to.
1971 Similarly, the topology @code{FROM_FILE} requires the option
1972 @code{OVERLAY_TOPOLOGY_FILE} to contain the path of the file containing
1973 the topology information. This option is ignored for the rest of the
1974 topologies. @xref{Topology file format}, for the format of this file.
1976 @c ***********************************************************************
1977 @node Hosts file format
1978 @subsection Hosts file format
1980 The testbed API offers the function
1981 @code{GNUNET_TESTBED_hosts_load_from_file()} to load from a given file
1982 details about the hosts which testbed can use for deploying peers.
1983 This function is useful to keep the data about hosts
1984 separate instead of hard coding them in code.
1986 Another helper function from testbed API, @code{GNUNET_TESTBED_run()}
1987 also takes a hosts file name as its parameter. It uses the above
1988 function to populate the hosts data structures and start controllers to
1991 These functions require the hosts file to be of the following format:
1993 @item Each line is interpreted to have details about a host
1994 @item Host details should include the username to use for logging into the
1995 host, the hostname of the host and the port number to use for the remote
1996 shell program. All thee values should be given.
1997 @item These details should be given in the following format:
1999 <username>@@<hostname>:<port>
2003 Note that having canonical hostnames may cause problems while resolving
2004 the IP addresses (See this bug). Hence it is advised to provide the hosts'
2005 IP numerical addresses as hostnames whenever possible.
2007 @c ***********************************************************************
2008 @node Topology file format
2009 @subsection Topology file format
2011 A topology file describes how peers are to be connected. It should adhere
2012 to the following format for testbed to parse it correctly.
2014 Each line should begin with the target peer id. This should be followed by
2015 a colon(`:') and origin peer ids separated by `|'. All spaces except for
2016 newline characters are ignored. The API will then try to connect each
2017 origin peer to the target peer.
2019 For example, the following file will result in 5 overlay connections:
2020 [2->1], [3->1],[4->3], [0->3], [2->0]@
2021 @code{@ 1:2|3@ 3:4| 0@ 0: 2@ }
2023 @c ***********************************************************************
2024 @node Testbed Barriers
2025 @subsection Testbed Barriers
2027 The testbed subsystem's barriers API facilitates coordination among the
2028 peers run by the testbed and the experiment driver. The concept is
2029 similar to the barrier synchronisation mechanism found in parallel
2030 programming or multi-threading paradigms - a peer waits at a barrier upon
2031 reaching it until the barrier is reached by a predefined number of peers.
2032 This predefined number of peers required to cross a barrier is also called
2033 quorum. We say a peer has reached a barrier if the peer is waiting for the
2034 barrier to be crossed. Similarly a barrier is said to be reached if the
2035 required quorum of peers reach the barrier. A barrier which is reached is
2036 deemed as crossed after all the peers waiting on it are notified.
2038 The barriers API provides the following functions:
2040 @item @strong{@code{GNUNET_TESTBED_barrier_init()}:} function to
2041 initialize a barrier in the experiment
2042 @item @strong{@code{GNUNET_TESTBED_barrier_cancel()}:} function to cancel
2043 a barrier which has been initialized before
2044 @item @strong{@code{GNUNET_TESTBED_barrier_wait()}:} function to signal
2045 barrier service that the caller has reached a barrier and is waiting for
2047 @item @strong{@code{GNUNET_TESTBED_barrier_wait_cancel()}:} function to
2048 stop waiting for a barrier to be crossed
2052 Among the above functions, the first two, namely
2053 @code{GNUNET_TESTBED_barrier_init()} and
2054 @code{GNUNET_TESTBED_barrier_cancel()} are used by experiment drivers. All
2055 barriers should be initialised by the experiment driver by calling
2056 @code{GNUNET_TESTBED_barrier_init()}. This function takes a name to
2057 identify the barrier, the quorum required for the barrier to be crossed
2058 and a notification callback for notifying the experiment driver when the
2059 barrier is crossed. @code{GNUNET_TESTBED_barrier_cancel()} cancels an
2060 initialised barrier and frees the resources allocated for it. This
2061 function can be called upon a initialised barrier before it is crossed.
2063 The remaining two functions @code{GNUNET_TESTBED_barrier_wait()} and
2064 @code{GNUNET_TESTBED_barrier_wait_cancel()} are used in the peer's
2065 processes. @code{GNUNET_TESTBED_barrier_wait()} connects to the local
2066 barrier service running on the same host the peer is running on and
2067 registers that the caller has reached the barrier and is waiting for the
2068 barrier to be crossed. Note that this function can only be used by peers
2069 which are started by testbed as this function tries to access the local
2070 barrier service which is part of the testbed controller service. Calling
2071 @code{GNUNET_TESTBED_barrier_wait()} on an uninitialised barrier results
2072 in failure. @code{GNUNET_TESTBED_barrier_wait_cancel()} cancels the
2073 notification registered by @code{GNUNET_TESTBED_barrier_wait()}.
2076 @c ***********************************************************************
2081 @node Implementation
2082 @subsubsection Implementation
2084 Since barriers involve coordination between experiment driver and peers,
2085 the barrier service in the testbed controller is split into two
2086 components. The first component responds to the message generated by the
2087 barrier API used by the experiment driver (functions
2088 @code{GNUNET_TESTBED_barrier_init()} and
2089 @code{GNUNET_TESTBED_barrier_cancel()}) and the second component to the
2090 messages generated by barrier API used by peers (functions
2091 @code{GNUNET_TESTBED_barrier_wait()} and
2092 @code{GNUNET_TESTBED_barrier_wait_cancel()}).
2094 Calling @code{GNUNET_TESTBED_barrier_init()} sends a
2095 @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_INIT} message to the master
2096 controller. The master controller then registers a barrier and calls
2097 @code{GNUNET_TESTBED_barrier_init()} for each its subcontrollers. In this
2098 way barrier initialisation is propagated to the controller hierarchy.
2099 While propagating initialisation, any errors at a subcontroller such as
2100 timeout during further propagation are reported up the hierarchy back to
2101 the experiment driver.
2103 Similar to @code{GNUNET_TESTBED_barrier_init()},
2104 @code{GNUNET_TESTBED_barrier_cancel()} propagates
2105 @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_CANCEL} message which causes
2106 controllers to remove an initialised barrier.
2108 The second component is implemented as a separate service in the binary
2109 `gnunet-service-testbed' which already has the testbed controller service.
2110 Although this deviates from the gnunet process architecture of having one
2111 service per binary, it is needed in this case as this component needs
2112 access to barrier data created by the first component. This component
2113 responds to @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_WAIT} messages from
2114 local peers when they call @code{GNUNET_TESTBED_barrier_wait()}. Upon
2115 receiving @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_WAIT} message, the
2116 service checks if the requested barrier has been initialised before and
2117 if it was not initialised, an error status is sent through
2118 @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_STATUS} message to the local
2119 peer and the connection from the peer is terminated. If the barrier is
2120 initialised before, the barrier's counter for reached peers is incremented
2121 and a notification is registered to notify the peer when the barrier is
2122 reached. The connection from the peer is left open.
2124 When enough peers required to attain the quorum send
2125 @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_WAIT} messages, the controller
2126 sends a @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_STATUS} message to its
2127 parent informing that the barrier is crossed. If the controller has
2128 started further subcontrollers, it delays this message until it receives
2129 a similar notification from each of those subcontrollers. Finally, the
2130 barriers API at the experiment driver receives the
2131 @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_STATUS} when the barrier is
2132 reached at all the controllers.
2134 The barriers API at the experiment driver responds to the
2135 @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_STATUS} message by echoing it
2136 back to the master controller and notifying the experiment controller
2137 through the notification callback that a barrier has been crossed. The
2138 echoed @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_STATUS} message is
2139 propagated by the master controller to the controller hierarchy. This
2140 propagation triggers the notifications registered by peers at each of the
2141 controllers in the hierarchy. Note the difference between this downward
2142 propagation of the @code{GNUNET_MESSAGE_TYPE_TESTBED_BARRIER_STATUS}
2143 message from its upward propagation --- the upward propagation is needed
2144 for ensuring that the barrier is reached by all the controllers and the
2145 downward propagation is for triggering that the barrier is crossed.
2147 @cindex TESTBED Caveats
2148 @node TESTBED Caveats
2149 @subsection TESTBED Caveats
2151 This section documents a few caveats when using the GNUnet testbed
2154 @c ***********************************************************************
2156 * CORE must be started::
2157 * ATS must want the connections::
2160 @node CORE must be started
2161 @subsubsection CORE must be started
2163 A uncomplicated issue is bug #3993
2164 (@uref{https://bugs.gnunet.org/view.php?id=3993, https://bugs.gnunet.org/view.php?id=3993}):
2165 Your configuration MUST somehow ensure that for each peer the
2166 @code{CORE} service is started when the peer is setup, otherwise
2167 @code{TESTBED} may fail to connect peers when the topology is initialized,
2168 as @code{TESTBED} will start some @code{CORE} services but not
2169 necessarily all (but it relies on all of them running). The easiest way
2174 IMMEDIATE_START = YES
2178 in the configuration file.
2179 Alternatively, having any service that directly or indirectly depends on
2180 @code{CORE} being started with @code{IMMEDIATE_START} will also do.
2181 This issue largely arises if users try to over-optimize by not
2182 starting any services with @code{IMMEDIATE_START}.
2184 @c ***********************************************************************
2185 @node ATS must want the connections
2186 @subsubsection ATS must want the connections
2188 When TESTBED sets up connections, it only offers the respective HELLO
2189 information to the TRANSPORT service. It is then up to the ATS service to
2190 @strong{decide} to use the connection. The ATS service will typically
2191 eagerly establish any connection if the number of total connections is
2192 low (relative to bandwidth). Details may further depend on the
2193 specific ATS backend that was configured. If ATS decides to NOT establish
2194 a connection (even though TESTBED provided the required information), then
2195 that connection will count as failed for TESTBED. Note that you can
2196 configure TESTBED to tolerate a certain number of connection failures
2197 (see '-e' option of gnunet-testbed-profiler). This issue largely arises
2198 for dense overlay topologies, especially if you try to create cliques
2199 with more than 20 peers.
2201 @cindex libgnunetutil
2203 @section libgnunetutil
2205 libgnunetutil is the fundamental library that all GNUnet code builds upon.
2206 Ideally, this library should contain most of the platform dependent code
2207 (except for user interfaces and really special needs that only few
2208 applications have). It is also supposed to offer basic services that most
2209 if not all GNUnet binaries require. The code of libgnunetutil is in the
2210 @file{src/util/} directory. The public interface to the library is in the
2211 gnunet_util.h header. The functions provided by libgnunetutil fall
2212 roughly into the following categories (in roughly the order of importance
2213 for new developers):
2216 @item logging (common_logging.c)
2217 @item memory allocation (common_allocation.c)
2218 @item endianess conversion (common_endian.c)
2219 @item internationalization (common_gettext.c)
2220 @item String manipulation (string.c)
2221 @item file access (disk.c)
2222 @item buffered disk IO (bio.c)
2223 @item time manipulation (time.c)
2224 @item configuration parsing (configuration.c)
2225 @item command-line handling (getopt*.c)
2226 @item cryptography (crypto_*.c)
2227 @item data structures (container_*.c)
2228 @item CPS-style scheduling (scheduler.c)
2229 @item Program initialization (program.c)
2230 @item Networking (network.c, client.c, server*.c, service.c)
2231 @item message queuing (mq.c)
2232 @item bandwidth calculations (bandwidth.c)
2233 @item Other OS-related (os*.c, plugin.c, signal.c)
2234 @item Pseudonym management (pseudonym.c)
2237 It should be noted that only developers that fully understand this entire
2238 API will be able to write good GNUnet code.
2240 Ideally, porting GNUnet should only require porting the gnunetutil
2241 library. More testcases for the gnunetutil APIs are therefore a great
2242 way to make porting of GNUnet easier.
2246 * Interprocess communication API (IPC)::
2247 * Cryptography API::
2248 * Message Queue API::
2250 * Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps::
2251 * CONTAINER_MDLL API::
2259 GNUnet is able to log its activity, mostly for the purposes of debugging
2260 the program at various levels.
2262 @file{gnunet_common.h} defines several @strong{log levels}:
2265 @item ERROR for errors
2266 (really problematic situations, often leading to crashes)
2267 @item WARNING for warnings
2268 (troubling situations that might have negative consequences, although
2270 @item INFO for various information.
2271 Used somewhat rarely, as GNUnet statistics is used to hold and display
2272 most of the information that users might find interesting.
2273 @item DEBUG for debugging.
2274 Does not produce much output on normal builds, but when extra logging is
2275 enabled at compile time, a staggering amount of data is outputted under
2280 Normal builds of GNUnet (configured with @code{--enable-logging[=yes]})
2281 are supposed to log nothing under DEBUG level. The
2282 @code{--enable-logging=verbose} configure option can be used to create a
2283 build with all logging enabled. However, such build will produce large
2284 amounts of log data, which is inconvenient when one tries to hunt down a
2287 To mitigate this problem, GNUnet provides facilities to apply a filter to
2291 @item Logging by default When no log levels are configured in any other
2292 way (see below), GNUnet will default to the WARNING log level. This
2293 mostly applies to GNUnet command line utilities, services and daemons;
2294 tests will always set log level to WARNING or, if
2295 @code{--enable-logging=verbose} was passed to configure, to DEBUG. The
2296 default level is suggested for normal operation.
2297 @item The -L option Most GNUnet executables accept an "-L loglevel" or
2298 "--log=loglevel" option. If used, it makes the process set a global log
2299 level to "loglevel". Thus it is possible to run some processes
2300 with -L DEBUG, for example, and others with -L ERROR to enable specific
2301 settings to diagnose problems with a particular process.
2302 @item Configuration files. Because GNUnet
2303 service and daemon processes are usually launched by gnunet-arm, it is not
2304 possible to pass different custom command line options directly to every
2305 one of them. The options passed to @code{gnunet-arm} only affect
2306 gnunet-arm and not the rest of GNUnet. However, one can specify a
2307 configuration key "OPTIONS" in the section that corresponds to a service
2308 or a daemon, and put a value of "-L loglevel" there. This will make the
2309 respective service or daemon set its log level to "loglevel" (as the
2310 value of OPTIONS will be passed as a command-line argument).
2312 To specify the same log level for all services without creating separate
2313 "OPTIONS" entries in the configuration for each one, the user can specify
2314 a config key "GLOBAL_POSTFIX" in the [arm] section of the configuration
2315 file. The value of GLOBAL_POSTFIX will be appended to all command lines
2316 used by the ARM service to run other services. It can contain any option
2317 valid for all GNUnet commands, thus in particular the "-L loglevel"
2318 option. The ARM service itself is, however, unaffected by GLOBAL_POSTFIX;
2319 to set log level for it, one has to specify "OPTIONS" key in the [arm]
2321 @item Environment variables.
2322 Setting global per-process log levels with "-L loglevel" does not offer
2323 sufficient log filtering granularity, as one service will call interface
2324 libraries and supporting libraries of other GNUnet services, potentially
2325 producing lots of debug log messages from these libraries. Also, changing
2326 the config file is not always convenient (especially when running the
2327 GNUnet test suite).@ To fix that, and to allow GNUnet to use different
2328 log filtering at runtime without re-compiling the whole source tree, the
2329 log calls were changed to be configurable at run time. To configure them
2330 one has to define environment variables "GNUNET_FORCE_LOGFILE",
2331 "GNUNET_LOG" and/or "GNUNET_FORCE_LOG":
2334 @item "GNUNET_LOG" only affects the logging when no global log level is
2335 configured by any other means (that is, the process does not explicitly
2336 set its own log level, there are no "-L loglevel" options on command line
2337 or in configuration files), and can be used to override the default
2340 @item "GNUNET_FORCE_LOG" will completely override any other log
2341 configuration options given.
2343 @item "GNUNET_FORCE_LOGFILE" will completely override the location of the
2344 file to log messages to. It should contain a relative or absolute file
2345 name. Setting GNUNET_FORCE_LOGFILE is equivalent to passing
2346 "--log-file=logfile" or "-l logfile" option (see below). It supports "[]"
2347 format in file names, but not "@{@}" (see below).
2351 Because environment variables are inherited by child processes when they
2352 are launched, starting or re-starting the ARM service with these
2353 variables will propagate them to all other services.
2355 "GNUNET_LOG" and "GNUNET_FORCE_LOG" variables must contain a specially
2356 formatted @strong{logging definition} string, which looks like this:@
2358 @c FIXME: Can we close this with [/component] instead?
2360 [component];[file];[function];[from_line[-to_line]];loglevel[/component...]
2363 That is, a logging definition consists of definition entries, separated by
2364 slashes ('/'). If only one entry is present, there is no need to add a
2365 slash to its end (although it is not forbidden either).@ All definition
2366 fields (component, file, function, lines and loglevel) are mandatory, but
2367 (except for the loglevel) they can be empty. An empty field means
2368 "match anything". Note that even if fields are empty, the semicolon (';')
2369 separators must be present.@ The loglevel field is mandatory, and must
2370 contain one of the log level names (ERROR, WARNING, INFO or DEBUG).@
2371 The lines field might contain one non-negative number, in which case it
2372 matches only one line, or a range "from_line-to_line", in which case it
2373 matches any line in the interval [from_line;to_line] (that is, including
2374 both start and end line).@ GNUnet mostly defaults component name to the
2375 name of the service that is implemented in a process ('transport',
2376 'core', 'peerinfo', etc), but logging calls can specify custom component
2377 names using @code{GNUNET_log_from}.@ File name and function name are
2378 provided by the compiler (__FILE__ and __FUNCTION__ built-ins).
2380 Component, file and function fields are interpreted as non-extended
2381 regular expressions (GNU libc regex functions are used). Matching is
2382 case-sensitive, "^" and "$" will match the beginning and the end of the
2383 text. If a field is empty, its contents are automatically replaced with
2384 a ".*" regular expression, which matches anything. Matching is done in
2385 the default way, which means that the expression matches as long as it's
2386 contained anywhere in the string. Thus "GNUNET_" will match both
2387 "GNUNET_foo" and "BAR_GNUNET_BAZ". Use '^' and/or '$' to make sure that
2388 the expression matches at the start and/or at the end of the string.
2389 The semicolon (';') can't be escaped, and GNUnet will not use it in
2390 component names (it can't be used in function names and file names
2396 Every logging call in GNUnet code will be (at run time) matched against
2397 the log definitions passed to the process. If a log definition fields are
2398 matching the call arguments, then the call log level is compared the the
2399 log level of that definition. If the call log level is less or equal to
2400 the definition log level, the call is allowed to proceed. Otherwise the
2401 logging call is forbidden, and nothing is logged. If no definitions
2402 matched at all, GNUnet will use the global log level or (if a global log
2403 level is not specified) will default to WARNING (that is, it will allow
2404 the call to proceed, if its level is less or equal to the global log
2405 level or to WARNING).
2407 That is, definitions are evaluated from left to right, and the first
2408 matching definition is used to allow or deny the logging call. Thus it is
2409 advised to place narrow definitions at the beginning of the logdef
2410 string, and generic definitions - at the end.
2412 Whether a call is allowed or not is only decided the first time this
2413 particular call is made. The evaluation result is then cached, so that
2414 any attempts to make the same call later will be allowed or disallowed
2415 right away. Because of that runtime log level evaluation should not
2416 significantly affect the process performance.
2417 Log definition parsing is only done once, at the first call to
2418 @code{GNUNET_log_setup ()} made by the process (which is usually done soon after
2421 At the moment of writing there is no way to specify logging definitions
2422 from configuration files, only via environment variables.
2424 At the moment GNUnet will stop processing a log definition when it
2425 encounters an error in definition formatting or an error in regular
2426 expression syntax, and will not report the failure in any way.
2429 @c ***********************************************************************
2433 * Updated behavior of GNUNET_log::
2437 @subsubsection Examples
2441 @item @code{GNUNET_FORCE_LOG=";;;;DEBUG" gnunet-arm -s} Start GNUnet
2442 process tree, running all processes with DEBUG level (one should be
2443 careful with it, as log files will grow at alarming rate!)
2444 @item @code{GNUNET_FORCE_LOG="core;;;;DEBUG" gnunet-arm -s} Start GNUnet
2445 process tree, running the core service under DEBUG level (everything else
2446 will use configured or default level).
2448 @item Start GNUnet process tree, allowing any logging calls from
2449 gnunet-service-transport_validation.c (everything else will use
2450 configured or default level).
2453 GNUNET_FORCE_LOG=";gnunet-service-transport_validation.c;;; DEBUG" \
2457 @item Start GNUnet process tree, allowing any logging calls from
2458 gnunet-gnunet-service-fs_push.c (everything else will use configured or
2462 GNUNET_FORCE_LOG="fs;gnunet-service-fs_push.c;;;DEBUG" gnunet-arm -s
2465 @item Start GNUnet process tree, allowing any logging calls from the
2466 GNUNET_NETWORK_socket_select function (everything else will use
2467 configured or default level).
2470 GNUNET_FORCE_LOG=";;GNUNET_NETWORK_socket_select;;DEBUG" gnunet-arm -s
2473 @item Start GNUnet process tree, allowing any logging calls from the
2474 components that have "transport" in their names, and are made from
2475 function that have "send" in their names. Everything else will be allowed
2476 to be logged only if it has WARNING level.
2479 GNUNET_FORCE_LOG="transport.*;;.*send.*;;DEBUG/;;;;WARNING" gnunet-arm -s
2485 On Windows, one can use batch files to run GNUnet processes with special
2486 environment variables, without affecting the whole system. Such batch
2487 file will look like this:
2490 set GNUNET_FORCE_LOG=;;do_transmit;;DEBUG@ gnunet-arm -s
2493 (note the absence of double quotes in the environment variable definition,
2494 as opposed to earlier examples, which use the shell).
2495 Another limitation, on Windows, GNUNET_FORCE_LOGFILE @strong{MUST} be set
2496 in order to GNUNET_FORCE_LOG to work.
2501 @subsubsection Log files
2503 GNUnet can be told to log everything into a file instead of stderr (which
2504 is the default) using the "--log-file=logfile" or "-l logfile" option.
2505 This option can also be passed via command line, or from the "OPTION" and
2506 "GLOBAL_POSTFIX" configuration keys (see above). The file name passed
2507 with this option is subject to GNUnet filename expansion. If specified in
2508 "GLOBAL_POSTFIX", it is also subject to ARM service filename expansion,
2509 in particular, it may contain "@{@}" (left and right curly brace)
2510 sequence, which will be replaced by ARM with the name of the service.
2511 This is used to keep logs from more than one service separate, while only
2512 specifying one template containing "@{@}" in GLOBAL_POSTFIX.
2514 As part of a secondary file name expansion, the first occurrence of "[]"
2515 sequence ("left square brace" followed by "right square brace") in the
2516 file name will be replaced with a process identifier or the process when
2517 it initializes its logging subsystem. As a result, all processes will log
2518 into different files. This is convenient for isolating messages of a
2519 particular process, and prevents I/O races when multiple processes try to
2520 write into the file at the same time. This expansion is done
2521 independently of "@{@}" expansion that ARM service does (see above).
2523 The log file name that is specified via "-l" can contain format characters
2524 from the 'strftime' function family. For example, "%Y" will be replaced
2525 with the current year. Using "basename-%Y-%m-%d.log" would include the
2526 current year, month and day in the log file. If a GNUnet process runs for
2527 long enough to need more than one log file, it will eventually clean up
2528 old log files. Currently, only the last three log files (plus the current
2529 log file) are preserved. So once the fifth log file goes into use (so
2530 after 4 days if you use "%Y-%m-%d" as above), the first log file will be
2531 automatically deleted. Note that if your log file name only contains "%Y",
2532 then log files would be kept for 4 years and the logs from the first year
2533 would be deleted once year 5 begins. If you do not use any date-related
2534 string format codes, logs would never be automatically deleted by GNUnet.
2537 @c ***********************************************************************
2539 @node Updated behavior of GNUNET_log
2540 @subsubsection Updated behavior of GNUNET_log
2542 It's currently quite common to see constructions like this all over the
2547 GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, "MESH: client disconnected\n");
2551 The reason for the #if is not to avoid displaying the message when
2552 disabled (GNUNET_ERROR_TYPE takes care of that), but to avoid the
2553 compiler including it in the binary at all, when compiling GNUnet for
2554 platforms with restricted storage space / memory (MIPS routers,
2555 ARM plug computers / dev boards, etc).
2557 This presents several problems: the code gets ugly, hard to write and it
2558 is very easy to forget to include the #if guards, creating non-consistent
2559 code. A new change in GNUNET_log aims to solve these problems.
2561 @strong{This change requires to @file{./configure} with at least
2562 @code{--enable-logging=verbose} to see debug messages.}
2564 Here is an example of code with dense debug statements:
2567 switch (restrict_topology) @{
2568 case GNUNET_TESTING_TOPOLOGY_CLIQUE:#if VERBOSE_TESTING
2569 GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, _("Blacklisting all but clique
2570 topology\n")); #endif unblacklisted_connections = create_clique (pg,
2571 &remove_connections, BLACKLIST, GNUNET_NO); break; case
2572 GNUNET_TESTING_TOPOLOGY_SMALL_WORLD_RING: #if VERBOSE_TESTING GNUNET_log
2573 (GNUNET_ERROR_TYPE_DEBUG, _("Blacklisting all but small world (ring)
2574 topology\n")); #endif unblacklisted_connections = create_small_world_ring
2575 (pg,&remove_connections, BLACKLIST); break;
2579 Pretty hard to follow, huh?
2581 From now on, it is not necessary to include the #if / #endif statements to
2582 achieve the same behavior. The @code{GNUNET_log} and @code{GNUNET_log_from}
2584 care of it for you, depending on the configure option:
2587 @item If @code{--enable-logging} is set to @code{no}, the binary will
2588 contain no log messages at all.
2589 @item If @code{--enable-logging} is set to @code{yes}, the binary will
2590 contain no DEBUG messages, and therefore running with @command{-L DEBUG}
2592 no effect. Other messages (ERROR, WARNING, INFO, etc) will be included.
2593 @item If @code{--enable-logging} is set to @code{verbose}, or
2594 @code{veryverbose} the binary will contain DEBUG messages (still, it will
2595 be necessary to run with @command{-L DEBUG} or set the DEBUG config option
2601 If you are a developer:
2603 @item please make sure that you @code{./configure
2604 --enable-logging=@{verbose,veryverbose@}}, so you can see DEBUG messages.
2605 @item please remove the @code{#if} statements around @code{GNUNET_log
2606 (GNUNET_ERROR_TYPE_DEBUG, ...)} lines, to improve the readability of your
2610 Since now activating DEBUG automatically makes it VERBOSE and activates
2611 @strong{all} debug messages by default, you probably want to use the
2612 @uref{https://old.gnunet.org/logging, https://old.gnunet.org/logging}
2613 functionality to filter only relevant messages.
2614 A suitable configuration could be:
2617 $ export GNUNET_FORCE_LOG="^YOUR_SUBSYSTEM$;;;;DEBUG/;;;;WARNING"
2620 Which will behave almost like enabling DEBUG in that subsystem before the
2621 change. Of course you can adapt it to your particular needs, this is only
2624 @cindex Interprocess communication API
2626 @node Interprocess communication API (IPC)
2627 @subsection Interprocess communication API (IPC)
2629 In GNUnet a variety of new message types might be defined and used in
2630 interprocess communication, in this tutorial we use the
2631 @code{struct AddressLookupMessage} as a example to introduce how to
2632 construct our own message type in GNUnet and how to implement the message
2633 communication between service and client.
2634 (Here, a client uses the @code{struct AddressLookupMessage} as a request
2635 to ask the server to return the address of any other peer connecting to
2639 @c ***********************************************************************
2641 * Define new message types::
2642 * Define message struct::
2643 * Client - Establish connection::
2644 * Client - Initialize request message::
2645 * Client - Send request and receive response::
2646 * Server - Startup service::
2647 * Server - Add new handles for specified messages::
2648 * Server - Process request message::
2649 * Server - Response to client::
2650 * Server - Notification of clients::
2651 * Conversion between Network Byte Order (Big Endian) and Host Byte Order::
2654 @node Define new message types
2655 @subsubsection Define new message types
2657 First of all, you should define the new message type in
2658 @file{gnunet_protocols.h}:
2661 // Request to look addresses of peers in server.
2662 #define GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_LOOKUP 29
2663 // Response to the address lookup request.
2664 #define GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_REPLY 30
2667 @c ***********************************************************************
2668 @node Define message struct
2669 @subsubsection Define message struct
2671 After the type definition, the specified message structure should also be
2672 described in the header file, e.g. transport.h in our case.
2675 struct AddressLookupMessage @{
2676 struct GNUNET_MessageHeader header;
2677 int32_t numeric_only GNUNET_PACKED;
2678 struct GNUNET_TIME_AbsoluteNBO timeout;
2679 uint32_t addrlen GNUNET_PACKED;
2680 /* followed by 'addrlen' bytes of the actual address, then
2681 followed by the 0-terminated name of the transport */ @};
2682 GNUNET_NETWORK_STRUCT_END
2686 Please note @code{GNUNET_NETWORK_STRUCT_BEGIN} and @code{GNUNET_PACKED}
2687 which both ensure correct alignment when sending structs over the network.
2692 @c ***********************************************************************
2693 @node Client - Establish connection
2694 @subsubsection Client - Establish connection
2698 At first, on the client side, the underlying API is employed to create a
2699 new connection to a service, in our example the transport service would be
2703 struct GNUNET_CLIENT_Connection *client;
2704 client = GNUNET_CLIENT_connect ("transport", cfg);
2707 @c ***********************************************************************
2708 @node Client - Initialize request message
2709 @subsubsection Client - Initialize request message
2712 When the connection is ready, we initialize the message. In this step,
2713 all the fields of the message should be properly initialized, namely the
2714 size, type, and some extra user-defined data, such as timeout, name of
2715 transport, address and name of transport.
2718 struct AddressLookupMessage *msg;
2719 size_t len = sizeof (struct AddressLookupMessage)
2721 + strlen (nameTrans)
2723 msg->header->size = htons (len);
2724 msg->header->type = htons
2725 (GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_LOOKUP);
2726 msg->timeout = GNUNET_TIME_absolute_hton (abs_timeout);
2727 msg->addrlen = htonl (addressLen);
2728 char *addrbuf = (char *) &msg[1];
2729 memcpy (addrbuf, address, addressLen);
2730 char *tbuf = &addrbuf[addressLen];
2731 memcpy (tbuf, nameTrans, strlen (nameTrans) + 1);
2734 Note that, here the functions @code{htonl}, @code{htons} and
2735 @code{GNUNET_TIME_absolute_hton} are applied to convert little endian
2736 into big endian, about the usage of the big/small endian order and the
2737 corresponding conversion function please refer to Introduction of
2738 Big Endian and Little Endian.
2740 @c ***********************************************************************
2741 @node Client - Send request and receive response
2742 @subsubsection Client - Send request and receive response
2745 @b{FIXME: This is very outdated, see the tutorial for the current API!}
2747 Next, the client would send the constructed message as a request to the
2748 service and wait for the response from the service. To accomplish this
2749 goal, there are a number of API calls that can be used. In this example,
2750 @code{GNUNET_CLIENT_transmit_and_get_response} is chosen as the most
2751 appropriate function to use.
2754 GNUNET_CLIENT_transmit_and_get_response
2755 (client, msg->header, timeout, GNUNET_YES, &address_response_processor,
2759 the argument @code{address_response_processor} is a function with
2760 @code{GNUNET_CLIENT_MessageHandler} type, which is used to process the
2761 reply message from the service.
2763 @node Server - Startup service
2764 @subsubsection Server - Startup service
2766 After receiving the request message, we run a standard GNUnet service
2767 startup sequence using @code{GNUNET_SERVICE_run}, as follows,
2770 int main(int argc, char**argv) @{
2771 GNUNET_SERVICE_run(argc, argv, "transport"
2772 GNUNET_SERVICE_OPTION_NONE, &run, NULL)); @}
2775 @c ***********************************************************************
2776 @node Server - Add new handles for specified messages
2777 @subsubsection Server - Add new handles for specified messages
2780 in the function above the argument @code{run} is used to initiate
2781 transport service,and defined like this:
2784 static void run (void *cls,
2785 struct GNUNET_SERVER_Handle *serv,
2786 const struct GNUNET_CONFIGURATION_Handle *cfg) @{
2787 GNUNET_SERVER_add_handlers (serv, handlers); @}
2791 Here, @code{GNUNET_SERVER_add_handlers} must be called in the run
2792 function to add new handlers in the service. The parameter
2793 @code{handlers} is a list of @code{struct GNUNET_SERVER_MessageHandler}
2794 to tell the service which function should be called when a particular
2795 type of message is received, and should be defined in this way:
2798 static struct GNUNET_SERVER_MessageHandler handlers[] = @{
2801 GNUNET_MESSAGE_TYPE_TRANSPORT_START,
2805 GNUNET_MESSAGE_TYPE_TRANSPORT_SEND,
2807 @{&handle_try_connect,
2809 GNUNET_MESSAGE_TYPE_TRANSPORT_TRY_CONNECT,
2810 sizeof (struct TryConnectMessage)
2812 @{&handle_address_lookup,
2814 GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_LOOKUP,
2824 As shown, the first member of the struct in the first area is a callback
2825 function, which is called to process the specified message types, given
2826 as the third member. The second parameter is the closure for the callback
2827 function, which is set to @code{NULL} in most cases, and the last
2828 parameter is the expected size of the message of this type, usually we
2829 set it to 0 to accept variable size, for special cases the exact size of
2830 the specified message also can be set. In addition, the terminator sign
2831 depicted as @code{@{NULL, NULL, 0, 0@}} is set in the last area.
2833 @c ***********************************************************************
2834 @node Server - Process request message
2835 @subsubsection Server - Process request message
2838 After the initialization of transport service, the request message would
2839 be processed. Before handling the main message data, the validity of this
2840 message should be checked out, e.g., to check whether the size of message
2844 size = ntohs (message->size);
2845 if (size < sizeof (struct AddressLookupMessage)) @{
2846 GNUNET_break_op (0);
2847 GNUNET_SERVER_receive_done (client, GNUNET_SYSERR);
2852 Note that, opposite to the construction method of the request message in
2853 the client, in the server the function @code{nothl} and @code{ntohs}
2854 should be employed during the extraction of the data from the message, so
2855 that the data in big endian order can be converted back into little
2856 endian order. See more in detail please refer to Introduction of
2857 Big Endian and Little Endian.
2859 Moreover in this example, the name of the transport stored in the message
2860 is a 0-terminated string, so we should also check whether the name of the
2861 transport in the received message is 0-terminated:
2864 nameTransport = (const char *) &address[addressLen];
2865 if (nameTransport[size - sizeof
2866 (struct AddressLookupMessage)
2867 - addressLen - 1] != '\0') @{
2868 GNUNET_break_op (0);
2869 GNUNET_SERVER_receive_done (client,
2874 Here, @code{GNUNET_SERVER_receive_done} should be called to tell the
2875 service that the request is done and can receive the next message. The
2876 argument @code{GNUNET_SYSERR} here indicates that the service didn't
2877 understand the request message, and the processing of this request would
2880 In comparison to the aforementioned situation, when the argument is equal
2881 to @code{GNUNET_OK}, the service would continue to process the request
2884 @c ***********************************************************************
2885 @node Server - Response to client
2886 @subsubsection Server - Response to client
2889 Once the processing of current request is done, the server should give the
2890 response to the client. A new @code{struct AddressLookupMessage} would be
2891 produced by the server in a similar way as the client did and sent to the
2892 client, but here the type should be
2893 @code{GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_REPLY} rather than
2894 @code{GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_LOOKUP} in client.
2896 struct AddressLookupMessage *msg;
2897 size_t len = sizeof (struct AddressLookupMessage)
2899 + strlen (nameTrans) + 1;
2900 msg->header->size = htons (len);
2901 msg->header->type = htons
2902 (GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_REPLY);
2906 struct GNUNET_SERVER_TransmitContext *tc;
2907 tc = GNUNET_SERVER_transmit_context_create (client);
2908 GNUNET_SERVER_transmit_context_append_data
2912 GNUNET_MESSAGE_TYPE_TRANSPORT_ADDRESS_REPLY);
2913 GNUNET_SERVER_transmit_context_run (tc, rtimeout);
2917 Note that, there are also a number of other APIs provided to the service
2918 to send the message.
2920 @c ***********************************************************************
2921 @node Server - Notification of clients
2922 @subsubsection Server - Notification of clients
2925 Often a service needs to (repeatedly) transmit notifications to a client
2926 or a group of clients. In these cases, the client typically has once
2927 registered for a set of events and then needs to receive a message
2928 whenever such an event happens (until the client disconnects). The use of
2929 a notification context can help manage message queues to clients and
2930 handle disconnects. Notification contexts can be used to send
2931 individualized messages to a particular client or to broadcast messages
2932 to a group of clients. An individualized notification might look like
2936 GNUNET_SERVER_notification_context_unicast(nc,
2943 Note that after processing the original registration message for
2944 notifications, the server code still typically needs to call
2945 @code{GNUNET_SERVER_receive_done} so that the client can transmit further
2946 messages to the server.
2948 @c ***********************************************************************
2949 @node Conversion between Network Byte Order (Big Endian) and Host Byte Order
2950 @subsubsection Conversion between Network Byte Order (Big Endian) and Host Byte Order
2951 @c %** subsub? it's a referenced page on the ipc document.
2954 Here we can simply comprehend big endian and little endian as Network Byte
2955 Order and Host Byte Order respectively. What is the difference between
2958 Usually in our host computer we store the data byte as Host Byte Order,
2959 for example, we store a integer in the RAM which might occupies 4 Byte,
2960 as Host Byte Order the higher Byte would be stored at the lower address
2961 of RAM, and the lower Byte would be stored at the higher address of RAM.
2962 However, contrast to this, Network Byte Order just take the totally
2963 opposite way to store the data, says, it will store the lower Byte at the
2964 lower address, and the higher Byte will stay at higher address.
2966 For the current communication of network, we normally exchange the
2967 information by surveying the data package, every two host wants to
2968 communicate with each other must send and receive data package through
2969 network. In order to maintain the identity of data through the
2970 transmission in the network, the order of the Byte storage must changed
2971 before sending and after receiving the data.
2973 There ten convenient functions to realize the conversion of Byte Order in
2974 GNUnet, as following:
2978 @item uint16_t htons(uint16_t hostshort) Convert host byte order to net
2979 byte order with short int
2980 @item uint32_t htonl(uint32_t hostlong) Convert host byte
2981 order to net byte order with long int
2982 @item uint16_t ntohs(uint16_t netshort)
2983 Convert net byte order to host byte order with short int
2985 ntohl(uint32_t netlong) Convert net byte order to host byte order with
2987 @item unsigned long long GNUNET_ntohll (unsigned long long netlonglong)
2988 Convert net byte order to host byte order with long long int
2989 @item unsigned long long GNUNET_htonll (unsigned long long hostlonglong)
2990 Convert host byte order to net byte order with long long int
2991 @item struct GNUNET_TIME_RelativeNBO GNUNET_TIME_relative_hton
2992 (struct GNUNET_TIME_Relative a) Convert relative time to network byte
2994 @item struct GNUNET_TIME_Relative GNUNET_TIME_relative_ntoh
2995 (struct GNUNET_TIME_RelativeNBO a) Convert relative time from network
2997 @item struct GNUNET_TIME_AbsoluteNBO GNUNET_TIME_absolute_hton
2998 (struct GNUNET_TIME_Absolute a) Convert relative time to network byte
3000 @item struct GNUNET_TIME_Absolute GNUNET_TIME_absolute_ntoh
3001 (struct GNUNET_TIME_AbsoluteNBO a) Convert relative time from network
3005 @cindex Cryptography API
3006 @node Cryptography API
3007 @subsection Cryptography API
3010 The gnunetutil APIs provides the cryptographic primitives used in GNUnet.
3011 GNUnet uses 2048 bit RSA keys for the session key exchange and for signing
3012 messages by peers and most other public-key operations. Most researchers
3013 in cryptography consider 2048 bit RSA keys as secure and practically
3014 unbreakable for a long time. The API provides functions to create a fresh
3015 key pair, read a private key from a file (or create a new file if the
3016 file does not exist), encrypt, decrypt, sign, verify and extraction of
3017 the public key into a format suitable for network transmission.
3019 For the encryption of files and the actual data exchanged between peers
3020 GNUnet uses 256-bit AES encryption. Fresh, session keys are negotiated
3021 for every new connection.@ Again, there is no published technique to
3022 break this cipher in any realistic amount of time. The API provides
3023 functions for generation of keys, validation of keys (important for
3024 checking that decryptions using RSA succeeded), encryption and decryption.
3026 GNUnet uses SHA-512 for computing one-way hash codes. The API provides
3027 functions to compute a hash over a block in memory or over a file on disk.
3029 The crypto API also provides functions for randomizing a block of memory,
3030 obtaining a single random number and for generating a permutation of the
3031 numbers 0 to n-1. Random number generation distinguishes between WEAK and
3032 STRONG random number quality; WEAK random numbers are pseudo-random
3033 whereas STRONG random numbers use entropy gathered from the operating
3036 Finally, the crypto API provides a means to deterministically generate a
3037 1024-bit RSA key from a hash code. These functions should most likely not
3038 be used by most applications; most importantly,
3039 GNUNET_CRYPTO_rsa_key_create_from_hash does not create an RSA-key that
3040 should be considered secure for traditional applications of RSA.
3042 @cindex Message Queue API
3043 @node Message Queue API
3044 @subsection Message Queue API
3047 @strong{ Introduction }@
3048 Often, applications need to queue messages that
3049 are to be sent to other GNUnet peers, clients or services. As all of
3050 GNUnet's message-based communication APIs, by design, do not allow
3051 messages to be queued, it is common to implement custom message queues
3052 manually when they are needed. However, writing very similar code in
3053 multiple places is tedious and leads to code duplication.
3055 MQ (for Message Queue) is an API that provides the functionality to
3056 implement and use message queues. We intend to eventually replace all of
3057 the custom message queue implementations in GNUnet with MQ.
3059 @strong{ Basic Concepts }@
3060 The two most important entities in MQ are queues and envelopes.
3062 Every queue is backed by a specific implementation (e.g. for mesh, stream,
3063 connection, server client, etc.) that will actually deliver the queued
3064 messages. For convenience,@ some queues also allow to specify a list of
3065 message handlers. The message queue will then also wait for incoming
3066 messages and dispatch them appropriately.
3068 An envelope holds the the memory for a message, as well as metadata
3069 (Where is the envelope queued? What should happen after it has been
3070 sent?). Any envelope can only be queued in one message queue.
3072 @strong{ Creating Queues }@
3073 The following is a list of currently available message queues. Note that
3074 to avoid layering issues, message queues for higher level APIs are not
3075 part of @code{libgnunetutil}, but@ the respective API itself provides the
3076 queue implementation.
3080 @item @code{GNUNET_MQ_queue_for_connection_client}
3081 Transmits queued messages over a @code{GNUNET_CLIENT_Connection} handle.
3082 Also supports receiving with message handlers.
3084 @item @code{GNUNET_MQ_queue_for_server_client}
3085 Transmits queued messages over a @code{GNUNET_SERVER_Client} handle. Does
3086 not support incoming message handlers.
3088 @item @code{GNUNET_MESH_mq_create} Transmits queued messages over a
3089 @code{GNUNET_MESH_Tunnel} handle. Does not support incoming message
3092 @item @code{GNUNET_MQ_queue_for_callbacks} This is the most general
3093 implementation. Instead of delivering and receiving messages with one of
3094 GNUnet's communication APIs, implementation callbacks are called. Refer to
3095 "Implementing Queues" for a more detailed explanation.
3099 @strong{ Allocating Envelopes }@
3100 A GNUnet message (as defined by the GNUNET_MessageHeader) has three
3101 parts: The size, the type, and the body.
3103 MQ provides macros to allocate an envelope containing a message
3104 conveniently, automatically setting the size and type fields of the
3107 Consider the following simple message, with the body consisting of a
3108 single number value.
3109 @c why the empty code function?
3113 struct NumberMessage @{
3114 /** Type: GNUNET_MESSAGE_TYPE_EXAMPLE_1 */
3115 struct GNUNET_MessageHeader header;
3116 uint32_t number GNUNET_PACKED;
3120 An envelope containing an instance of the NumberMessage can be
3121 constructed like this:
3124 struct GNUNET_MQ_Envelope *ev;
3125 struct NumberMessage *msg;
3126 ev = GNUNET_MQ_msg (msg, GNUNET_MESSAGE_TYPE_EXAMPLE_1);
3127 msg->number = htonl (42);
3130 In the above code, @code{GNUNET_MQ_msg} is a macro. The return value is
3131 the newly allocated envelope. The first argument must be a pointer to some
3132 @code{struct} containing a @code{struct GNUNET_MessageHeader header}
3133 field, while the second argument is the desired message type, in host
3136 The @code{msg} pointer now points to an allocated message, where the
3137 message type and the message size are already set. The message's size is
3138 inferred from the type of the @code{msg} pointer: It will be set to
3139 'sizeof(*msg)', properly converted to network byte order.
3141 If the message body's size is dynamic, the the macro
3142 @code{GNUNET_MQ_msg_extra} can be used to allocate an envelope whose
3143 message has additional space allocated after the @code{msg} structure.
3145 If no structure has been defined for the message,
3146 @code{GNUNET_MQ_msg_header_extra} can be used to allocate additional space
3147 after the message header. The first argument then must be a pointer to a
3148 @code{GNUNET_MessageHeader}.
3150 @strong{Envelope Properties}@
3151 A few functions in MQ allow to set additional properties on envelopes:
3155 @item @code{GNUNET_MQ_notify_sent} Allows to specify a function that will
3156 be called once the envelope's message has been sent irrevocably.
3157 An envelope can be canceled precisely up to the@ point where the notify
3158 sent callback has been called.
3160 @item @code{GNUNET_MQ_disable_corking} No corking will be used when
3161 sending the message. Not every@ queue supports this flag, per default,
3162 envelopes are sent with corking.@
3167 @strong{Sending Envelopes}@
3168 Once an envelope has been constructed, it can be queued for sending with
3169 @code{GNUNET_MQ_send}.
3171 Note that in order to avoid memory leaks, an envelope must either be sent
3172 (the queue will free it) or destroyed explicitly with
3173 @code{GNUNET_MQ_discard}.
3175 @strong{Canceling Envelopes}@
3176 An envelope queued with @code{GNUNET_MQ_send} can be canceled with
3177 @code{GNUNET_MQ_cancel}. Note that after the notify sent callback has
3178 been called, canceling a message results in undefined behavior.
3179 Thus it is unsafe to cancel an envelope that does not have a notify sent
3180 callback. When canceling an envelope, it is not necessary@ to call
3181 @code{GNUNET_MQ_discard}, and the envelope can't be sent again.
3183 @strong{ Implementing Queues }@
3188 @subsection Service API
3191 Most GNUnet code lives in the form of services. Services are processes
3192 that offer an API for other components of the system to build on. Those
3193 other components can be command-line tools for users, graphical user
3194 interfaces or other services. Services provide their API using an IPC
3195 protocol. For this, each service must listen on either a TCP port or a
3196 UNIX domain socket; for this, the service implementation uses the server
3197 API. This use of server is exposed directly to the users of the service
3198 API. Thus, when using the service API, one is usually also often using
3199 large parts of the server API. The service API provides various
3200 convenience functions, such as parsing command-line arguments and the
3201 configuration file, which are not found in the server API.
3202 The dual to the service/server API is the client API, which can be used to
3205 The most common way to start a service is to use the
3206 @code{GNUNET_SERVICE_run} function from the program's main function.
3207 @code{GNUNET_SERVICE_run} will then parse the command line and
3208 configuration files and, based on the options found there,
3209 start the server. It will then give back control to the main
3210 program, passing the server and the configuration to the
3211 @code{GNUNET_SERVICE_Main} callback. @code{GNUNET_SERVICE_run}
3212 will also take care of starting the scheduler loop.
3213 If this is inappropriate (for example, because the scheduler loop
3214 is already running), @code{GNUNET_SERVICE_start} and
3215 related functions provide an alternative to @code{GNUNET_SERVICE_run}.
3217 When starting a service, the service_name option is used to determine
3218 which sections in the configuration file should be used to configure the
3219 service. A typical value here is the name of the @file{src/}
3220 sub-directory, for example @file{statistics}.
3221 The same string would also be given to
3222 @code{GNUNET_CLIENT_connect} to access the service.
3224 Once a service has been initialized, the program should use the
3225 @code{GNUNET_SERVICE_Main} callback to register message handlers
3226 using @code{GNUNET_SERVER_add_handlers}.
3227 The service will already have registered a handler for the
3230 @findex GNUNET_SERVICE_Options
3231 The option bitfield (@code{enum GNUNET_SERVICE_Options})
3232 determines how a service should behave during shutdown.
3233 There are three key strategies:
3237 @item instant (@code{GNUNET_SERVICE_OPTION_NONE})
3238 Upon receiving the shutdown
3239 signal from the scheduler, the service immediately terminates the server,
3240 closing all existing connections with clients.
3241 @item manual (@code{GNUNET_SERVICE_OPTION_MANUAL_SHUTDOWN})
3242 The service does nothing by itself
3243 during shutdown. The main program will need to take the appropriate
3244 action by calling GNUNET_SERVER_destroy or GNUNET_SERVICE_stop (depending
3245 on how the service was initialized) to terminate the service. This method
3246 is used by gnunet-service-arm and rather uncommon.
3247 @item soft (@code{GNUNET_SERVICE_OPTION_SOFT_SHUTDOWN})
3248 Upon receiving the shutdown signal from the scheduler,
3249 the service immediately tells the server to stop
3250 listening for incoming clients. Requests from normal existing clients are
3251 still processed and the server/service terminates once all normal clients
3252 have disconnected. Clients that are not expected to ever disconnect (such
3253 as clients that monitor performance values) can be marked as 'monitor'
3254 clients using GNUNET_SERVER_client_mark_monitor. Those clients will
3255 continue to be processed until all 'normal' clients have disconnected.
3256 Then, the server will terminate, closing the monitor connections.
3257 This mode is for example used by 'statistics', allowing existing 'normal'
3258 clients to set (possibly persistent) statistic values before terminating.
3262 @c ***********************************************************************
3263 @node Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps
3264 @subsection Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps
3267 A commonly used data structure in GNUnet is a (multi-)hash map. It is most
3268 often used to map a peer identity to some data structure, but also to map
3269 arbitrary keys to values (for example to track requests in the distributed
3270 hash table or in file-sharing). As it is commonly used, the DHT is
3271 actually sometimes responsible for a large share of GNUnet's overall
3272 memory consumption (for some processes, 30% is not uncommon). The
3273 following text documents some API quirks (and their implications for
3274 applications) that were recently introduced to minimize the footprint of
3278 @c ***********************************************************************
3288 @subsubsection Analysis
3291 The main reason for the "excessive" memory consumption by the hash map is
3292 that GNUnet uses 512-bit cryptographic hash codes --- and the
3293 (multi-)hash map also uses the same 512-bit 'struct GNUNET_HashCode'. As
3294 a result, storing just the keys requires 64 bytes of memory for each key.
3295 As some applications like to keep a large number of entries in the hash
3296 map (after all, that's what maps are good for), 64 bytes per hash is
3297 significant: keeping a pointer to the value and having a linked list for
3298 collisions consume between 8 and 16 bytes, and 'malloc' may add about the
3299 same overhead per allocation, putting us in the 16 to 32 byte per entry
3300 ballpark. Adding a 64-byte key then triples the overall memory
3301 requirement for the hash map.
3303 To make things "worse", most of the time storing the key in the hash map
3304 is not required: it is typically already in memory elsewhere! In most
3305 cases, the values stored in the hash map are some application-specific
3306 struct that _also_ contains the hash. Here is a simplified example:
3310 struct GNUNET_HashCode key;
3311 unsigned int my_data; @};
3314 val = GNUNET_malloc (sizeof (struct MyValue));
3317 GNUNET_CONTAINER_multihashmap_put (map, &key, val, ...);
3320 This is a common pattern as later the entries might need to be removed,
3321 and at that time it is convenient to have the key immediately at hand:
3324 GNUNET_CONTAINER_multihashmap_remove (map, &val->key, val);
3328 Note that here we end up with two times 64 bytes for the key, plus maybe
3329 64 bytes total for the rest of the 'struct MyValue' and the map entry in
3330 the hash map. The resulting redundant storage of the key increases
3331 overall memory consumption per entry from the "optimal" 128 bytes to 192
3332 bytes. This is not just an extreme example: overheads in practice are
3333 actually sometimes close to those highlighted in this example. This is
3334 especially true for maps with a significant number of entries, as there
3335 we tend to really try to keep the entries small.
3337 @c ***********************************************************************
3339 @subsubsection Solution
3342 The solution that has now been implemented is to @strong{optionally}
3343 allow the hash map to not make a (deep) copy of the hash but instead have
3344 a pointer to the hash/key in the entry. This reduces the memory
3345 consumption for the key from 64 bytes to 4 to 8 bytes. However, it can
3346 also only work if the key is actually stored in the entry (which is the
3347 case most of the time) and if the entry does not modify the key (which in
3348 all of the code I'm aware of has been always the case if there key is
3349 stored in the entry). Finally, when the client stores an entry in the
3350 hash map, it @strong{must} provide a pointer to the key within the entry,
3351 not just a pointer to a transient location of the key. If
3352 the client code does not meet these requirements, the result is a dangling
3353 pointer and undefined behavior of the (multi-)hash map API.
3355 @c ***********************************************************************
3357 @subsubsection Migration
3360 To use the new feature, first check that the values contain the respective
3361 key (and never modify it). Then, all calls to
3362 @code{GNUNET_CONTAINER_multihashmap_put} on the respective map must be
3363 audited and most likely changed to pass a pointer into the value's struct.
3364 For the initial example, the new code would look like this:
3368 struct GNUNET_HashCode key;
3369 unsigned int my_data; @};
3372 val = GNUNET_malloc (sizeof (struct MyValue));
3373 val->key = key; val->my_data = 42;
3374 GNUNET_CONTAINER_multihashmap_put (map, &val->key, val, ...);
3378 Note that @code{&val} was changed to @code{&val->key} in the argument to
3379 the @code{put} call. This is critical as often @code{key} is on the stack
3380 or in some other transient data structure and thus having the hash map
3381 keep a pointer to @code{key} would not work. Only the key inside of
3382 @code{val} has the same lifetime as the entry in the map (this must of
3383 course be checked as well). Naturally, @code{val->key} must be
3384 initialized before the @code{put} call. Once all @code{put} calls have
3385 been converted and double-checked, you can change the call to create the
3390 GNUNET_CONTAINER_multihashmap_create (SIZE, GNUNET_NO);
3396 map = GNUNET_CONTAINER_multihashmap_create (SIZE, GNUNET_YES);
3399 If everything was done correctly, you now use about 60 bytes less memory
3400 per entry in @code{map}. However, if now (or in the future) any call to
3401 @code{put} does not ensure that the given key is valid until the entry is
3402 removed from the map, undefined behavior is likely to be observed.
3404 @c ***********************************************************************
3406 @subsubsection Conclusion
3409 The new optimization can is often applicable and can result in a
3410 reduction in memory consumption of up to 30% in practice. However, it
3411 makes the code less robust as additional invariants are imposed on the
3412 multi hash map client. Thus applications should refrain from enabling the
3413 new mode unless the resulting performance increase is deemed significant
3414 enough. In particular, it should generally not be used in new code (wait
3415 at least until benchmarks exist).
3417 @c ***********************************************************************
3419 @subsubsection Availability
3422 The new multi hash map code was committed in SVN 24319 (which made its
3423 way into GNUnet version 0.9.4).
3424 Various subsystems (transport, core, dht, file-sharing) were
3425 previously audited and modified to take advantage of the new capability.
3426 In particular, memory consumption of the file-sharing service is expected
3427 to drop by 20-30% due to this change.
3430 @cindex CONTAINER_MDLL API
3431 @node CONTAINER_MDLL API
3432 @subsection CONTAINER_MDLL API
3435 This text documents the GNUNET_CONTAINER_MDLL API. The
3436 GNUNET_CONTAINER_MDLL API is similar to the GNUNET_CONTAINER_DLL API in
3437 that it provides operations for the construction and manipulation of
3438 doubly-linked lists. The key difference to the (simpler) DLL-API is that
3439 the MDLL-version allows a single element (instance of a "struct") to be
3440 in multiple linked lists at the same time.
3442 Like the DLL API, the MDLL API stores (most of) the data structures for
3443 the doubly-linked list with the respective elements; only the 'head' and
3444 'tail' pointers are stored "elsewhere" --- and the application needs to
3445 provide the locations of head and tail to each of the calls in the
3446 MDLL API. The key difference for the MDLL API is that the "next" and
3447 "previous" pointers in the struct can no longer be simply called "next"
3448 and "prev" --- after all, the element may be in multiple doubly-linked
3449 lists, so we cannot just have one "next" and one "prev" pointer!
3451 The solution is to have multiple fields that must have a name of the
3452 format "next_XX" and "prev_XX" where "XX" is the name of one of the
3453 doubly-linked lists. Here is a simple example:
3456 struct MyMultiListElement @{
3457 struct MyMultiListElement *next_ALIST;
3458 struct MyMultiListElement *prev_ALIST;
3459 struct MyMultiListElement *next_BLIST;
3460 struct MyMultiListElement *prev_BLIST;
3467 Note that by convention, we use all-uppercase letters for the list names.
3468 In addition, the program needs to have a location for the head and tail
3469 pointers for both lists, for example:
3472 static struct MyMultiListElement *head_ALIST;
3473 static struct MyMultiListElement *tail_ALIST;
3474 static struct MyMultiListElement *head_BLIST;
3475 static struct MyMultiListElement *tail_BLIST;
3479 Using the MDLL-macros, we can now insert an element into the ALIST:
3482 GNUNET_CONTAINER_MDLL_insert (ALIST, head_ALIST, tail_ALIST, element);
3486 Passing "ALIST" as the first argument to MDLL specifies which of the
3487 next/prev fields in the 'struct MyMultiListElement' should be used. The
3488 extra "ALIST" argument and the "_ALIST" in the names of the
3489 next/prev-members are the only differences between the MDDL and DLL-API.
3490 Like the DLL-API, the MDLL-API offers functions for inserting (at head,
3491 at tail, after a given element) and removing elements from the list.
3492 Iterating over the list should be done by directly accessing the
3493 "next_XX" and/or "prev_XX" members.
3495 @cindex Automatic Restart Manager
3497 @node Automatic Restart Manager (ARM)
3498 @section Automatic Restart Manager (ARM)
3501 GNUnet's Automated Restart Manager (ARM) is the GNUnet service responsible
3502 for system initialization and service babysitting. ARM starts and halts
3503 services, detects configuration changes and restarts services impacted by
3504 the changes as needed. It's also responsible for restarting services in
3505 case of crashes and is planned to incorporate automatic debugging for
3506 diagnosing service crashes providing developers insights about crash
3507 reasons. The purpose of this document is to give GNUnet developer an idea
3508 about how ARM works and how to interact with it.
3511 * Basic functionality::
3512 * Key configuration options::
3513 * ARM - Availability::
3517 @c ***********************************************************************
3518 @node Basic functionality
3519 @subsection Basic functionality
3523 @item ARM source code can be found under "src/arm".@ Service processes are
3524 managed by the functions in "gnunet-service-arm.c" which is controlled
3525 with "gnunet-arm.c" (main function in that file is ARM's entry point).
3527 @item The functions responsible for communicating with ARM , starting and
3528 stopping services -including ARM service itself- are provided by the
3529 ARM API "arm_api.c".@ Function: GNUNET_ARM_connect() returns to the caller
3530 an ARM handle after setting it to the caller's context (configuration and
3531 scheduler in use). This handle can be used afterwards by the caller to
3532 communicate with ARM. Functions GNUNET_ARM_start_service() and
3533 GNUNET_ARM_stop_service() are used for starting and stopping services
3536 @item A typical example of using these basic ARM services can be found in
3537 file test_arm_api.c. The test case connects to ARM, starts it, then uses
3538 it to start a service "resolver", stops the "resolver" then stops "ARM".
3541 @c ***********************************************************************
3542 @node Key configuration options
3543 @subsection Key configuration options
3546 Configurations for ARM and services should be available in a .conf file
3547 (As an example, see test_arm_api_data.conf). When running ARM, the
3548 configuration file to use should be passed to the command:
3551 $ gnunet-arm -s -c configuration_to_use.conf
3554 If no configuration is passed, the default configuration file will be used
3555 (see GNUNET_PREFIX/share/gnunet/defaults.conf which is created from
3556 contrib/defaults.conf).@ Each of the services is having a section starting
3557 by the service name between square brackets, for example: "[arm]".
3558 The following options configure how ARM configures or interacts with the
3563 @item PORT Port number on which the service is listening for incoming TCP
3564 connections. ARM will start the services should it notice a request at
3567 @item HOSTNAME Specifies on which host the service is deployed. Note
3568 that ARM can only start services that are running on the local system
3569 (but will not check that the hostname matches the local machine name).
3570 This option is used by the @code{gnunet_client_lib.h} implementation to
3571 determine which system to connect to. The default is "localhost".
3573 @item BINARY The name of the service binary file.
3575 @item OPTIONS To be passed to the service.
3577 @item PREFIX A command to pre-pend to the actual command, for example,
3578 running a service with "valgrind" or "gdb"
3580 @item DEBUG Run in debug mode (much verbosity).
3582 @item START_ON_DEMAND ARM will listen to UNIX domain socket and/or TCP port of
3583 the service and start the service on-demand.
3585 @item IMMEDIATE_START ARM will always start this service when the peer
3588 @item ACCEPT_FROM IPv4 addresses the service accepts connections from.
3590 @item ACCEPT_FROM6 IPv6 addresses the service accepts connections from.
3595 Options that impact the operation of ARM overall are in the "[arm]"
3596 section. ARM is a normal service and has (except for START_ON_DEMAND) all of the
3597 options that other services do. In addition, ARM has the
3602 @item GLOBAL_PREFIX Command to be pre-pended to all services that are
3605 @item GLOBAL_POSTFIX Global option that will be supplied to all the
3606 services that are going to run.
3610 @c ***********************************************************************
3611 @node ARM - Availability
3612 @subsection ARM - Availability
3615 As mentioned before, one of the features provided by ARM is starting
3616 services on demand. Consider the example of one service "client" that
3617 wants to connect to another service a "server". The "client" will ask ARM
3618 to run the "server". ARM starts the "server". The "server" starts
3619 listening to incoming connections. The "client" will establish a
3620 connection with the "server". And then, they will start to communicate
3621 together.@ One problem with that scheme is that it's slow!@
3622 The "client" service wants to communicate with the "server" service at
3623 once and is not willing wait for it to be started and listening to
3624 incoming connections before serving its request.@ One solution for that
3625 problem will be that ARM starts all services as default services. That
3626 solution will solve the problem, yet, it's not quite practical, for some
3627 services that are going to be started can never be used or are going to
3628 be used after a relatively long time.@
3629 The approach followed by ARM to solve this problem is as follows:
3633 @item For each service having a PORT field in the configuration file and
3634 that is not one of the default services ( a service that accepts incoming
3635 connections from clients), ARM creates listening sockets for all addresses
3636 associated with that service.
3638 @item The "client" will immediately establish a connection with
3641 @item ARM --- pretending to be the "server" --- will listen on the
3642 respective port and notice the incoming connection from the "client"
3643 (but not accept it), instead
3645 @item Once there is an incoming connection, ARM will start the "server",
3646 passing on the listen sockets (now, the service is started and can do its
3649 @item Other client services now can directly connect directly to the
3654 @c ***********************************************************************
3656 @subsection Reliability
3658 One of the features provided by ARM, is the automatic restart of crashed
3659 services.@ ARM needs to know which of the running services died. Function
3660 "gnunet-service-arm.c/maint_child_death()" is responsible for that. The
3661 function is scheduled to run upon receiving a SIGCHLD signal. The
3662 function, then, iterates ARM's list of services running and monitors
3663 which service has died (crashed). For all crashing services, ARM restarts
3665 Now, considering the case of a service having a serious problem causing it
3666 to crash each time it's started by ARM. If ARM keeps blindly restarting
3667 such a service, we are going to have the pattern:
3668 start-crash-restart-crash-restart-crash and so forth!! Which is of course
3670 For that reason, ARM schedules the service to be restarted after waiting
3671 for some delay that grows exponentially with each crash/restart of that
3672 service.@ To clarify the idea, considering the following example:
3676 @item Service S crashed.
3678 @item ARM receives the SIGCHLD and inspects its list of services to find
3681 @item ARM finds S dead and schedules it for restarting after "backoff"
3682 time which is initially set to 1ms. ARM will double the backoff time
3683 correspondent to S (now backoff(S) = 2ms)
3685 @item Because there is a severe problem with S, it crashed again.
3687 @item Again ARM receives the SIGCHLD and detects that it's S again that's
3688 crashed. ARM schedules it for restarting but after its new backoff time
3689 (which became 2ms), and doubles its backoff time (now backoff(S) = 4).
3691 @item and so on, until backoff(S) reaches a certain threshold
3692 (@code{EXPONENTIAL_BACKOFF_THRESHOLD} is set to half an hour),
3693 after reaching it, backoff(S) will remain half an hour,
3694 hence ARM won't be busy for a lot of time trying to restart a
3695 problematic service.
3698 @cindex TRANSPORT Subsystem
3699 @node TRANSPORT Subsystem
3700 @section TRANSPORT Subsystem
3703 This chapter documents how the GNUnet transport subsystem works. The
3704 GNUnet transport subsystem consists of three main components: the
3705 transport API (the interface used by the rest of the system to access the
3706 transport service), the transport service itself (most of the interesting
3707 functions, such as choosing transports, happens here) and the transport
3708 plugins. A transport plugin is a concrete implementation for how two
3709 GNUnet peers communicate; many plugins exist, for example for
3710 communication via TCP, UDP, HTTP, HTTPS and others. Finally, the
3711 transport subsystem uses supporting code, especially the NAT/UPnP
3712 library to help with tasks such as NAT traversal.
3714 Key tasks of the transport service include:
3718 @item Create our HELLO message, notify clients and neighbours if our HELLO
3719 changes (using NAT library as necessary)
3721 @item Validate HELLOs from other peers (send PING), allow other peers to
3722 validate our HELLO's addresses (send PONG)
3724 @item Upon request, establish connections to other peers (using address
3725 selection from ATS subsystem) and maintain them (again using PINGs and
3726 PONGs) as long as desired
3728 @item Accept incoming connections, give ATS service the opportunity to
3729 switch communication channels
3731 @item Notify clients about peers that have connected to us or that have
3732 been disconnected from us
3734 @item If a (stateful) connection goes down unexpectedly (without explicit
3735 DISCONNECT), quickly attempt to recover (without notifying clients) but do
3736 notify clients quickly if reconnecting fails
3738 @item Send (payload) messages arriving from clients to other peers via
3739 transport plugins and receive messages from other peers, forwarding
3742 @item Enforce inbound traffic limits (using flow-control if it is
3743 applicable); outbound traffic limits are enforced by CORE, not by us (!)
3745 @item Enforce restrictions on P2P connection as specified by the blacklist
3746 configuration and blacklisting clients
3749 Note that the term "clients" in the list above really refers to the
3750 GNUnet-CORE service, as CORE is typically the only client of the
3754 * Address validation protocol::
3757 @node Address validation protocol
3758 @subsection Address validation protocol
3761 This section documents how the GNUnet transport service validates
3762 connections with other peers. It is a high-level description of the
3763 protocol necessary to understand the details of the implementation. It
3764 should be noted that when we talk about PING and PONG messages in this
3765 section, we refer to transport-level PING and PONG messages, which are
3766 different from core-level PING and PONG messages (both in implementation
3769 The goal of transport-level address validation is to minimize the chances
3770 of a successful man-in-the-middle attack against GNUnet peers on the
3771 transport level. Such an attack would not allow the adversary to decrypt
3772 the P2P transmissions, but a successful attacker could at least measure
3773 traffic volumes and latencies (raising the adversaries capabilities by
3774 those of a global passive adversary in the worst case). The scenarios we
3775 are concerned about is an attacker, Mallory, giving a @code{HELLO} to
3776 Alice that claims to be for Bob, but contains Mallory's IP address
3777 instead of Bobs (for some transport).
3778 Mallory would then forward the traffic to Bob (by initiating a
3779 connection to Bob and claiming to be Alice). As a further
3780 complication, the scheme has to work even if say Alice is behind a NAT
3781 without traversal support and hence has no address of her own (and thus
3782 Alice must always initiate the connection to Bob).
3784 An additional constraint is that @code{HELLO} messages do not contain a
3785 cryptographic signature since other peers must be able to edit
3786 (i.e. remove) addresses from the @code{HELLO} at any time (this was
3787 not true in GNUnet 0.8.x). A basic @strong{assumption} is that each peer
3788 knows the set of possible network addresses that it @strong{might}
3789 be reachable under (so for example, the external IP address of the
3790 NAT plus the LAN address(es) with the respective ports).
3792 The solution is the following. If Alice wants to validate that a given
3793 address for Bob is valid (i.e. is actually established @strong{directly}
3794 with the intended target), she sends a PING message over that connection
3795 to Bob. Note that in this case, Alice initiated the connection so only
3796 Alice knows which address was used for sure (Alice may be behind NAT, so
3797 whatever address Bob sees may not be an address Alice knows she has).
3798 Bob checks that the address given in the @code{PING} is actually one
3799 of Bob's addresses (ie: does not belong to Mallory), and if it is,
3800 sends back a @code{PONG} (with a signature that says that Bob
3801 owns/uses the address from the @code{PING}).
3802 Alice checks the signature and is happy if it is valid and the address
3803 in the @code{PONG} is the address Alice used.
3804 This is similar to the 0.8.x protocol where the @code{HELLO} contained a
3805 signature from Bob for each address used by Bob.
3806 Here, the purpose code for the signature is
3807 @code{GNUNET_SIGNATURE_PURPOSE_TRANSPORT_PONG_OWN}. After this, Alice will
3808 remember Bob's address and consider the address valid for a while (12h in
3809 the current implementation). Note that after this exchange, Alice only
3810 considers Bob's address to be valid, the connection itself is not
3811 considered 'established'. In particular, Alice may have many addresses
3812 for Bob that Alice considers valid.
3814 The @code{PONG} message is protected with a nonce/challenge against replay
3815 attacks (@uref{http://en.wikipedia.org/wiki/Replay_attack, replay})
3816 and uses an expiration time for the signature (but those are almost
3817 implementation details).
3821 @section NAT library
3824 The goal of the GNUnet NAT library is to provide a general-purpose API for
3825 NAT traversal @strong{without} third-party support. So protocols that
3826 involve contacting a third peer to help establish a connection between
3827 two peers are outside of the scope of this API. That does not mean that
3828 GNUnet doesn't support involving a third peer (we can do this with the
3829 distance-vector transport or using application-level protocols), it just
3830 means that the NAT API is not concerned with this possibility. The API is
3831 written so that it will work for IPv6-NAT in the future as well as
3832 current IPv4-NAT. Furthermore, the NAT API is always used, even for peers
3833 that are not behind NAT --- in that case, the mapping provided is simply
3836 NAT traversal is initiated by calling @code{GNUNET_NAT_register}. Given a
3837 set of addresses that the peer has locally bound to (TCP or UDP), the NAT
3838 library will return (via callback) a (possibly longer) list of addresses
3839 the peer @strong{might} be reachable under. Internally, depending on the
3840 configuration, the NAT library will try to punch a hole (using UPnP) or
3841 just "know" that the NAT was manually punched and generate the respective
3842 external IP address (the one that should be globally visible) based on
3843 the given information.
3845 The NAT library also supports ICMP-based NAT traversal. Here, the other
3846 peer can request connection-reversal by this peer (in this special case,
3847 the peer is even allowed to configure a port number of zero). If the NAT
3848 library detects a connection-reversal request, it returns the respective
3849 target address to the client as well. It should be noted that
3850 connection-reversal is currently only intended for TCP, so other plugins
3851 @strong{must} pass @code{NULL} for the reversal callback. Naturally, the
3852 NAT library also supports requesting connection reversal from a remote
3853 peer (@code{GNUNET_NAT_run_client}).
3855 Once initialized, the NAT handle can be used to test if a given address is
3856 possibly a valid address for this peer (@code{GNUNET_NAT_test_address}).
3857 This is used for validating our addresses when generating PONGs.
3859 Finally, the NAT library contains an API to test if our NAT configuration
3860 is correct. Using @code{GNUNET_NAT_test_start} @strong{before} binding to
3861 the respective port, the NAT library can be used to test if the
3862 configuration works. The test function act as a local client, initialize
3863 the NAT traversal and then contact a @code{gnunet-nat-server} (running by
3864 default on @code{gnunet.org}) and ask for a connection to be established.
3865 This way, it is easy to test if the current NAT configuration is valid.
3867 @node Distance-Vector plugin
3868 @section Distance-Vector plugin
3871 The Distance Vector (DV) transport is a transport mechanism that allows
3872 peers to act as relays for each other, thereby connecting peers that would
3873 otherwise be unable to connect. This gives a larger connection set to
3874 applications that may work better with more peers to choose from (for
3875 example, File Sharing and/or DHT).
3877 The Distance Vector transport essentially has two functions. The first is
3878 "gossiping" connection information about more distant peers to directly
3879 connected peers. The second is taking messages intended for non-directly
3880 connected peers and encapsulating them in a DV wrapper that contains the
3881 required information for routing the message through forwarding peers. Via
3882 gossiping, optimal routes through the known DV neighborhood are discovered
3883 and utilized and the message encapsulation provides some benefits in
3884 addition to simply getting the message from the correct source to the
3887 The gossiping function of DV provides an up to date routing table of
3888 peers that are available up to some number of hops. We call this a
3889 fisheye view of the network (like a fish, nearby objects are known while
3890 more distant ones unknown). Gossip messages are sent only to directly
3891 connected peers, but they are sent about other knowns peers within the
3892 "fisheye distance". Whenever two peers connect, they immediately gossip
3893 to each other about their appropriate other neighbors. They also gossip
3894 about the newly connected peer to previously
3895 connected neighbors. In order to keep the routing tables up to date,
3896 disconnect notifications are propagated as gossip as well (because
3897 disconnects may not be sent/received, timeouts are also used remove
3898 stagnant routing table entries).
3900 Routing of messages via DV is straightforward. When the DV transport is
3901 notified of a message destined for a non-direct neighbor, the appropriate
3902 forwarding peer is selected, and the base message is encapsulated in a DV
3903 message which contains information about the initial peer and the intended
3904 recipient. At each forwarding hop, the initial peer is validated (the
3905 forwarding peer ensures that it has the initial peer in its neighborhood,
3906 otherwise the message is dropped). Next the base message is
3907 re-encapsulated in a new DV message for the next hop in the forwarding
3908 chain (or delivered to the current peer, if it has arrived at the
3911 Assume a three peer network with peers Alice, Bob and Carol. Assume that
3914 Alice <-> Bob and Bob <-> Carol
3918 are direct (e.g. over TCP or UDP transports) connections, but that
3919 Alice cannot directly connect to Carol.
3920 This may be the case due to NAT or firewall restrictions, or perhaps
3921 based on one of the peers respective configurations. If the Distance
3922 Vector transport is enabled on all three peers, it will automatically
3923 discover (from the gossip protocol) that Alice and Carol can connect via
3924 Bob and provide a "virtual" Alice <-> Carol connection. Routing between
3925 Alice and Carol happens as follows; Alice creates a message destined for
3926 Carol and notifies the DV transport about it. The DV transport at Alice
3927 looks up Carol in the routing table and finds that the message must be
3928 sent through Bob for Carol. The message is encapsulated setting Alice as
3929 the initiator and Carol as the destination and sent to Bob. Bob receives
3930 the messages, verifies that both Alice and Carol are known to Bob, and
3931 re-wraps the message in a new DV message for Carol.
3932 The DV transport at Carol receives this message, unwraps the original
3933 message, and delivers it to Carol as though it came directly from Alice.
3937 @section SMTP plugin
3941 This section describes the new SMTP transport plugin for GNUnet as it
3942 exists in the 0.7.x and 0.8.x branch. SMTP support is currently not
3943 available in GNUnet 0.9.x. This page also describes the transport layer
3944 abstraction (as it existed in 0.7.x and 0.8.x) in more detail and gives
3945 some benchmarking results. The performance results presented are quite
3946 old and maybe outdated at this point.
3947 For the readers in the year 2019, you will notice by the mention of
3948 version 0.7, 0.8, and 0.9 that this section has to be taken with your
3949 usual grain of salt and be updated eventually.
3952 @item Why use SMTP for a peer-to-peer transport?
3953 @item SMTPHow does it work?
3954 @item How do I configure my peer?
3955 @item How do I test if it works?
3956 @item How fast is it?
3957 @item Is there any additional documentation?
3962 * Why use SMTP for a peer-to-peer transport?::
3963 * How does it work?::
3964 * How do I configure my peer?::
3965 * How do I test if it works?::
3969 @node Why use SMTP for a peer-to-peer transport?
3970 @subsection Why use SMTP for a peer-to-peer transport?
3973 There are many reasons why one would not want to use SMTP:
3976 @item SMTP is using more bandwidth than TCP, UDP or HTTP
3977 @item SMTP has a much higher latency.
3978 @item SMTP requires significantly more computation (encoding and decoding
3979 time) for the peers.
3980 @item SMTP is significantly more complicated to configure.
3981 @item SMTP may be abused by tricking GNUnet into sending mail to@
3982 non-participating third parties.
3985 So why would anybody want to use SMTP?
3987 @item SMTP can be used to contact peers behind NAT boxes (in virtual
3989 @item SMTP can be used to circumvent policies that limit or prohibit
3990 peer-to-peer traffic by masking as "legitimate" traffic.
3991 @item SMTP uses E-mail addresses which are independent of a specific IP,
3992 which can be useful to address peers that use dynamic IP addresses.
3993 @item SMTP can be used to initiate a connection (e.g. initial address
3994 exchange) and peers can then negotiate the use of a more efficient
3995 protocol (e.g. TCP) for the actual communication.
3998 In summary, SMTP can for example be used to send a message to a peer
3999 behind a NAT box that has a dynamic IP to tell the peer to establish a
4000 TCP connection to a peer outside of the private network. Even an
4001 extraordinary overhead for this first message would be irrelevant in this
4004 @node How does it work?
4005 @subsection How does it work?
4008 When a GNUnet peer needs to send a message to another GNUnet peer that has
4009 advertised (only) an SMTP transport address, GNUnet base64-encodes the
4010 message and sends it in an E-mail to the advertised address. The
4011 advertisement contains a filter which is placed in the E-mail header,
4012 such that the receiving host can filter the tagged E-mails and forward it
4013 to the GNUnet peer process. The filter can be specified individually by
4014 each peer and be changed over time. This makes it impossible to censor
4015 GNUnet E-mail messages by searching for a generic filter.
4017 @node How do I configure my peer?
4018 @subsection How do I configure my peer?
4021 First, you need to configure @code{procmail} to filter your inbound E-mail
4022 for GNUnet traffic. The GNUnet messages must be delivered into a pipe, for
4023 example @code{/tmp/gnunet.smtp}. You also need to define a filter that is
4024 used by @command{procmail} to detect GNUnet messages. You are free to
4025 choose whichever filter you like, but you should make sure that it does
4026 not occur in your other E-mail. In our example, we will use
4027 @code{X-mailer: GNUnet}. The @code{~/.procmailrc} configuration file then
4034 # where do you want your other e-mail delivered to
4035 # (default: /var/spool/mail/)
4036 :0: /var/spool/mail/
4039 After adding this file, first make sure that your regular E-mail still
4040 works (e.g. by sending an E-mail to yourself). Then edit the GNUnet
4041 configuration. In the section @code{SMTP} you need to specify your E-mail
4042 address under @code{EMAIL}, your mail server (for outgoing mail) under
4043 @code{SERVER}, the filter (X-mailer: GNUnet in the example) under
4044 @code{FILTER} and the name of the pipe under @code{PIPE}.@ The completed
4045 section could then look like this:
4048 EMAIL = me@@mail.gnu.org MTU = 65000 SERVER = mail.gnu.org:25 FILTER =
4049 "X-mailer: GNUnet" PIPE = /tmp/gnunet.smtp
4052 Finally, you need to add @code{smtp} to the list of @code{TRANSPORTS} in
4053 the @code{GNUNETD} section. GNUnet peers will use the E-mail address that
4054 you specified to contact your peer until the advertisement times out.
4055 Thus, if you are not sure if everything works properly or if you are not
4056 planning to be online for a long time, you may want to configure this
4057 timeout to be short, e.g. just one hour. For this, set
4058 @code{HELLOEXPIRES} to @code{1} in the @code{GNUNETD} section.
4060 This should be it, but you may probably want to test it first.
4062 @node How do I test if it works?
4063 @subsection How do I test if it works?
4066 Any transport can be subjected to some rudimentary tests using the
4067 @code{gnunet-transport-check} tool. The tool sends a message to the local
4068 node via the transport and checks that a valid message is received. While
4069 this test does not involve other peers and can not check if firewalls or
4070 other network obstacles prohibit proper operation, this is a great
4071 testcase for the SMTP transport since it tests pretty much nearly all of
4074 @code{gnunet-transport-check} should only be used without running
4075 @code{gnunetd} at the same time. By default, @code{gnunet-transport-check}
4076 tests all transports that are specified in the configuration file. But
4077 you can specifically test SMTP by giving the option
4078 @code{--transport=smtp}.
4080 Note that this test always checks if a transport can receive and send.
4081 While you can configure most transports to only receive or only send
4082 messages, this test will only work if you have configured the transport
4083 to send and receive messages.
4085 @node How fast is it?
4086 @subsection How fast is it?
4089 We have measured the performance of the UDP, TCP and SMTP transport layer
4090 directly and when used from an application using the GNUnet core.
4091 Measuring just the transport layer gives the better view of the actual
4092 overhead of the protocol, whereas evaluating the transport from the
4093 application puts the overhead into perspective from a practical point of
4096 The loopback measurements of the SMTP transport were performed on three
4097 different machines spanning a range of modern SMTP configurations. We
4098 used a PIII-800 running RedHat 7.3 with the Purdue Computer Science
4099 configuration which includes filters for spam. We also used a Xenon 2 GHZ
4100 with a vanilla RedHat 8.0 sendmail configuration. Furthermore, we used
4101 qmail on a PIII-1000 running Sorcerer GNU Linux (SGL). The numbers for
4102 UDP and TCP are provided using the SGL configuration. The qmail benchmark
4103 uses qmail's internal filtering whereas the sendmail benchmarks relies on
4104 procmail to filter and deliver the mail. We used the transport layer to
4105 send a message of b bytes (excluding transport protocol headers) directly
4106 to the local machine. This way, network latency and packet loss on the
4107 wire have no impact on the timings. n messages were sent sequentially over
4108 the transport layer, sending message i+1 after the i-th message was
4109 received. All messages were sent over the same connection and the time to
4110 establish the connection was not taken into account since this overhead is
4111 minuscule in practice --- as long as a connection is used for a
4112 significant number of messages.
4114 @multitable @columnfractions .20 .15 .15 .15 .15 .15
4115 @headitem Transport @tab UDP @tab TCP @tab SMTP (Purdue sendmail)
4116 @tab SMTP (RH 8.0) @tab SMTP (SGL qmail)
4117 @item 11 bytes @tab 31 ms @tab 55 ms @tab 781 s @tab 77 s @tab 24 s
4118 @item 407 bytes @tab 37 ms @tab 62 ms @tab 789 s @tab 78 s @tab 25 s
4119 @item 1,221 bytes @tab 46 ms @tab 73 ms @tab 804 s @tab 78 s @tab 25 s
4122 The benchmarks show that UDP and TCP are, as expected, both significantly
4123 faster compared with any of the SMTP services. Among the SMTP
4124 implementations, there can be significant differences depending on the
4125 SMTP configuration. Filtering with an external tool like procmail that
4126 needs to re-parse its configuration for each mail can be very expensive.
4127 Applying spam filters can also significantly impact the performance of
4128 the underlying SMTP implementation. The microbenchmark shows that SMTP
4129 can be a viable solution for initiating peer-to-peer sessions: a couple of
4130 seconds to connect to a peer are probably not even going to be noticed by
4131 users. The next benchmark measures the possible throughput for a
4132 transport. Throughput can be measured by sending multiple messages in
4133 parallel and measuring packet loss. Note that not only UDP but also the
4134 TCP transport can actually loose messages since the TCP implementation
4135 drops messages if the @code{write} to the socket would block. While the
4136 SMTP protocol never drops messages itself, it is often so
4137 slow that only a fraction of the messages can be sent and received in the
4138 given time-bounds. For this benchmark we report the message loss after
4139 allowing t time for sending m messages. If messages were not sent (or
4140 received) after an overall timeout of t, they were considered lost. The
4141 benchmark was performed using two Xeon 2 GHZ machines running RedHat 8.0
4142 with sendmail. The machines were connected with a direct 100 MBit Ethernet
4143 connection.@ Figures udp1200, tcp1200 and smtp-MTUs show that the
4144 throughput for messages of size 1,200 octets is 2,343 kbps, 3,310 kbps
4145 and 6 kbps for UDP, TCP and SMTP respectively. The high per-message
4146 overhead of SMTP can be improved by increasing the MTU, for example, an
4147 MTU of 12,000 octets improves the throughput to 13 kbps as figure
4148 smtp-MTUs shows. Our research paper) has some more details on the
4149 benchmarking results.
4151 @cindex Bluetooth plugin
4152 @node Bluetooth plugin
4153 @section Bluetooth plugin
4156 This page describes the new Bluetooth transport plugin for GNUnet. The
4157 plugin is still in the testing stage so don't expect it to work
4158 perfectly. If you have any questions or problems just post them here or
4159 ask on the IRC channel.
4162 @item What do I need to use the Bluetooth plugin transport?
4163 @item BluetoothHow does it work?
4164 @item What possible errors should I be aware of?
4165 @item How do I configure my peer?
4166 @item How can I test it?
4170 * What do I need to use the Bluetooth plugin transport?::
4171 * How does it work2?::
4172 * What possible errors should I be aware of?::
4173 * How do I configure my peer2?::
4174 * How can I test it?::
4175 * The implementation of the Bluetooth transport plugin::
4178 @node What do I need to use the Bluetooth plugin transport?
4179 @subsection What do I need to use the Bluetooth plugin transport?
4182 If you are a GNU/Linux user and you want to use the Bluetooth
4183 transport plugin you should install the
4184 @command{BlueZ development libraries} (if they aren't already
4186 For instructions about how to install the libraries you should
4187 check out the BlueZ site
4188 (@uref{http://www.bluez.org/, http://www.bluez.org}). If you don't know if
4189 you have the necessary libraries, don't worry, just run the GNUnet
4190 configure script and you will be able to see a notification at the end
4191 which will warn you if you don't have the necessary libraries.
4193 If you are a Windows user you should have installed the
4194 @emph{MinGW}/@emph{MSys2} with the latest updates (especially the
4195 @emph{ws2bth} header). If this is your first build of GNUnet on Windows
4196 you should check out the SBuild repository. It will semi-automatically
4197 assembles a @emph{MinGW}/@emph{MSys2} installation with a lot of extra
4198 packages which are needed for the GNUnet build. So this will ease your
4199 work!@ Finally you just have to be sure that you have the correct drivers
4200 for your Bluetooth device installed and that your device is on and in a
4201 discoverable mode. The Windows Bluetooth Stack supports only the RFCOMM
4202 protocol so we cannot turn on your device programatically!
4204 @c FIXME: Change to unique title
4205 @node How does it work2?
4206 @subsection How does it work2?
4209 The Bluetooth transport plugin uses virtually the same code as the WLAN
4210 plugin and only the helper binary is different. The helper takes a single
4211 argument, which represents the interface name and is specified in the
4212 configuration file. Here are the basic steps that are followed by the
4213 helper binary used on GNU/Linux:
4216 @item it verifies if the name corresponds to a Bluetooth interface name
4217 @item it verifies if the interface is up (if it is not, it tries to bring
4219 @item it tries to enable the page and inquiry scan in order to make the
4220 device discoverable and to accept incoming connection requests
4221 @emph{The above operations require root access so you should start the
4222 transport plugin with root privileges.}
4223 @item it finds an available port number and registers a SDP service which
4224 will be used to find out on which port number is the server listening on
4225 and switch the socket in listening mode
4226 @item it sends a HELLO message with its address
4227 @item finally it forwards traffic from the reading sockets to the STDOUT
4228 and from the STDIN to the writing socket
4231 Once in a while the device will make an inquiry scan to discover the
4232 nearby devices and it will send them randomly HELLO messages for peer
4235 @node What possible errors should I be aware of?
4236 @subsection What possible errors should I be aware of?
4239 @emph{This section is dedicated for GNU/Linux users}
4241 Well there are many ways in which things could go wrong but I will try to
4242 present some tools that you could use to debug and some scenarios.
4246 @item @code{bluetoothd -n -d} : use this command to enable logging in the
4247 foreground and to print the logging messages
4249 @item @code{hciconfig}: can be used to configure the Bluetooth devices.
4250 If you run it without any arguments it will print information about the
4251 state of the interfaces. So if you receive an error that the device
4252 couldn't be brought up you should try to bring it manually and to see if
4253 it works (use @code{hciconfig -a hciX up}). If you can't and the
4254 Bluetooth address has the form 00:00:00:00:00:00 it means that there is
4255 something wrong with the D-Bus daemon or with the Bluetooth daemon. Use
4256 @code{bluetoothd} tool to see the logs
4258 @item @code{sdptool} can be used to control and interrogate SDP servers.
4259 If you encounter problems regarding the SDP server (like the SDP server is
4260 down) you should check out if the D-Bus daemon is running correctly and to
4261 see if the Bluetooth daemon started correctly(use @code{bluetoothd} tool).
4262 Also, sometimes the SDP service could work but somehow the device couldn't
4263 register its service. Use @code{sdptool browse [dev-address]} to see if
4264 the service is registered. There should be a service with the name of the
4265 interface and GNUnet as provider.
4267 @item @code{hcitool} : another useful tool which can be used to configure
4268 the device and to send some particular commands to it.
4270 @item @code{hcidump} : could be used for low level debugging
4273 @c FIXME: A more unique name
4274 @node How do I configure my peer2?
4275 @subsection How do I configure my peer2?
4278 On GNU/Linux, you just have to be sure that the interface name
4279 corresponds to the one that you want to use.
4280 Use the @code{hciconfig} tool to check that.
4281 By default it is set to hci0 but you can change it.
4283 A basic configuration looks like this:
4286 [transport-bluetooth]
4287 # Name of the interface (typically hciX)
4289 # Real hardware, no testing
4290 TESTMODE = 0 TESTING_IGNORE_KEYS = ACCEPT_FROM;
4293 In order to use the Bluetooth transport plugin when the transport service
4294 is started, you must add the plugin name to the default transport service
4295 plugins list. For example:
4298 [transport] ... PLUGINS = dns bluetooth ...
4301 If you want to use only the Bluetooth plugin set
4302 @emph{PLUGINS = bluetooth}
4304 On Windows, you cannot specify which device to use. The only thing that
4305 you should do is to add @emph{bluetooth} on the plugins list of the
4308 @node How can I test it?
4309 @subsection How can I test it?
4312 If you have two Bluetooth devices on the same machine and you are using
4317 @item create two different file configuration (one which will use the
4318 first interface (@emph{hci0}) and the other which will use the second
4319 interface (@emph{hci1})). Let's name them @emph{peer1.conf} and
4322 @item run @emph{gnunet-peerinfo -c peerX.conf -s} in order to generate the
4323 peers private keys. The @strong{X} must be replace with 1 or 2.
4325 @item run @emph{gnunet-arm -c peerX.conf -s -i=transport} in order to
4326 start the transport service. (Make sure that you have "bluetooth" on the
4327 transport plugins list if the Bluetooth transport service doesn't start.)
4329 @item run @emph{gnunet-peerinfo -c peer1.conf -s} to get the first peer's
4330 ID. If you already know your peer ID (you saved it from the first
4331 command), this can be skipped.
4333 @item run @emph{gnunet-transport -c peer2.conf -p=PEER1_ID -s} to start
4334 sending data for benchmarking to the other peer.
4339 This scenario will try to connect the second peer to the first one and
4340 then start sending data for benchmarking.
4342 On Windows you cannot test the plugin functionality using two Bluetooth
4343 devices from the same machine because after you install the drivers there
4344 will occur some conflicts between the Bluetooth stacks. (At least that is
4345 what happened on my machine : I wasn't able to use the Bluesoleil stack and
4346 the WINDCOMM one in the same time).
4348 If you have two different machines and your configuration files are good
4349 you can use the same scenario presented on the beginning of this section.
4351 Another way to test the plugin functionality is to create your own
4352 application which will use the GNUnet framework with the Bluetooth
4355 @node The implementation of the Bluetooth transport plugin
4356 @subsection The implementation of the Bluetooth transport plugin
4359 This page describes the implementation of the Bluetooth transport plugin.
4361 First I want to remind you that the Bluetooth transport plugin uses
4362 virtually the same code as the WLAN plugin and only the helper binary is
4363 different. Also the scope of the helper binary from the Bluetooth
4364 transport plugin is the same as the one used for the WLAN transport
4365 plugin: it accesses the interface and then it forwards traffic in both
4366 directions between the Bluetooth interface and stdin/stdout of the
4369 The Bluetooth plugin transport could be used both on GNU/Linux and Windows
4373 @item Linux functionality
4374 @item Windows functionality
4375 @item Pending Features
4381 * Linux functionality::
4382 * THE INITIALIZATION::
4384 * Details about the broadcast implementation::
4385 * Windows functionality::
4386 * Pending features::
4389 @node Linux functionality
4390 @subsubsection Linux functionality
4393 In order to implement the plugin functionality on GNU/Linux I
4394 used the BlueZ stack.
4395 For the communication with the other devices I used the RFCOMM
4396 protocol. Also I used the HCI protocol to gain some control over the
4397 device. The helper binary takes a single argument (the name of the
4398 Bluetooth interface) and is separated in two stages:
4400 @c %** 'THE INITIALIZATION' should be in bigger letters or stand out, not
4401 @c %** starting a new section?
4402 @node THE INITIALIZATION
4403 @subsubsection THE INITIALIZATION
4406 @item first, it checks if we have root privileges
4407 (@emph{Remember that we need to have root privileges in order to be able
4408 to bring the interface up if it is down or to change its state.}).
4410 @item second, it verifies if the interface with the given name exists.
4412 @strong{If the interface with that name exists and it is a Bluetooth
4415 @item it creates a RFCOMM socket which will be used for listening and call
4416 the @emph{open_device} method
4418 On the @emph{open_device} method:
4420 @item creates a HCI socket used to send control events to the the device
4421 @item searches for the device ID using the interface name
4422 @item saves the device MAC address
4423 @item checks if the interface is down and tries to bring it UP
4424 @item checks if the interface is in discoverable mode and tries to make it
4426 @item closes the HCI socket and binds the RFCOMM one
4427 @item switches the RFCOMM socket in listening mode
4428 @item registers the SDP service (the service will be used by the other
4429 devices to get the port on which this device is listening on)
4432 @item drops the root privileges
4434 @strong{If the interface is not a Bluetooth interface the helper exits
4435 with a suitable error}
4438 @c %** Same as for @node entry above
4440 @subsubsection THE LOOP
4442 The helper binary uses a list where it saves all the connected neighbour
4443 devices (@emph{neighbours.devices}) and two buffers (@emph{write_pout} and
4444 @emph{write_std}). The first message which is send is a control message
4445 with the device's MAC address in order to announce the peer presence to
4446 the neighbours. Here are a short description of what happens in the main
4450 @item Every time when it receives something from the STDIN it processes
4451 the data and saves the message in the first buffer (@emph{write_pout}).
4452 When it has something in the buffer, it gets the destination address from
4453 the buffer, searches the destination address in the list (if there is no
4454 connection with that device, it creates a new one and saves it to the
4455 list) and sends the message.
4456 @item Every time when it receives something on the listening socket it
4457 accepts the connection and saves the socket on a list with the reading
4458 sockets. @item Every time when it receives something from a reading
4459 socket it parses the message, verifies the CRC and saves it in the
4460 @emph{write_std} buffer in order to be sent later to the STDOUT.
4463 So in the main loop we use the select function to wait until one of the
4464 file descriptor saved in one of the two file descriptors sets used is
4465 ready to use. The first set (@emph{rfds}) represents the reading set and
4466 it could contain the list with the reading sockets, the STDIN file
4467 descriptor or the listening socket. The second set (@emph{wfds}) is the
4468 writing set and it could contain the sending socket or the STDOUT file
4469 descriptor. After the select function returns, we check which file
4470 descriptor is ready to use and we do what is supposed to do on that kind
4471 of event. @emph{For example:} if it is the listening socket then we
4472 accept a new connection and save the socket in the reading list; if it is
4473 the STDOUT file descriptor, then we write to STDOUT the message from the
4474 @emph{write_std} buffer.
4476 To find out on which port a device is listening on we connect to the local
4477 SDP server and search the registered service for that device.
4479 @emph{You should be aware of the fact that if the device fails to connect
4480 to another one when trying to send a message it will attempt one more
4481 time. If it fails again, then it skips the message.}
4482 @emph{Also you should know that the transport Bluetooth plugin has
4483 support for @strong{broadcast messages}.}
4485 @node Details about the broadcast implementation
4486 @subsubsection Details about the broadcast implementation
4489 First I want to point out that the broadcast functionality for the CONTROL
4490 messages is not implemented in a conventional way. Since the inquiry scan
4491 time is too big and it will take some time to send a message to all the
4492 discoverable devices I decided to tackle the problem in a different way.
4493 Here is how I did it:
4496 @item If it is the first time when I have to broadcast a message I make an
4497 inquiry scan and save all the devices' addresses to a vector.
4498 @item After the inquiry scan ends I take the first address from the list
4499 and I try to connect to it. If it fails, I try to connect to the next one.
4500 If it succeeds, I save the socket to a list and send the message to the
4502 @item When I have to broadcast another message, first I search on the list
4503 for a new device which I'm not connected to. If there is no new device on
4504 the list I go to the beginning of the list and send the message to the
4505 old devices. After 5 cycles I make a new inquiry scan to check out if
4506 there are new discoverable devices and save them to the list. If there
4507 are no new discoverable devices I reset the cycling counter and go again
4508 through the old list and send messages to the devices saved in it.
4514 @item every time when I have a broadcast message I look up on the list
4515 for a new device and send the message to it
4516 @item if I reached the end of the list for 5 times and I'm connected to
4517 all the devices from the list I make a new inquiry scan.
4518 @emph{The number of the list's cycles after an inquiry scan could be
4519 increased by redefining the MAX_LOOPS variable}
4520 @item when there are no new devices I send messages to the old ones.
4523 Doing so, the broadcast control messages will reach the devices but with
4526 @emph{NOTICE:} When I have to send a message to a certain device first I
4527 check on the broadcast list to see if we are connected to that device. If
4528 not we try to connect to it and in case of success we save the address and
4529 the socket on the list. If we are already connected to that device we
4530 simply use the socket.
4532 @node Windows functionality
4533 @subsubsection Windows functionality
4536 For Windows I decided to use the Microsoft Bluetooth stack which has the
4537 advantage of coming standard from Windows XP SP2. The main disadvantage is
4538 that it only supports the RFCOMM protocol so we will not be able to have
4539 a low level control over the Bluetooth device. Therefore it is the user
4540 responsibility to check if the device is up and in the discoverable mode.
4541 Also there are no tools which could be used for debugging in order to read
4542 the data coming from and going to a Bluetooth device, which obviously
4543 hindered my work. Another thing that slowed down the implementation of the
4544 plugin (besides that I wasn't too accommodated with the win32 API) was that
4545 there were some bugs on MinGW regarding the Bluetooth. Now they are solved
4546 but you should keep in mind that you should have the latest updates
4547 (especially the @emph{ws2bth} header).
4549 Besides the fact that it uses the Windows Sockets, the Windows
4550 implementation follows the same principles as the GNU/Linux one:
4553 @item It has a initalization part where it initializes the
4554 Windows Sockets, creates a RFCOMM socket which will be binded and switched
4555 to the listening mode and registers a SDP service. In the Microsoft
4556 Bluetooth API there are two ways to work with the SDP:
4558 @item an easy way which works with very simple service records
4559 @item a hard way which is useful when you need to update or to delete the
4564 Since I only needed the SDP service to find out on which port the device
4565 is listening on and that did not change, I decided to use the easy way.
4566 In order to register the service I used the @emph{WSASetService} function
4567 and I generated the @emph{Universally Unique Identifier} with the
4568 @emph{guidgen.exe} Windows's tool.
4570 In the loop section the only difference from the GNU/Linux implementation
4571 is that I used the @code{GNUNET_NETWORK} library for
4572 functions like @emph{accept}, @emph{bind}, @emph{connect} or
4573 @emph{select}. I decided to use the
4574 @code{GNUNET_NETWORK} library because I also needed to interact
4575 with the STDIN and STDOUT handles and on Windows
4576 the select function is only defined for sockets,
4577 and it will not work for arbitrary file handles.
4579 Another difference between GNU/Linux and Windows implementation is that in
4580 GNU/Linux, the Bluetooth address is represented in 48 bits
4581 while in Windows is represented in 64 bits.
4582 Therefore I had to do some changes on @emph{plugin_transport_wlan} header.
4584 Also, currently on Windows the Bluetooth plugin doesn't have support for
4585 broadcast messages. When it receives a broadcast message it will skip it.
4587 @node Pending features
4588 @subsubsection Pending features
4592 @item Implement the broadcast functionality on Windows @emph{(currently
4594 @item Implement a testcase for the helper :@ @emph{The testcase
4595 consists of a program which emulates the plugin and uses the helper. It
4596 will simulate connections, disconnections and data transfers.}
4599 If you have a new idea about a feature of the plugin or suggestions about
4600 how I could improve the implementation you are welcome to comment or to
4604 @section WLAN plugin
4607 This section documents how the wlan transport plugin works. Parts which
4608 are not implemented yet or could be better implemented are described at
4611 @cindex ATS Subsystem
4613 @section ATS Subsystem
4616 ATS stands for "automatic transport selection", and the function of ATS in
4617 GNUnet is to decide on which address (and thus transport plugin) should
4618 be used for two peers to communicate, and what bandwidth limits should be
4619 imposed on such an individual connection. To help ATS make an informed
4620 decision, higher-level services inform the ATS service about their
4621 requirements and the quality of the service rendered. The ATS service
4622 also interacts with the transport service to be appraised of working
4623 addresses and to communicate its resource allocation decisions. Finally,
4624 the ATS service's operation can be observed using a monitoring API.
4626 The main logic of the ATS service only collects the available addresses,
4627 their performance characteristics and the applications requirements, but
4628 does not make the actual allocation decision. This last critical step is
4629 left to an ATS plugin, as we have implemented (currently three) different
4630 allocation strategies which differ significantly in their performance and
4631 maturity, and it is still unclear if any particular plugin is generally
4634 @cindex CORE Subsystem
4635 @node CORE Subsystem
4636 @section CORE Subsystem
4639 The CORE subsystem in GNUnet is responsible for securing link-layer
4640 communications between nodes in the GNUnet overlay network. CORE builds
4641 on the TRANSPORT subsystem which provides for the actual, insecure,
4642 unreliable link-layer communication (for example, via UDP or WLAN), and
4643 then adds fundamental security to the connections:
4646 @item confidentiality with so-called perfect forward secrecy; we use
4648 (@uref{http://en.wikipedia.org/wiki/Elliptic_curve_Diffie%E2%80%93Hellman, Elliptic-curve Diffie---Hellman})
4649 powered by Curve25519
4650 (@uref{http://cr.yp.to/ecdh.html, Curve25519}) for the key
4651 exchange and then use symmetric encryption, encrypting with both AES-256
4652 (@uref{http://en.wikipedia.org/wiki/Rijndael, AES-256}) and
4653 Twofish (@uref{http://en.wikipedia.org/wiki/Twofish, Twofish})
4654 @item @uref{http://en.wikipedia.org/wiki/Authentication, authentication}
4655 is achieved by signing the ephemeral keys using Ed25519
4656 (@uref{http://ed25519.cr.yp.to/, Ed25519}), a deterministic
4658 (@uref{http://en.wikipedia.org/wiki/ECDSA, ECDSA})
4659 @item integrity protection (using SHA-512
4660 (@uref{http://en.wikipedia.org/wiki/SHA-2, SHA-512}) to do
4662 (@uref{http://en.wikipedia.org/wiki/Authenticated_encryption, encrypt-then-MAC}))
4664 (@uref{http://en.wikipedia.org/wiki/Replay_attack, replay})
4665 protection (using nonces, timestamps, challenge-response,
4666 message counters and ephemeral keys)
4667 @item liveness (keep-alive messages, timeout)
4672 * When is a peer "connected"?::
4674 * The CORE Client-Service Protocol::
4675 * The CORE Peer-to-Peer Protocol::
4678 @cindex core subsystem limitations
4680 @subsection Limitations
4683 CORE does not perform
4684 @uref{http://en.wikipedia.org/wiki/Routing, routing}; using CORE it is
4685 only possible to communicate with peers that happen to already be
4686 "directly" connected with each other. CORE also does not have an
4687 API to allow applications to establish such "direct" connections --- for
4688 this, applications can ask TRANSPORT, but TRANSPORT might not be able to
4689 establish a "direct" connection. The TOPOLOGY subsystem is responsible for
4690 trying to keep a few "direct" connections open at all times. Applications
4691 that need to talk to particular peers should use the CADET subsystem, as
4692 it can establish arbitrary "indirect" connections.
4694 Because CORE does not perform routing, CORE must only be used directly by
4695 applications that either perform their own routing logic (such as
4696 anonymous file-sharing) or that do not require routing, for example
4697 because they are based on flooding the network. CORE communication is
4698 unreliable and delivery is possibly out-of-order. Applications that
4699 require reliable communication should use the CADET service. Each
4700 application can only queue one message per target peer with the CORE
4701 service at any time; messages cannot be larger than approximately
4702 63 kilobytes. If messages are small, CORE may group multiple messages
4703 (possibly from different applications) prior to encryption. If permitted
4704 by the application (using the @uref{http://baus.net/on-tcp_cork/, cork}
4705 option), CORE may delay transmissions to facilitate grouping of multiple
4706 small messages. If cork is not enabled, CORE will transmit the message as
4707 soon as TRANSPORT allows it (TRANSPORT is responsible for limiting
4708 bandwidth and congestion control). CORE does not allow flow control;
4709 applications are expected to process messages at line-speed. If flow
4710 control is needed, applications should use the CADET service.
4712 @cindex when is a peer connected
4713 @node When is a peer "connected"?
4714 @subsection When is a peer "connected"?
4717 In addition to the security features mentioned above, CORE also provides
4718 one additional key feature to applications using it, and that is a
4719 limited form of protocol-compatibility checking. CORE distinguishes
4720 between TRANSPORT-level connections (which enable communication with other
4721 peers) and application-level connections. Applications using the CORE API
4722 will (typically) learn about application-level connections from CORE, and
4723 not about TRANSPORT-level connections. When a typical application uses
4724 CORE, it will specify a set of message types
4725 (from @code{gnunet_protocols.h}) that it understands. CORE will then
4726 notify the application about connections it has with other peers if and
4727 only if those applications registered an intersecting set of message
4728 types with their CORE service. Thus, it is quite possible that CORE only
4729 exposes a subset of the established direct connections to a particular
4730 application --- and different applications running above CORE might see
4731 different sets of connections at the same time.
4733 A special case are applications that do not register a handler for any
4735 CORE assumes that these applications merely want to monitor connections
4736 (or "all" messages via other callbacks) and will notify those applications
4737 about all connections. This is used, for example, by the
4738 @code{gnunet-core} command-line tool to display the active connections.
4739 Note that it is also possible that the TRANSPORT service has more active
4740 connections than the CORE service, as the CORE service first has to
4741 perform a key exchange with connecting peers before exchanging information
4742 about supported message types and notifying applications about the new
4745 @cindex libgnunetcore
4747 @subsection libgnunetcore
4750 The CORE API (defined in @file{gnunet_core_service.h}) is the basic
4751 messaging API used by P2P applications built using GNUnet. It provides
4752 applications the ability to send and receive encrypted messages to the
4753 peer's "directly" connected neighbours.
4755 As CORE connections are generally "direct" connections,@ applications must
4756 not assume that they can connect to arbitrary peers this way, as "direct"
4757 connections may not always be possible. Applications using CORE are
4758 notified about which peers are connected. Creating new "direct"
4759 connections must be done using the TRANSPORT API.
4761 The CORE API provides unreliable, out-of-order delivery. While the
4762 implementation tries to ensure timely, in-order delivery, both message
4763 losses and reordering are not detected and must be tolerated by the
4764 application. Most important, the core will NOT perform retransmission if
4765 messages could not be delivered.
4767 Note that CORE allows applications to queue one message per connected
4768 peer. The rate at which each connection operates is influenced by the
4769 preferences expressed by local application as well as restrictions
4770 imposed by the other peer. Local applications can express their
4771 preferences for particular connections using the "performance" API of the
4774 Applications that require more sophisticated transmission capabilities
4775 such as TCP-like behavior, or if you intend to send messages to arbitrary
4776 remote peers, should use the CADET API.
4778 The typical use of the CORE API is to connect to the CORE service using
4779 @code{GNUNET_CORE_connect}, process events from the CORE service (such as
4780 peers connecting, peers disconnecting and incoming messages) and send
4781 messages to connected peers using
4782 @code{GNUNET_CORE_notify_transmit_ready}. Note that applications must
4783 cancel pending transmission requests if they receive a disconnect event
4784 for a peer that had a transmission pending; furthermore, queuing more
4785 than one transmission request per peer per application using the
4786 service is not permitted.
4788 The CORE API also allows applications to monitor all communications of the
4789 peer prior to encryption (for outgoing messages) or after decryption (for
4790 incoming messages). This can be useful for debugging, diagnostics or to
4791 establish the presence of cover traffic (for anonymity). As monitoring
4792 applications are often not interested in the payload, the monitoring
4793 callbacks can be configured to only provide the message headers (including
4794 the message type and size) instead of copying the full data stream to the
4797 The init callback of the @code{GNUNET_CORE_connect} function is called
4798 with the hash of the public key of the peer. This public key is used to
4799 identify the peer globally in the GNUnet network. Applications are
4800 encouraged to check that the provided hash matches the hash that they are
4801 using (as theoretically the application may be using a different
4802 configuration file with a different private key, which would result in
4805 As with most service APIs, the CORE API isolates applications from crashes
4806 of the CORE service. If the CORE service crashes, the application will see
4807 disconnect events for all existing connections. Once the connections are
4808 re-established, the applications will be receive matching connect events.
4810 @cindex core clinet-service protocol
4811 @node The CORE Client-Service Protocol
4812 @subsection The CORE Client-Service Protocol
4815 This section describes the protocol between an application using the CORE
4816 service (the client) and the CORE service process itself.
4826 @subsubsection Setup2
4829 When a client connects to the CORE service, it first sends a
4830 @code{InitMessage} which specifies options for the connection and a set of
4831 message type values which are supported by the application. The options
4832 bitmask specifies which events the client would like to be notified about.
4833 The options include:
4836 @item GNUNET_CORE_OPTION_NOTHING No notifications
4837 @item GNUNET_CORE_OPTION_STATUS_CHANGE Peers connecting and disconnecting
4838 @item GNUNET_CORE_OPTION_FULL_INBOUND All inbound messages (after
4839 decryption) with full payload
4840 @item GNUNET_CORE_OPTION_HDR_INBOUND Just the @code{MessageHeader}
4841 of all inbound messages
4842 @item GNUNET_CORE_OPTION_FULL_OUTBOUND All outbound
4843 messages (prior to encryption) with full payload
4844 @item GNUNET_CORE_OPTION_HDR_OUTBOUND Just the @code{MessageHeader} of all
4848 Typical applications will only monitor for connection status changes.
4850 The CORE service responds to the @code{InitMessage} with an
4851 @code{InitReplyMessage} which contains the peer's identity. Afterwards,
4852 both CORE and the client can send messages.
4855 @subsubsection Notifications
4858 The CORE will send @code{ConnectNotifyMessage}s and
4859 @code{DisconnectNotifyMessage}s whenever peers connect or disconnect from
4860 the CORE (assuming their type maps overlap with the message types
4861 registered by the client). When the CORE receives a message that matches
4862 the set of message types specified during the @code{InitMessage} (or if
4863 monitoring is enabled in for inbound messages in the options), it sends a
4864 @code{NotifyTrafficMessage} with the peer identity of the sender and the
4865 decrypted payload. The same message format (except with
4866 @code{GNUNET_MESSAGE_TYPE_CORE_NOTIFY_OUTBOUND} for the message type) is
4867 used to notify clients monitoring outbound messages; here, the peer
4868 identity given is that of the receiver.
4871 @subsubsection Sending
4874 When a client wants to transmit a message, it first requests a
4875 transmission slot by sending a @code{SendMessageRequest} which specifies
4876 the priority, deadline and size of the message. Note that these values
4877 may be ignored by CORE. When CORE is ready for the message, it answers
4878 with a @code{SendMessageReady} response. The client can then transmit the
4879 payload with a @code{SendMessage} message. Note that the actual message
4880 size in the @code{SendMessage} is allowed to be smaller than the size in
4881 the original request. A client may at any time send a fresh
4882 @code{SendMessageRequest}, which then superceeds the previous
4883 @code{SendMessageRequest}, which is then no longer valid. The client can
4884 tell which @code{SendMessageRequest} the CORE service's
4885 @code{SendMessageReady} message is for as all of these messages contain a
4886 "unique" request ID (based on a counter incremented by the client
4889 @cindex CORE Peer-to-Peer Protocol
4890 @node The CORE Peer-to-Peer Protocol
4891 @subsection The CORE Peer-to-Peer Protocol
4896 * Creating the EphemeralKeyMessage::
4897 * Establishing a connection::
4898 * Encryption and Decryption::
4902 @cindex EphemeralKeyMessage creation
4903 @node Creating the EphemeralKeyMessage
4904 @subsubsection Creating the EphemeralKeyMessage
4907 When the CORE service starts, each peer creates a fresh ephemeral (ECC)
4908 public-private key pair and signs the corresponding
4909 @code{EphemeralKeyMessage} with its long-term key (which we usually call
4910 the peer's identity; the hash of the public long term key is what results
4911 in a @code{struct GNUNET_PeerIdentity} in all GNUnet APIs. The ephemeral
4912 key is ONLY used for an ECDHE
4913 (@uref{http://en.wikipedia.org/wiki/Elliptic_curve_Diffie%E2%80%93Hellman, Elliptic-curve Diffie---Hellman})
4914 exchange by the CORE service to establish symmetric session keys. A peer
4915 will use the same @code{EphemeralKeyMessage} for all peers for
4916 @code{REKEY_FREQUENCY}, which is usually 12 hours. After that time, it
4917 will create a fresh ephemeral key (forgetting the old one) and broadcast
4918 the new @code{EphemeralKeyMessage} to all connected peers, resulting in
4919 fresh symmetric session keys. Note that peers independently decide on
4920 when to discard ephemeral keys; it is not a protocol violation to discard
4921 keys more often. Ephemeral keys are also never stored to disk; restarting
4922 a peer will thus always create a fresh ephemeral key. The use of ephemeral
4923 keys is what provides @uref{http://en.wikipedia.org/wiki/Forward_secrecy, forward secrecy}.
4925 Just before transmission, the @code{EphemeralKeyMessage} is patched to
4926 reflect the current sender_status, which specifies the current state of
4927 the connection from the point of view of the sender. The possible values
4931 @item @code{KX_STATE_DOWN} Initial value, never used on the network
4932 @item @code{KX_STATE_KEY_SENT} We sent our ephemeral key, do not know the
4933 key of the other peer
4934 @item @code{KX_STATE_KEY_RECEIVED} This peer has received a valid
4935 ephemeral key of the other peer, but we are waiting for the other peer to
4936 confirm it's authenticity (ability to decode) via challenge-response.
4937 @item @code{KX_STATE_UP} The connection is fully up from the point of
4938 view of the sender (now performing keep-alives)
4939 @item @code{KX_STATE_REKEY_SENT} The sender has initiated a rekeying
4940 operation; the other peer has so far failed to confirm a working
4941 connection using the new ephemeral key
4944 @node Establishing a connection
4945 @subsubsection Establishing a connection
4948 Peers begin their interaction by sending a @code{EphemeralKeyMessage} to
4949 the other peer once the TRANSPORT service notifies the CORE service about
4951 A peer receiving an @code{EphemeralKeyMessage} with a status
4952 indicating that the sender does not have the receiver's ephemeral key, the
4953 receiver's @code{EphemeralKeyMessage} is sent in response.
4954 Additionally, if the receiver has not yet confirmed the authenticity of
4955 the sender, it also sends an (encrypted)@code{PingMessage} with a
4956 challenge (and the identity of the target) to the other peer. Peers
4957 receiving a @code{PingMessage} respond with an (encrypted)
4958 @code{PongMessage} which includes the challenge. Peers receiving a
4959 @code{PongMessage} check the challenge, and if it matches set the
4960 connection to @code{KX_STATE_UP}.
4962 @node Encryption and Decryption
4963 @subsubsection Encryption and Decryption
4966 All functions related to the key exchange and encryption/decryption of
4967 messages can be found in @file{gnunet-service-core_kx.c} (except for the
4968 cryptographic primitives, which are in @file{util/crypto*.c}).
4969 Given the key material from ECDHE, a Key derivation function
4970 (@uref{https://en.wikipedia.org/wiki/Key_derivation_function, Key derivation function})
4971 is used to derive two pairs of encryption and decryption keys for AES-256
4972 and TwoFish, as well as initialization vectors and authentication keys
4974 (@uref{https://en.wikipedia.org/wiki/HMAC, HMAC})).
4975 The HMAC is computed over the encrypted payload.
4976 Encrypted messages include an iv_seed and the HMAC in the header.
4978 Each encrypted message in the CORE service includes a sequence number and
4979 a timestamp in the encrypted payload. The CORE service remembers the
4980 largest observed sequence number and a bit-mask which represents which of
4981 the previous 32 sequence numbers were already used.
4982 Messages with sequence numbers lower than the largest observed sequence
4983 number minus 32 are discarded. Messages with a timestamp that is less
4984 than @code{REKEY_TOLERANCE} off (5 minutes) are also discarded. This of
4985 course means that system clocks need to be reasonably synchronized for
4986 peers to be able to communicate. Additionally, as the ephemeral key
4987 changes every 12 hours, a peer would not even be able to decrypt messages
4988 older than 12 hours.
4991 @subsubsection Type maps
4994 Once an encrypted connection has been established, peers begin to exchange
4995 type maps. Type maps are used to allow the CORE service to determine which
4996 (encrypted) connections should be shown to which applications. A type map
4997 is an array of 65536 bits representing the different types of messages
4998 understood by applications using the CORE service. Each CORE service
4999 maintains this map, simply by setting the respective bit for each message
5000 type supported by any of the applications using the CORE service. Note
5001 that bits for message types embedded in higher-level protocols (such as
5002 MESH) will not be included in these type maps.
5004 Typically, the type map of a peer will be sparse. Thus, the CORE service
5005 attempts to compress its type map using @code{gzip}-style compression
5006 ("deflate") prior to transmission. However, if the compression fails to
5007 compact the map, the map may also be transmitted without compression
5008 (resulting in @code{GNUNET_MESSAGE_TYPE_CORE_COMPRESSED_TYPE_MAP} or
5009 @code{GNUNET_MESSAGE_TYPE_CORE_BINARY_TYPE_MAP} messages respectively).
5010 Upon receiving a type map, the respective CORE service notifies
5011 applications about the connection to the other peer if they support any
5012 message type indicated in the type map (or no message type at all).
5013 If the CORE service experience a connect or disconnect event from an
5014 application, it updates its type map (setting or unsetting the respective
5015 bits) and notifies its neighbours about the change.
5016 The CORE services of the neighbours then in turn generate connect and
5017 disconnect events for the peer that sent the type map for their respective
5018 applications. As CORE messages may be lost, the CORE service confirms
5019 receiving a type map by sending back a
5020 @code{GNUNET_MESSAGE_TYPE_CORE_CONFIRM_TYPE_MAP}. If such a confirmation
5021 (with the correct hash of the type map) is not received, the sender will
5022 retransmit the type map (with exponential back-off).
5024 @cindex CADET Subsystem
5027 @node CADET Subsystem
5028 @section CADET Subsystem
5030 The CADET subsystem in GNUnet is responsible for secure end-to-end
5031 communications between nodes in the GNUnet overlay network. CADET builds
5032 on the CORE subsystem which provides for the link-layer communication and
5033 then adds routing, forwarding and additional security to the connections.
5034 CADET offers the same cryptographic services as CORE, but on an
5035 end-to-end level. This is done so peers retransmitting traffic on behalf
5036 of other peers cannot access the payload data.
5039 @item CADET provides confidentiality with so-called perfect forward
5040 secrecy; we use ECDHE powered by Curve25519 for the key exchange and then
5041 use symmetric encryption, encrypting with both AES-256 and Twofish
5042 @item authentication is achieved by signing the ephemeral keys using
5043 Ed25519, a deterministic variant of ECDSA
5044 @item integrity protection (using SHA-512 to do encrypt-then-MAC, although
5045 only 256 bits are sent to reduce overhead)
5046 @item replay protection (using nonces, timestamps, challenge-response,
5047 message counters and ephemeral keys)
5048 @item liveness (keep-alive messages, timeout)
5051 Additional to the CORE-like security benefits, CADET offers other
5052 properties that make it a more universal service than CORE.
5055 @item CADET can establish channels to arbitrary peers in GNUnet. If a
5056 peer is not immediately reachable, CADET will find a path through the
5057 network and ask other peers to retransmit the traffic on its behalf.
5058 @item CADET offers (optional) reliability mechanisms. In a reliable
5059 channel traffic is guaranteed to arrive complete, unchanged and in-order.
5060 @item CADET takes care of flow and congestion control mechanisms, not
5061 allowing the sender to send more traffic than the receiver or the network
5062 are able to process.
5069 @cindex libgnunetcadet
5070 @node libgnunetcadet
5071 @subsection libgnunetcadet
5074 The CADET API (defined in @file{gnunet_cadet_service.h}) is the
5075 messaging API used by P2P applications built using GNUnet.
5076 It provides applications the ability to send and receive encrypted
5077 messages to any peer participating in GNUnet.
5078 The API is heavily base on the CORE API.
5080 CADET delivers messages to other peers in "channels".
5081 A channel is a permanent connection defined by a destination peer
5082 (identified by its public key) and a port number.
5083 Internally, CADET tunnels all channels towards a destination peer
5084 using one session key and relays the data on multiple "connections",
5085 independent from the channels.
5087 Each channel has optional parameters, the most important being the
5089 Should a message get lost on TRANSPORT/CORE level, if a channel is
5090 created with as reliable, CADET will retransmit the lost message and
5091 deliver it in order to the destination application.
5093 @pindex GNUNET_CADET_connect
5094 To communicate with other peers using CADET, it is necessary to first
5095 connect to the service using @code{GNUNET_CADET_connect}.
5096 This function takes several parameters in form of callbacks, to allow the
5097 client to react to various events, like incoming channels or channels that
5098 terminate, as well as specify a list of ports the client wishes to listen
5099 to (at the moment it is not possible to start listening on further ports
5100 once connected, but nothing prevents a client to connect several times to
5101 CADET, even do one connection per listening port).
5102 The function returns a handle which has to be used for any further
5103 interaction with the service.
5105 @pindex GNUNET_CADET_channel_create
5106 To connect to a remote peer, a client has to call the
5107 @code{GNUNET_CADET_channel_create} function. The most important parameters
5108 given are the remote peer's identity (it public key) and a port, which
5109 specifies which application on the remote peer to connect to, similar to
5110 TCP/UDP ports. CADET will then find the peer in the GNUnet network and
5111 establish the proper low-level connections and do the necessary key
5112 exchanges to assure and authenticated, secure and verified communication.
5113 Similar to @code{GNUNET_CADET_connect},@code{GNUNET_CADET_create_channel}
5114 returns a handle to interact with the created channel.
5116 @pindex GNUNET_CADET_notify_transmit_ready
5117 For every message the client wants to send to the remote application,
5118 @code{GNUNET_CADET_notify_transmit_ready} must be called, indicating the
5119 channel on which the message should be sent and the size of the message
5120 (but not the message itself!). Once CADET is ready to send the message,
5121 the provided callback will fire, and the message contents are provided to
5124 Please note the CADET does not provide an explicit notification of when a
5125 channel is connected. In loosely connected networks, like big wireless
5126 mesh networks, this can take several seconds, even minutes in the worst
5127 case. To be alerted when a channel is online, a client can call
5128 @code{GNUNET_CADET_notify_transmit_ready} immediately after
5129 @code{GNUNET_CADET_create_channel}. When the callback is activated, it
5130 means that the channel is online. The callback can give 0 bytes to CADET
5131 if no message is to be sent, this is OK.
5133 @pindex GNUNET_CADET_notify_transmit_cancel
5134 If a transmission was requested but before the callback fires it is no
5135 longer needed, it can be canceled with
5136 @code{GNUNET_CADET_notify_transmit_ready_cancel}, which uses the handle
5137 given back by @code{GNUNET_CADET_notify_transmit_ready}.
5138 As in the case of CORE, only one message can be requested at a time: a
5139 client must not call @code{GNUNET_CADET_notify_transmit_ready} again until
5140 the callback is called or the request is canceled.
5142 @pindex GNUNET_CADET_channel_destroy
5143 When a channel is no longer needed, a client can call
5144 @code{GNUNET_CADET_channel_destroy} to get rid of it.
5145 Note that CADET will try to transmit all pending traffic before notifying
5146 the remote peer of the destruction of the channel, including
5147 retransmitting lost messages if the channel was reliable.
5149 Incoming channels, channels being closed by the remote peer, and traffic
5150 on any incoming or outgoing channels are given to the client when CADET
5151 executes the callbacks given to it at the time of
5152 @code{GNUNET_CADET_connect}.
5154 @pindex GNUNET_CADET_disconnect
5155 Finally, when an application no longer wants to use CADET, it should call
5156 @code{GNUNET_CADET_disconnect}, but first all channels and pending
5157 transmissions must be closed (otherwise CADET will complain).
5159 @cindex NSE Subsystem
5161 @section NSE Subsystem
5164 NSE stands for @dfn{Network Size Estimation}. The NSE subsystem provides
5165 other subsystems and users with a rough estimate of the number of peers
5166 currently participating in the GNUnet overlay.
5167 The computed value is not a precise number as producing a precise number
5168 in a decentralized, efficient and secure way is impossible.
5169 While NSE's estimate is inherently imprecise, NSE also gives the expected
5170 range. For a peer that has been running in a stable network for a
5171 while, the real network size will typically (99.7% of the time) be in the
5172 range of [2/3 estimate, 3/2 estimate]. We will now give an overview of the
5173 algorithm used to calculate the estimate;
5174 all of the details can be found in this technical report.
5176 @c FIXME: link to the report.
5182 * The NSE Client-Service Protocol::
5183 * The NSE Peer-to-Peer Protocol::
5187 @subsection Motivation
5190 Some subsystems, like DHT, need to know the size of the GNUnet network to
5191 optimize some parameters of their own protocol. The decentralized nature
5192 of GNUnet makes efficient and securely counting the exact number of peers
5193 infeasible. Although there are several decentralized algorithms to count
5194 the number of peers in a system, so far there is none to do so securely.
5195 Other protocols may allow any malicious peer to manipulate the final
5196 result or to take advantage of the system to perform
5197 @dfn{Denial of Service} (DoS) attacks against the network.
5198 GNUnet's NSE protocol avoids these drawbacks.
5206 @cindex NSE security
5207 @cindex nse security
5209 @subsubsection Security
5212 The NSE subsystem is designed to be resilient against these attacks.
5213 It uses @uref{http://en.wikipedia.org/wiki/Proof-of-work_system, proofs of work}
5214 to prevent one peer from impersonating a large number of participants,
5215 which would otherwise allow an adversary to artificially inflate the
5217 The DoS protection comes from the time-based nature of the protocol:
5218 the estimates are calculated periodically and out-of-time traffic is
5219 either ignored or stored for later retransmission by benign peers.
5220 In particular, peers cannot trigger global network communication at will.
5222 @cindex NSE principle
5223 @cindex nse principle
5225 @subsection Principle
5228 The algorithm calculates the estimate by finding the globally closest
5229 peer ID to a random, time-based value.
5231 The idea is that the closer the ID is to the random value, the more
5232 "densely packed" the ID space is, and therefore, more peers are in the
5242 * Controlled Flooding::
5243 * Calculating the estimate::
5247 @subsubsection Example
5250 Suppose all peers have IDs between 0 and 100 (our ID space), and the
5252 If the closest peer has the ID 70 we can imagine that the average
5253 "distance" between peers is around 30 and therefore the are around 3
5254 peers in the whole ID space. On the other hand, if the closest peer has
5255 the ID 44, we can imagine that the space is rather packed with peers,
5256 maybe as much as 50 of them.
5257 Naturally, we could have been rather unlucky, and there is only one peer
5258 and happens to have the ID 44. Thus, the current estimate is calculated
5259 as the average over multiple rounds, and not just a single sample.
5262 @subsubsection Algorithm
5265 Given that example, one can imagine that the job of the subsystem is to
5266 efficiently communicate the ID of the closest peer to the target value
5267 to all the other peers, who will calculate the estimate from it.
5270 @subsubsection Target value
5274 The target value itself is generated by hashing the current time, rounded
5275 down to an agreed value. If the rounding amount is 1h (default) and the
5276 time is 12:34:56, the time to hash would be 12:00:00. The process is
5277 repeated each rounding amount (in this example would be every hour).
5278 Every repetition is called a round.
5281 @subsubsection Timing
5284 The NSE subsystem has some timing control to avoid everybody broadcasting
5285 its ID all at one. Once each peer has the target random value, it
5286 compares its own ID to the target and calculates the hypothetical size of
5287 the network if that peer were to be the closest.
5288 Then it compares the hypothetical size with the estimate from the previous
5289 rounds. For each value there is an associated point in the period,
5290 let's call it "broadcast time". If its own hypothetical estimate
5291 is the same as the previous global estimate, its "broadcast time" will be
5292 in the middle of the round. If its bigger it will be earlier and if its
5293 smaller (the most likely case) it will be later. This ensures that the
5294 peers closest to the target value start broadcasting their ID the first.
5296 @node Controlled Flooding
5297 @subsubsection Controlled Flooding
5301 When a peer receives a value, first it verifies that it is closer than the
5302 closest value it had so far, otherwise it answers the incoming message
5303 with a message containing the better value. Then it checks a proof of
5304 work that must be included in the incoming message, to ensure that the
5305 other peer's ID is not made up (otherwise a malicious peer could claim to
5306 have an ID of exactly the target value every round). Once validated, it
5307 compares the broadcast time of the received value with the current time
5308 and if it's not too early, sends the received value to its neighbors.
5309 Otherwise it stores the value until the correct broadcast time comes.
5310 This prevents unnecessary traffic of sub-optimal values, since a better
5311 value can come before the broadcast time, rendering the previous one
5312 obsolete and saving the traffic that would have been used to broadcast it
5315 @node Calculating the estimate
5316 @subsubsection Calculating the estimate
5320 Once the closest ID has been spread across the network each peer gets the
5321 exact distance between this ID and the target value of the round and
5322 calculates the estimate with a mathematical formula described in the tech
5323 report. The estimate generated with this method for a single round is not
5324 very precise. Remember the case of the example, where the only peer is the
5325 ID 44 and we happen to generate the target value 42, thinking there are
5326 50 peers in the network. Therefore, the NSE subsystem remembers the last
5327 64 estimates and calculates an average over them, giving a result of which
5328 usually has one bit of uncertainty (the real size could be half of the
5329 estimate or twice as much). Note that the actual network size is
5330 calculated in powers of two of the raw input, thus one bit of uncertainty
5331 means a factor of two in the size estimate.
5333 @cindex libgnunetnse
5335 @subsection libgnunetnse
5339 The NSE subsystem has the simplest API of all services, with only two
5340 calls: @code{GNUNET_NSE_connect} and @code{GNUNET_NSE_disconnect}.
5342 The connect call gets a callback function as a parameter and this function
5343 is called each time the network agrees on an estimate. This usually is
5344 once per round, with some exceptions: if the closest peer has a late
5345 local clock and starts spreading its ID after everyone else agreed on a
5346 value, the callback might be activated twice in a round, the second value
5347 being always bigger than the first. The default round time is set to
5350 The disconnect call disconnects from the NSE subsystem and the callback
5351 is no longer called with new estimates.
5357 * libgnunetnse - Examples::
5361 @subsubsection Results
5365 The callback provides two values: the average and the
5366 @uref{http://en.wikipedia.org/wiki/Standard_deviation, standard deviation}
5367 of the last 64 rounds. The values provided by the callback function are
5368 logarithmic, this means that the real estimate numbers can be obtained by
5369 calculating 2 to the power of the given value (2average). From a
5370 statistics point of view this means that:
5373 @item 68% of the time the real size is included in the interval
5374 [(2average-stddev), 2]
5375 @item 95% of the time the real size is included in the interval
5376 [(2average-2*stddev, 2^average+2*stddev]
5377 @item 99.7% of the time the real size is included in the interval
5378 [(2average-3*stddev, 2average+3*stddev]
5381 The expected standard variation for 64 rounds in a network of stable size
5382 is 0.2. Thus, we can say that normally:
5385 @item 68% of the time the real size is in the range [-13%, +15%]
5386 @item 95% of the time the real size is in the range [-24%, +32%]
5387 @item 99.7% of the time the real size is in the range [-34%, +52%]
5390 As said in the introduction, we can be quite sure that usually the real
5391 size is between one third and three times the estimate. This can of
5392 course vary with network conditions.
5393 Thus, applications may want to also consider the provided standard
5394 deviation value, not only the average (in particular, if the standard
5395 variation is very high, the average maybe meaningless: the network size is
5398 @node libgnunetnse - Examples
5399 @subsubsection libgnunetnse -Examples
5403 Let's close with a couple examples.
5407 @item Average: 10, std dev: 1 Here the estimate would be
5408 2^10 = 1024 peers. (The range in which we can be 95% sure is:
5409 [2^8, 2^12] = [256, 4096]. We can be very (>99.7%) sure that the network
5410 is not a hundred peers and absolutely sure that it is not a million peers,
5411 but somewhere around a thousand.)
5413 @item Average 22, std dev: 0.2 Here the estimate would be
5414 2^22 = 4 Million peers. (The range in which we can be 99.7% sure
5415 is: [2^21.4, 2^22.6] = [2.8M, 6.3M]. We can be sure that the network size
5416 is around four million, with absolutely way of it being 1 million.)
5420 To put this in perspective, if someone remembers the LHC Higgs boson
5421 results, were announced with "5 sigma" and "6 sigma" certainties. In this
5422 case a 5 sigma minimum would be 2 million and a 6 sigma minimum,
5425 @node The NSE Client-Service Protocol
5426 @subsection The NSE Client-Service Protocol
5430 As with the API, the client-service protocol is very simple, only has 2
5431 different messages, defined in @code{src/nse/nse.h}:
5434 @item @code{GNUNET_MESSAGE_TYPE_NSE_START}@ This message has no parameters
5435 and is sent from the client to the service upon connection.
5436 @item @code{GNUNET_MESSAGE_TYPE_NSE_ESTIMATE}@ This message is sent from
5437 the service to the client for every new estimate and upon connection.
5438 Contains a timestamp for the estimate, the average and the standard
5439 deviation for the respective round.
5442 When the @code{GNUNET_NSE_disconnect} API call is executed, the client
5443 simply disconnects from the service, with no message involved.
5445 @cindex NSE Peer-to-Peer Protocol
5446 @node The NSE Peer-to-Peer Protocol
5447 @subsection The NSE Peer-to-Peer Protocol
5450 @pindex GNUNET_MESSAGE_TYPE_NSE_P2P_FLOOD
5451 The NSE subsystem only has one message in the P2P protocol, the
5452 @code{GNUNET_MESSAGE_TYPE_NSE_P2P_FLOOD} message.
5454 This message key contents are the timestamp to identify the round
5455 (differences in system clocks may cause some peers to send messages way
5456 too early or way too late, so the timestamp allows other peers to
5457 identify such messages easily), the
5458 @uref{http://en.wikipedia.org/wiki/Proof-of-work_system, proof of work}
5459 used to make it difficult to mount a
5460 @uref{http://en.wikipedia.org/wiki/Sybil_attack, Sybil attack}, and the
5461 public key, which is used to verify the signature on the message.
5463 Every peer stores a message for the previous, current and next round. The
5464 messages for the previous and current round are given to peers that
5465 connect to us. The message for the next round is simply stored until our
5466 system clock advances to the next round. The message for the current round
5467 is what we are flooding the network with right now.
5468 At the beginning of each round the peer does the following:
5471 @item calculates its own distance to the target value
5472 @item creates, signs and stores the message for the current round (unless
5473 it has a better message in the "next round" slot which came early in the
5475 @item calculates, based on the stored round message (own or received) when
5476 to start flooding it to its neighbors
5479 Upon receiving a message the peer checks the validity of the message
5480 (round, proof of work, signature). The next action depends on the
5481 contents of the incoming message:
5484 @item if the message is worse than the current stored message, the peer
5485 sends the current message back immediately, to stop the other peer from
5486 spreading suboptimal results
5487 @item if the message is better than the current stored message, the peer
5488 stores the new message and calculates the new target time to start
5489 spreading it to its neighbors (excluding the one the message came from)
5490 @item if the message is for the previous round, it is compared to the
5491 message stored in the "previous round slot", which may then be updated
5492 @item if the message is for the next round, it is compared to the message
5493 stored in the "next round slot", which again may then be updated
5496 Finally, when it comes to send the stored message for the current round to
5497 the neighbors there is a random delay added for each neighbor, to avoid
5498 traffic spikes and minimize cross-messages.
5500 @cindex HOSTLIST Subsystem
5501 @node HOSTLIST Subsystem
5502 @section HOSTLIST Subsystem
5506 Peers in the GNUnet overlay network need address information so that they
5507 can connect with other peers. GNUnet uses so called HELLO messages to
5508 store and exchange peer addresses.
5509 GNUnet provides several methods for peers to obtain this information:
5512 @item out-of-band exchange of HELLO messages (manually, using for example
5514 @item HELLO messages shipped with GNUnet (automatic with distribution)
5515 @item UDP neighbor discovery in LAN (IPv4 broadcast, IPv6 multicast)
5516 @item topology gossiping (learning from other peers we already connected
5518 @item the HOSTLIST daemon covered in this section, which is particularly
5519 relevant for bootstrapping new peers.
5522 New peers have no existing connections (and thus cannot learn from gossip
5523 among peers), may not have other peers in their LAN and might be started
5524 with an outdated set of HELLO messages from the distribution.
5525 In this case, getting new peers to connect to the network requires either
5526 manual effort or the use of a HOSTLIST to obtain HELLOs.
5530 * Overview for the HOSTLIST subsystem::
5531 * Interacting with the HOSTLIST daemon::
5532 * Hostlist security address validation::
5533 * The HOSTLIST daemon::
5534 * The HOSTLIST server::
5535 * The HOSTLIST client::
5544 The basic information peers require to connect to other peers are
5545 contained in so called HELLO messages you can think of as a business card.
5546 Besides the identity of the peer (based on the cryptographic public key) a
5547 HELLO message may contain address information that specifies ways to
5548 contact a peer. By obtaining HELLO messages, a peer can learn how to
5549 contact other peers.
5551 @node Overview for the HOSTLIST subsystem
5552 @subsection Overview for the HOSTLIST subsystem
5556 The HOSTLIST subsystem provides a way to distribute and obtain contact
5557 information to connect to other peers using a simple HTTP GET request.
5558 It's implementation is split in three parts, the main file for the daemon
5559 itself (@file{gnunet-daemon-hostlist.c}), the HTTP client used to download
5560 peer information (@file{hostlist-client.c}) and the server component used
5561 to provide this information to other peers (@file{hostlist-server.c}).
5562 The server is basically a small HTTP web server (based on GNU
5563 libmicrohttpd) which provides a list of HELLOs known to the local peer for
5564 download. The client component is basically a HTTP client
5565 (based on libcurl) which can download hostlists from one or more websites.
5566 The hostlist format is a binary blob containing a sequence of HELLO
5567 messages. Note that any HTTP server can theoretically serve a hostlist,
5568 the build-in hostlist server makes it simply convenient to offer this
5574 * HOSTLIST - Limitations::
5578 @subsubsection Features
5582 The HOSTLIST daemon can:
5585 @item provide HELLO messages with validated addresses obtained from
5586 PEERINFO to download for other peers
5587 @item download HELLO messages and forward these message to the TRANSPORT
5588 subsystem for validation
5589 @item advertises the URL of this peer's hostlist address to other peers
5591 @item automatically learn about hostlist servers from the gossip of other
5595 @node HOSTLIST - Limitations
5596 @subsubsection HOSTLIST - Limitations
5600 The HOSTLIST daemon does not:
5603 @item verify the cryptographic information in the HELLO messages
5604 @item verify the address information in the HELLO messages
5607 @node Interacting with the HOSTLIST daemon
5608 @subsection Interacting with the HOSTLIST daemon
5612 The HOSTLIST subsystem is currently implemented as a daemon, so there is
5613 no need for the user to interact with it and therefore there is no
5614 command line tool and no API to communicate with the daemon. In the
5615 future, we can envision changing this to allow users to manually trigger
5616 the download of a hostlist.
5618 Since there is no command line interface to interact with HOSTLIST, the
5619 only way to interact with the hostlist is to use STATISTICS to obtain or
5620 modify information about the status of HOSTLIST:
5623 $ gnunet-statistics -s hostlist
5627 In particular, HOSTLIST includes a @strong{persistent} value in statistics
5628 that specifies when the hostlist server might be queried next. As this
5629 value is exponentially increasing during runtime, developers may want to
5630 reset or manually adjust it. Note that HOSTLIST (but not STATISTICS) needs
5631 to be shutdown if changes to this value are to have any effect on the
5632 daemon (as HOSTLIST does not monitor STATISTICS for changes to the
5633 download frequency).
5635 @node Hostlist security address validation
5636 @subsection Hostlist security address validation
5640 Since information obtained from other parties cannot be trusted without
5641 validation, we have to distinguish between @emph{validated} and
5642 @emph{not validated} addresses. Before using (and so trusting)
5643 information from other parties, this information has to be double-checked
5644 (validated). Address validation is not done by HOSTLIST but by the
5647 The HOSTLIST component is functionally located between the PEERINFO and
5648 the TRANSPORT subsystem. When acting as a server, the daemon obtains valid
5649 (@emph{validated}) peer information (HELLO messages) from the PEERINFO
5650 service and provides it to other peers. When acting as a client, it
5651 contacts the HOSTLIST servers specified in the configuration, downloads
5652 the (unvalidated) list of HELLO messages and forwards these information
5653 to the TRANSPORT server to validate the addresses.
5655 @cindex HOSTLIST daemon
5656 @node The HOSTLIST daemon
5657 @subsection The HOSTLIST daemon
5661 The hostlist daemon is the main component of the HOSTLIST subsystem. It is
5662 started by the ARM service and (if configured) starts the HOSTLIST client
5663 and server components.
5665 @pindex GNUNET_MESSAGE_TYPE_HOSTLIST_ADVERTISEMENT
5666 If the daemon provides a hostlist itself it can advertise it's own
5667 hostlist to other peers. To do so it sends a
5668 @code{GNUNET_MESSAGE_TYPE_HOSTLIST_ADVERTISEMENT} message to other peers
5669 when they connect to this peer on the CORE level. This hostlist
5670 advertisement message contains the URL to access the HOSTLIST HTTP
5671 server of the sender. The daemon may also subscribe to this type of
5672 message from CORE service, and then forward these kind of message to the
5673 HOSTLIST client. The client then uses all available URLs to download peer
5674 information when necessary.
5676 When starting, the HOSTLIST daemon first connects to the CORE subsystem
5677 and if hostlist learning is enabled, registers a CORE handler to receive
5678 this kind of messages. Next it starts (if configured) the client and
5679 server. It passes pointers to CORE connect and disconnect and receive
5680 handlers where the client and server store their functions, so the daemon
5681 can notify them about CORE events.
5683 To clean up on shutdown, the daemon has a cleaning task, shutting down all
5684 subsystems and disconnecting from CORE.
5686 @cindex HOSTLIST server
5687 @node The HOSTLIST server
5688 @subsection The HOSTLIST server
5692 The server provides a way for other peers to obtain HELLOs. Basically it
5693 is a small web server other peers can connect to and download a list of
5694 HELLOs using standard HTTP; it may also advertise the URL of the hostlist
5695 to other peers connecting on CORE level.
5700 * Advertising the URL::
5703 @node The HTTP Server
5704 @subsubsection The HTTP Server
5708 During startup, the server starts a web server listening on the port
5709 specified with the HTTPPORT value (default 8080). In addition it connects
5710 to the PEERINFO service to obtain peer information. The HOSTLIST server
5711 uses the GNUNET_PEERINFO_iterate function to request HELLO information for
5712 all peers and adds their information to a new hostlist if they are
5713 suitable (expired addresses and HELLOs without addresses are both not
5714 suitable) and the maximum size for a hostlist is not exceeded
5715 (MAX_BYTES_PER_HOSTLISTS = 500000).
5716 When PEERINFO finishes (with a last NULL callback), the server destroys
5717 the previous hostlist response available for download on the web server
5718 and replaces it with the updated hostlist. The hostlist format is
5719 basically a sequence of HELLO messages (as obtained from PEERINFO) without
5720 any special tokenization. Since each HELLO message contains a size field,
5721 the response can easily be split into separate HELLO messages by the
5724 A HOSTLIST client connecting to the HOSTLIST server will receive the
5725 hostlist as a HTTP response and the the server will terminate the
5726 connection with the result code @code{HTTP 200 OK}.
5727 The connection will be closed immediately if no hostlist is available.
5729 @node Advertising the URL
5730 @subsubsection Advertising the URL
5734 The server also advertises the URL to download the hostlist to other peers
5735 if hostlist advertisement is enabled.
5736 When a new peer connects and has hostlist learning enabled, the server
5737 sends a @code{GNUNET_MESSAGE_TYPE_HOSTLIST_ADVERTISEMENT} message to this
5738 peer using the CORE service.
5740 @cindex HOSTLIST client
5741 @node The HOSTLIST client
5742 @subsection The HOSTLIST client
5746 The client provides the functionality to download the list of HELLOs from
5748 It performs a standard HTTP request to the URLs configured and learned
5749 from advertisement messages received from other peers. When a HELLO is
5750 downloaded, the HOSTLIST client forwards the HELLO to the TRANSPORT
5751 service for validation.
5753 The client supports two modes of operation:
5756 @item download of HELLOs (bootstrapping)
5757 @item learning of URLs
5766 @subsubsection Bootstrapping
5770 For bootstrapping, it schedules a task to download the hostlist from the
5772 The downloads are only performed if the number of current
5773 connections is smaller than a minimum number of connections
5775 The interval between downloads increases exponentially; however, the
5776 exponential growth is limited if it becomes longer than an hour.
5777 At that point, the frequency growth is capped at
5778 (#number of connections * 1h).
5780 Once the decision has been taken to download HELLOs, the daemon chooses a
5781 random URL from the list of known URLs. URLs can be configured in the
5782 configuration or be learned from advertisement messages.
5783 The client uses a HTTP client library (libcurl) to initiate the download
5784 using the libcurl multi interface.
5785 Libcurl passes the data to the callback_download function which
5786 stores the data in a buffer if space is available and the maximum size for
5787 a hostlist download is not exceeded (MAX_BYTES_PER_HOSTLISTS = 500000).
5788 When a full HELLO was downloaded, the HOSTLIST client offers this
5789 HELLO message to the TRANSPORT service for validation.
5790 When the download is finished or failed, statistical information about the
5791 quality of this URL is updated.
5793 @cindex HOSTLIST learning
5795 @subsubsection Learning
5799 The client also manages hostlist advertisements from other peers. The
5800 HOSTLIST daemon forwards @code{GNUNET_MESSAGE_TYPE_HOSTLIST_ADVERTISEMENT}
5801 messages to the client subsystem, which extracts the URL from the message.
5802 Next, a test of the newly obtained URL is performed by triggering a
5803 download from the new URL. If the URL works correctly, it is added to the
5804 list of working URLs.
5806 The size of the list of URLs is restricted, so if an additional server is
5807 added and the list is full, the URL with the worst quality ranking
5808 (determined through successful downloads and number of HELLOs e.g.) is
5809 discarded. During shutdown the list of URLs is saved to a file for
5810 persistance and loaded on startup. URLs from the configuration file are
5818 To start HOSTLIST by default, it has to be added to the DEFAULTSERVICES
5819 section for the ARM services. This is done in the default configuration.
5821 For more information on how to configure the HOSTLIST subsystem see the
5822 installation handbook:@
5823 Configuring the hostlist to bootstrap@
5824 Configuring your peer to provide a hostlist
5826 @cindex IDENTITY Subsystem
5827 @node IDENTITY Subsystem
5828 @section IDENTITY Subsystem
5832 Identities of "users" in GNUnet are called egos.
5833 Egos can be used as pseudonyms ("fake names") or be tied to an
5834 organization (for example, "GNU") or even the actual identity of a human.
5835 GNUnet users are expected to have many egos. They might have one tied to
5836 their real identity, some for organizations they manage, and more for
5837 different domains where they want to operate under a pseudonym.
5839 The IDENTITY service allows users to manage their egos. The identity
5840 service manages the private keys egos of the local user; it does not
5841 manage identities of other users (public keys). Public keys for other
5842 users need names to become manageable. GNUnet uses the
5843 @dfn{GNU Name System} (GNS) to give names to other users and manage their
5844 public keys securely. This chapter is about the IDENTITY service,
5845 which is about the management of private keys.
5847 On the network, an ego corresponds to an ECDSA key (over Curve25519,
5848 using RFC 6979, as required by GNS). Thus, users can perform actions
5849 under a particular ego by using (signing with) a particular private key.
5850 Other users can then confirm that the action was really performed by that
5851 ego by checking the signature against the respective public key.
5853 The IDENTITY service allows users to associate a human-readable name with
5854 each ego. This way, users can use names that will remind them of the
5855 purpose of a particular ego.
5856 The IDENTITY service will store the respective private keys and
5857 allows applications to access key information by name.
5858 Users can change the name that is locally (!) associated with an ego.
5859 Egos can also be deleted, which means that the private key will be removed
5860 and it thus will not be possible to perform actions with that ego in the
5863 Additionally, the IDENTITY subsystem can associate service functions with
5865 For example, GNS requires the ego that should be used for the shorten
5866 zone. GNS will ask IDENTITY for an ego for the "gns-short" service.
5867 The IDENTITY service has a mapping of such service strings to the name of
5868 the ego that the user wants to use for this service, for example
5869 "my-short-zone-ego".
5871 Finally, the IDENTITY API provides access to a special ego, the
5872 anonymous ego. The anonymous ego is special in that its private key is not
5873 really private, but fixed and known to everyone.
5874 Thus, anyone can perform actions as anonymous. This can be useful as with
5875 this trick, code does not have to contain a special case to distinguish
5876 between anonymous and pseudonymous egos.
5879 * libgnunetidentity::
5880 * The IDENTITY Client-Service Protocol::
5883 @cindex libgnunetidentity
5884 @node libgnunetidentity
5885 @subsection libgnunetidentity
5890 * Connecting to the service::
5891 * Operations on Egos::
5892 * The anonymous Ego::
5893 * Convenience API to lookup a single ego::
5894 * Associating egos with service functions::
5897 @node Connecting to the service
5898 @subsubsection Connecting to the service
5902 First, typical clients connect to the identity service using
5903 @code{GNUNET_IDENTITY_connect}. This function takes a callback as a
5905 If the given callback parameter is non-null, it will be invoked to notify
5906 the application about the current state of the identities in the system.
5909 @item First, it will be invoked on all known egos at the time of the
5910 connection. For each ego, a handle to the ego and the user's name for the
5911 ego will be passed to the callback. Furthermore, a @code{void **} context
5912 argument will be provided which gives the client the opportunity to
5913 associate some state with the ego.
5914 @item Second, the callback will be invoked with NULL for the ego, the name
5915 and the context. This signals that the (initial) iteration over all egos
5917 @item Then, the callback will be invoked whenever something changes about
5919 If an ego is renamed, the callback is invoked with the ego handle of the
5920 ego that was renamed, and the new name. If an ego is deleted, the callback
5921 is invoked with the ego handle and a name of NULL. In the deletion case,
5922 the application should also release resources stored in the context.
5923 @item When the application destroys the connection to the identity service
5924 using @code{GNUNET_IDENTITY_disconnect}, the callback is again invoked
5925 with the ego and a name of NULL (equivalent to deletion of the egos).
5926 This should again be used to clean up the per-ego context.
5929 The ego handle passed to the callback remains valid until the callback is
5930 invoked with a name of NULL, so it is safe to store a reference to the
5933 @node Operations on Egos
5934 @subsubsection Operations on Egos
5938 Given an ego handle, the main operations are to get its associated private
5939 key using @code{GNUNET_IDENTITY_ego_get_private_key} or its associated
5940 public key using @code{GNUNET_IDENTITY_ego_get_public_key}.
5942 The other operations on egos are pretty straightforward.
5943 Using @code{GNUNET_IDENTITY_create}, an application can request the
5944 creation of an ego by specifying the desired name.
5945 The operation will fail if that name is
5946 already in use. Using @code{GNUNET_IDENTITY_rename} the name of an
5947 existing ego can be changed. Finally, egos can be deleted using
5948 @code{GNUNET_IDENTITY_delete}. All of these operations will trigger
5949 updates to the callback given to the @code{GNUNET_IDENTITY_connect}
5950 function of all applications that are connected with the identity service
5951 at the time. @code{GNUNET_IDENTITY_cancel} can be used to cancel the
5952 operations before the respective continuations would be called.
5953 It is not guaranteed that the operation will not be completed anyway,
5954 only the continuation will no longer be called.
5956 @node The anonymous Ego
5957 @subsubsection The anonymous Ego
5961 A special way to obtain an ego handle is to call
5962 @code{GNUNET_IDENTITY_ego_get_anonymous}, which returns an ego for the
5963 "anonymous" user --- anyone knows and can get the private key for this
5964 user, so it is suitable for operations that are supposed to be anonymous
5965 but require signatures (for example, to avoid a special path in the code).
5966 The anonymous ego is always valid and accessing it does not require a
5967 connection to the identity service.
5969 @node Convenience API to lookup a single ego
5970 @subsubsection Convenience API to lookup a single ego
5973 As applications commonly simply have to lookup a single ego, there is a
5974 convenience API to do just that. Use @code{GNUNET_IDENTITY_ego_lookup} to
5975 lookup a single ego by name. Note that this is the user's name for the
5976 ego, not the service function. The resulting ego will be returned via a
5977 callback and will only be valid during that callback. The operation can
5978 be canceled via @code{GNUNET_IDENTITY_ego_lookup_cancel}
5979 (cancellation is only legal before the callback is invoked).
5981 @node Associating egos with service functions
5982 @subsubsection Associating egos with service functions
5985 The @code{GNUNET_IDENTITY_set} function is used to associate a particular
5986 ego with a service function. The name used by the service and the ego are
5988 Afterwards, the service can use its name to lookup the associated ego
5989 using @code{GNUNET_IDENTITY_get}.
5991 @node The IDENTITY Client-Service Protocol
5992 @subsection The IDENTITY Client-Service Protocol
5996 A client connecting to the identity service first sends a message with
5998 @code{GNUNET_MESSAGE_TYPE_IDENTITY_START} to the service. After that, the
5999 client will receive information about changes to the egos by receiving
6000 messages of type @code{GNUNET_MESSAGE_TYPE_IDENTITY_UPDATE}.
6001 Those messages contain the private key of the ego and the user's name of
6002 the ego (or zero bytes for the name to indicate that the ego was deleted).
6003 A special bit @code{end_of_list} is used to indicate the end of the
6004 initial iteration over the identity service's egos.
6006 The client can trigger changes to the egos by sending @code{CREATE},
6007 @code{RENAME} or @code{DELETE} messages.
6008 The CREATE message contains the private key and the desired name.@
6009 The RENAME message contains the old name and the new name.@
6010 The DELETE message only needs to include the name of the ego to delete.@
6011 The service responds to each of these messages with a @code{RESULT_CODE}
6012 message which indicates success or error of the operation, and possibly
6013 a human-readable error message.
6015 Finally, the client can bind the name of a service function to an ego by
6016 sending a @code{SET_DEFAULT} message with the name of the service function
6017 and the private key of the ego.
6018 Such bindings can then be resolved using a @code{GET_DEFAULT} message,
6019 which includes the name of the service function. The identity service
6020 will respond to a GET_DEFAULT request with a SET_DEFAULT message
6021 containing the respective information, or with a RESULT_CODE to
6024 @cindex NAMESTORE Subsystem
6025 @node NAMESTORE Subsystem
6026 @section NAMESTORE Subsystem
6028 The NAMESTORE subsystem provides persistent storage for local GNS zone
6029 information. All local GNS zone information are managed by NAMESTORE. It
6030 provides both the functionality to administer local GNS information (e.g.
6031 delete and add records) as well as to retrieve GNS information (e.g to
6032 list name information in a client).
6033 NAMESTORE does only manage the persistent storage of zone information
6034 belonging to the user running the service: GNS information from other
6035 users obtained from the DHT are stored by the NAMECACHE subsystem.
6037 NAMESTORE uses a plugin-based database backend to store GNS information
6038 with good performance. Here sqlite, MySQL and PostgreSQL are supported
6040 NAMESTORE clients interact with the IDENTITY subsystem to obtain
6041 cryptographic information about zones based on egos as described with the
6042 IDENTITY subsystem, but internally NAMESTORE refers to zones using the
6044 In addition, it collaborates with the NAMECACHE subsystem and
6045 stores zone information when local information are modified in the
6046 GNS cache to increase look-up performance for local information.
6048 NAMESTORE provides functionality to look-up and store records, to iterate
6049 over a specific or all zones and to monitor zones for changes. NAMESTORE
6050 functionality can be accessed using the NAMESTORE api or the NAMESTORE
6054 * libgnunetnamestore::
6057 @cindex libgnunetnamestore
6058 @node libgnunetnamestore
6059 @subsection libgnunetnamestore
6061 To interact with NAMESTORE clients first connect to the NAMESTORE service
6062 using the @code{GNUNET_NAMESTORE_connect} passing a configuration handle.
6063 As a result they obtain a NAMESTORE handle, they can use for operations,
6064 or NULL is returned if the connection failed.
6066 To disconnect from NAMESTORE, clients use
6067 @code{GNUNET_NAMESTORE_disconnect} and specify the handle to disconnect.
6069 NAMESTORE internally uses the ECDSA private key to refer to zones. These
6070 private keys can be obtained from the IDENTITY subsytem.
6071 Here @emph{egos} @emph{can be used to refer to zones or the default ego
6072 assigned to the GNS subsystem can be used to obtained the master zone's
6077 * Editing Zone Information::
6078 * Iterating Zone Information::
6079 * Monitoring Zone Information::
6082 @node Editing Zone Information
6083 @subsubsection Editing Zone Information
6087 NAMESTORE provides functions to lookup records stored under a label in a
6088 zone and to store records under a label in a zone.
6090 To store (and delete) records, the client uses the
6091 @code{GNUNET_NAMESTORE_records_store} function and has to provide
6092 namestore handle to use, the private key of the zone, the label to store
6093 the records under, the records and number of records plus an callback
6095 After the operation is performed NAMESTORE will call the provided
6096 callback function with the result GNUNET_SYSERR on failure
6097 (including timeout/queue drop/failure to validate), GNUNET_NO if content
6098 was already there or not found GNUNET_YES (or other positive value) on
6099 success plus an additional error message.
6101 Records are deleted by using the store command with 0 records to store.
6102 It is important to note, that records are not merged when records exist
6104 So a client has first to retrieve records, merge with existing records
6105 and then store the result.
6107 To perform a lookup operation, the client uses the
6108 @code{GNUNET_NAMESTORE_records_store} function. Here it has to pass the
6109 namestore handle, the private key of the zone and the label. It also has
6110 to provide a callback function which will be called with the result of
6111 the lookup operation:
6112 the zone for the records, the label, and the records including the
6113 number of records included.
6115 A special operation is used to set the preferred nickname for a zone.
6116 This nickname is stored with the zone and is automatically merged with
6117 all labels and records stored in a zone. Here the client uses the
6118 @code{GNUNET_NAMESTORE_set_nick} function and passes the private key of
6119 the zone, the nickname as string plus a the callback with the result of
6122 @node Iterating Zone Information
6123 @subsubsection Iterating Zone Information
6127 A client can iterate over all information in a zone or all zones managed
6129 Here a client uses the @code{GNUNET_NAMESTORE_zone_iteration_start}
6130 function and passes the namestore handle, the zone to iterate over and a
6131 callback function to call with the result.
6132 If the client wants to iterate over all the WHAT!? FIXME, it passes NULL for the zone.
6133 A @code{GNUNET_NAMESTORE_ZoneIterator} handle is returned to be used to
6136 NAMESTORE calls the callback for every result and expects the client to
6137 call @code{GNUNET_NAMESTORE_zone_iterator_next} to continue to iterate or
6138 @code{GNUNET_NAMESTORE_zone_iterator_stop} to interrupt the iteration.
6139 When NAMESTORE reached the last item it will call the callback with a
6140 NULL value to indicate.
6142 @node Monitoring Zone Information
6143 @subsubsection Monitoring Zone Information
6147 Clients can also monitor zones to be notified about changes. Here the
6148 clients uses the @code{GNUNET_NAMESTORE_zone_monitor_start} function and
6149 passes the private key of the zone and and a callback function to call
6150 with updates for a zone.
6151 The client can specify to obtain zone information first by iterating over
6152 the zone and specify a synchronization callback to be called when the
6153 client and the namestore are synced.
6155 On an update, NAMESTORE will call the callback with the private key of the
6156 zone, the label and the records and their number.
6158 To stop monitoring, the client calls
6159 @code{GNUNET_NAMESTORE_zone_monitor_stop} and passes the handle obtained
6160 from the function to start the monitoring.
6162 @cindex PEERINFO Subsystem
6163 @node PEERINFO Subsystem
6164 @section PEERINFO Subsystem
6168 The PEERINFO subsystem is used to store verified (validated) information
6169 about known peers in a persistent way. It obtains these addresses for
6170 example from TRANSPORT service which is in charge of address validation.
6171 Validation means that the information in the HELLO message are checked by
6172 connecting to the addresses and performing a cryptographic handshake to
6173 authenticate the peer instance stating to be reachable with these
6175 Peerinfo does not validate the HELLO messages itself but only stores them
6176 and gives them to interested clients.
6178 As future work, we think about moving from storing just HELLO messages to
6179 providing a generic persistent per-peer information store.
6180 More and more subsystems tend to need to store per-peer information in
6182 To not duplicate this functionality we plan to provide a PEERSTORE
6183 service providing this functionality.
6186 * PEERINFO - Features::
6187 * PEERINFO - Limitations::
6188 * DeveloperPeer Information::
6190 * Managing Information::
6191 * Obtaining Information::
6192 * The PEERINFO Client-Service Protocol::
6193 * libgnunetpeerinfo::
6196 @node PEERINFO - Features
6197 @subsection PEERINFO - Features
6202 @item Persistent storage
6203 @item Client notification mechanism on update
6204 @item Periodic clean up for expired information
6205 @item Differentiation between public and friend-only HELLO
6208 @node PEERINFO - Limitations
6209 @subsection PEERINFO - Limitations
6213 @item Does not perform HELLO validation
6216 @node DeveloperPeer Information
6217 @subsection DeveloperPeer Information
6221 The PEERINFO subsystem stores these information in the form of HELLO
6222 messages you can think of as business cards.
6223 These HELLO messages contain the public key of a peer and the addresses
6224 a peer can be reached under.
6225 The addresses include an expiration date describing how long they are
6226 valid. This information is updated regularly by the TRANSPORT service by
6227 revalidating the address.
6228 If an address is expired and not renewed, it can be removed from the
6231 Some peer do not want to have their HELLO messages distributed to other
6232 peers, especially when GNUnet's friend-to-friend modus is enabled.
6233 To prevent this undesired distribution. PEERINFO distinguishes between
6234 @emph{public} and @emph{friend-only} HELLO messages.
6235 Public HELLO messages can be freely distributed to other (possibly
6236 unknown) peers (for example using the hostlist, gossiping, broadcasting),
6237 whereas friend-only HELLO messages may not be distributed to other peers.
6238 Friend-only HELLO messages have an additional flag @code{friend_only} set
6239 internally. For public HELLO message this flag is not set.
6240 PEERINFO does and cannot not check if a client is allowed to obtain a
6241 specific HELLO type.
6243 The HELLO messages can be managed using the GNUnet HELLO library.
6244 Other GNUnet systems can obtain these information from PEERINFO and use
6245 it for their purposes.
6246 Clients are for example the HOSTLIST component providing these
6247 information to other peers in form of a hostlist or the TRANSPORT
6248 subsystem using these information to maintain connections to other peers.
6255 During startup the PEERINFO services loads persistent HELLOs from disk.
6256 First PEERINFO parses the directory configured in the HOSTS value of the
6257 @code{PEERINFO} configuration section to store PEERINFO information.
6258 For all files found in this directory valid HELLO messages are extracted.
6259 In addition it loads HELLO messages shipped with the GNUnet distribution.
6260 These HELLOs are used to simplify network bootstrapping by providing
6261 valid peer information with the distribution.
6262 The use of these HELLOs can be prevented by setting the
6263 @code{USE_INCLUDED_HELLOS} in the @code{PEERINFO} configuration section to
6264 @code{NO}. Files containing invalid information are removed.
6266 @node Managing Information
6267 @subsection Managing Information
6271 The PEERINFO services stores information about known PEERS and a single
6272 HELLO message for every peer.
6273 A peer does not need to have a HELLO if no information are available.
6274 HELLO information from different sources, for example a HELLO obtained
6275 from a remote HOSTLIST and a second HELLO stored on disk, are combined
6276 and merged into one single HELLO message per peer which will be given to
6277 clients. During this merge process the HELLO is immediately written to
6278 disk to ensure persistence.
6280 PEERINFO in addition periodically scans the directory where information
6281 are stored for empty HELLO messages with expired TRANSPORT addresses.
6282 This periodic task scans all files in the directory and recreates the
6283 HELLO messages it finds.
6284 Expired TRANSPORT addresses are removed from the HELLO and if the
6285 HELLO does not contain any valid addresses, it is discarded and removed
6288 @node Obtaining Information
6289 @subsection Obtaining Information
6293 When a client requests information from PEERINFO, PEERINFO performs a
6294 lookup for the respective peer or all peers if desired and transmits this
6295 information to the client.
6296 The client can specify if friend-only HELLOs have to be included or not
6297 and PEERINFO filters the respective HELLO messages before transmitting
6300 To notify clients about changes to PEERINFO information, PEERINFO
6301 maintains a list of clients interested in this notifications.
6302 Such a notification occurs if a HELLO for a peer was updated (due to a
6303 merge for example) or a new peer was added.
6305 @node The PEERINFO Client-Service Protocol
6306 @subsection The PEERINFO Client-Service Protocol
6310 To connect and disconnect to and from the PEERINFO Service PEERINFO
6311 utilizes the util client/server infrastructure, so no special messages
6312 types are used here.
6314 To add information for a peer, the plain HELLO message is transmitted to
6315 the service without any wrapping. All pieces of information required are
6316 stored within the HELLO message.
6317 The PEERINFO service provides a message handler accepting and processing
6318 these HELLO messages.
6320 When obtaining PEERINFO information using the iterate functionality
6321 specific messages are used. To obtain information for all peers, a
6322 @code{struct ListAllPeersMessage} with message type
6323 @code{GNUNET_MESSAGE_TYPE_PEERINFO_GET_ALL} and a flag
6324 include_friend_only to indicate if friend-only HELLO messages should be
6325 included are transmitted. If information for a specific peer is required
6326 a @code{struct ListAllPeersMessage} with
6327 @code{GNUNET_MESSAGE_TYPE_PEERINFO_GET} containing the peer identity is
6330 For both variants the PEERINFO service replies for each HELLO message it
6331 wants to transmit with a @code{struct ListAllPeersMessage} with type
6332 @code{GNUNET_MESSAGE_TYPE_PEERINFO_INFO} containing the plain HELLO.
6333 The final message is @code{struct GNUNET_MessageHeader} with type
6334 @code{GNUNET_MESSAGE_TYPE_PEERINFO_INFO}. If the client receives this
6335 message, it can proceed with the next request if any is pending.
6337 @node libgnunetpeerinfo
6338 @subsection libgnunetpeerinfo
6342 The PEERINFO API consists mainly of three different functionalities:
6345 @item maintaining a connection to the service
6346 @item adding new information to the PEERINFO service
6347 @item retrieving information from the PEERINFO service
6351 * Connecting to the PEERINFO Service::
6352 * Adding Information to the PEERINFO Service::
6353 * Obtaining Information from the PEERINFO Service::
6356 @node Connecting to the PEERINFO Service
6357 @subsubsection Connecting to the PEERINFO Service
6361 To connect to the PEERINFO service the function
6362 @code{GNUNET_PEERINFO_connect} is used, taking a configuration handle as
6363 an argument, and to disconnect from PEERINFO the function
6364 @code{GNUNET_PEERINFO_disconnect}, taking the PEERINFO
6365 handle returned from the connect function has to be called.
6367 @node Adding Information to the PEERINFO Service
6368 @subsubsection Adding Information to the PEERINFO Service
6372 @code{GNUNET_PEERINFO_add_peer} adds a new peer to the PEERINFO subsystem
6373 storage. This function takes the PEERINFO handle as an argument, the HELLO
6374 message to store and a continuation with a closure to be called with the
6375 result of the operation.
6376 The @code{GNUNET_PEERINFO_add_peer} returns a handle to this operation
6377 allowing to cancel the operation with the respective cancel function
6378 @code{GNUNET_PEERINFO_add_peer_cancel}. To retrieve information from
6379 PEERINFO you can iterate over all information stored with PEERINFO or you
6380 can tell PEERINFO to notify if new peer information are available.
6382 @node Obtaining Information from the PEERINFO Service
6383 @subsubsection Obtaining Information from the PEERINFO Service
6387 To iterate over information in PEERINFO you use
6388 @code{GNUNET_PEERINFO_iterate}.
6389 This function expects the PEERINFO handle, a flag if HELLO messages
6390 intended for friend only mode should be included, a timeout how long the
6391 operation should take and a callback with a callback closure to be called
6393 If you want to obtain information for a specific peer, you can specify
6394 the peer identity, if this identity is NULL, information for all peers are
6395 returned. The function returns a handle to allow to cancel the operation
6396 using @code{GNUNET_PEERINFO_iterate_cancel}.
6398 To get notified when peer information changes, you can use
6399 @code{GNUNET_PEERINFO_notify}.
6400 This function expects a configuration handle and a flag if friend-only
6401 HELLO messages should be included. The PEERINFO service will notify you
6402 about every change and the callback function will be called to notify you
6403 about changes. The function returns a handle to cancel notifications
6404 with @code{GNUNET_PEERINFO_notify_cancel}.
6406 @cindex PEERSTORE Subsystem
6407 @node PEERSTORE Subsystem
6408 @section PEERSTORE Subsystem
6412 GNUnet's PEERSTORE subsystem offers persistent per-peer storage for other
6413 GNUnet subsystems. GNUnet subsystems can use PEERSTORE to persistently
6414 store and retrieve arbitrary data.
6415 Each data record stored with PEERSTORE contains the following fields:
6418 @item subsystem: Name of the subsystem responsible for the record.
6419 @item peerid: Identity of the peer this record is related to.
6420 @item key: a key string identifying the record.
6421 @item value: binary record value.
6422 @item expiry: record expiry date.
6428 * libgnunetpeerstore::
6432 @subsection Functionality
6436 Subsystems can store any type of value under a (subsystem, peerid, key)
6437 combination. A "replace" flag set during store operations forces the
6438 PEERSTORE to replace any old values stored under the same
6439 (subsystem, peerid, key) combination with the new value.
6440 Additionally, an expiry date is set after which the record is *possibly*
6441 deleted by PEERSTORE.
6443 Subsystems can iterate over all values stored under any of the following
6444 combination of fields:
6448 @item (subsystem, peerid)
6449 @item (subsystem, key)
6450 @item (subsystem, peerid, key)
6453 Subsystems can also request to be notified about any new values stored
6454 under a (subsystem, peerid, key) combination by sending a "watch"
6455 request to PEERSTORE.
6458 @subsection Architecture
6462 PEERSTORE implements the following components:
6465 @item PEERSTORE service: Handles store, iterate and watch operations.
6466 @item PEERSTORE API: API to be used by other subsystems to communicate and
6467 issue commands to the PEERSTORE service.
6468 @item PEERSTORE plugins: Handles the persistent storage. At the moment,
6469 only an "sqlite" plugin is implemented.
6472 @cindex libgnunetpeerstore
6473 @node libgnunetpeerstore
6474 @subsection libgnunetpeerstore
6478 libgnunetpeerstore is the library containing the PEERSTORE API. Subsystems
6479 wishing to communicate with the PEERSTORE service use this API to open a
6480 connection to PEERSTORE. This is done by calling
6481 @code{GNUNET_PEERSTORE_connect} which returns a handle to the newly
6483 This handle has to be used with any further calls to the API.
6485 To store a new record, the function @code{GNUNET_PEERSTORE_store} is to
6486 be used which requires the record fields and a continuation function that
6487 will be called by the API after the STORE request is sent to the
6489 Note that calling the continuation function does not mean that the record
6490 is successfully stored, only that the STORE request has been successfully
6491 sent to the PEERSTORE service.
6492 @code{GNUNET_PEERSTORE_store_cancel} can be called to cancel the STORE
6493 request only before the continuation function has been called.
6495 To iterate over stored records, the function
6496 @code{GNUNET_PEERSTORE_iterate} is
6497 to be used. @emph{peerid} and @emph{key} can be set to NULL. An iterator
6498 callback function will be called with each matching record found and a
6499 NULL record at the end to signal the end of result set.
6500 @code{GNUNET_PEERSTORE_iterate_cancel} can be used to cancel the ITERATE
6501 request before the iterator callback is called with a NULL record.
6503 To be notified with new values stored under a (subsystem, peerid, key)
6504 combination, the function @code{GNUNET_PEERSTORE_watch} is to be used.
6505 This will register the watcher with the PEERSTORE service, any new
6506 records matching the given combination will trigger the callback
6507 function passed to @code{GNUNET_PEERSTORE_watch}. This continues until
6508 @code{GNUNET_PEERSTORE_watch_cancel} is called or the connection to the
6509 service is destroyed.
6511 After the connection is no longer needed, the function
6512 @code{GNUNET_PEERSTORE_disconnect} can be called to disconnect from the
6514 Any pending ITERATE or WATCH requests will be destroyed.
6515 If the @code{sync_first} flag is set to @code{GNUNET_YES}, the API will
6516 delay the disconnection until all pending STORE requests are sent to
6517 the PEERSTORE service, otherwise, the pending STORE requests will be
6520 @cindex SET Subsystem
6522 @section SET Subsystem
6526 The SET service implements efficient set operations between two peers
6528 Currently, set union and set intersection are the only supported
6529 operations. Elements of a set consist of an @emph{element type} and
6530 arbitrary binary @emph{data}.
6531 The size of an element's data is limited to around 62 KB.
6535 * Set Modifications::
6539 * The SET Client-Service Protocol::
6540 * The SET Intersection Peer-to-Peer Protocol::
6541 * The SET Union Peer-to-Peer Protocol::
6545 @subsection Local Sets
6549 Sets created by a local client can be modified and reused for multiple
6550 operations. As each set operation requires potentially expensive special
6551 auxiliary data to be computed for each element of a set, a set can only
6552 participate in one type of set operation (i.e. union or intersection).
6553 The type of a set is determined upon its creation.
6554 If a the elements of a set are needed for an operation of a different
6555 type, all of the set's element must be copied to a new set of appropriate
6558 @node Set Modifications
6559 @subsection Set Modifications
6563 Even when set operations are active, one can add to and remove elements
6565 However, these changes will only be visible to operations that have been
6566 created after the changes have taken place. That is, every set operation
6567 only sees a snapshot of the set from the time the operation was started.
6568 This mechanism is @emph{not} implemented by copying the whole set, but by
6569 attaching @emph{generation information} to each element and operation.
6571 @node Set Operations
6572 @subsection Set Operations
6576 Set operations can be started in two ways: Either by accepting an
6577 operation request from a remote peer, or by requesting a set operation
6579 Set operations are uniquely identified by the involved @emph{peers}, an
6580 @emph{application id} and the @emph{operation type}.
6582 The client is notified of incoming set operations by @emph{set listeners}.
6583 A set listener listens for incoming operations of a specific operation
6584 type and application id.
6585 Once notified of an incoming set request, the client can accept the set
6586 request (providing a local set for the operation) or reject it.
6588 @node Result Elements
6589 @subsection Result Elements
6593 The SET service has three @emph{result modes} that determine how an
6594 operation's result set is delivered to the client:
6597 @item @strong{Full Result Set.} All elements of set resulting from the set
6598 operation are returned to the client.
6599 @item @strong{Added Elements.} Only elements that result from the
6600 operation and are not already in the local peer's set are returned.
6601 Note that for some operations (like set intersection) this result mode
6602 will never return any elements.
6603 This can be useful if only the remove peer is actually interested in
6604 the result of the set operation.
6605 @item @strong{Removed Elements.} Only elements that are in the local
6606 peer's initial set but not in the operation's result set are returned.
6607 Note that for some operations (like set union) this result mode will
6608 never return any elements. This can be useful if only the remove peer is
6609 actually interested in the result of the set operation.
6612 @cindex libgnunetset
6614 @subsection libgnunetset
6623 * The Result Callback::
6631 New sets are created with @code{GNUNET_SET_create}. Both the local peer's
6632 configuration (as each set has its own client connection) and the
6633 operation type must be specified.
6634 The set exists until either the client calls @code{GNUNET_SET_destroy} or
6635 the client's connection to the service is disrupted.
6636 In the latter case, the client is notified by the return value of
6637 functions dealing with sets. This return value must always be checked.
6639 Elements are added and removed with @code{GNUNET_SET_add_element} and
6640 @code{GNUNET_SET_remove_element}.
6643 @subsubsection Listeners
6647 Listeners are created with @code{GNUNET_SET_listen}. Each time time a
6648 remote peer suggests a set operation with an application id and operation
6649 type matching a listener, the listener's callback is invoked.
6650 The client then must synchronously call either @code{GNUNET_SET_accept}
6651 or @code{GNUNET_SET_reject}. Note that the operation will not be started
6652 until the client calls @code{GNUNET_SET_commit}
6653 (see Section "Supplying a Set").
6656 @subsubsection Operations
6660 Operations to be initiated by the local peer are created with
6661 @code{GNUNET_SET_prepare}. Note that the operation will not be started
6662 until the client calls @code{GNUNET_SET_commit}
6663 (see Section "Supplying a Set").
6665 @node Supplying a Set
6666 @subsubsection Supplying a Set
6670 To create symmetry between the two ways of starting a set operation
6671 (accepting and initiating it), the operation handles returned by
6672 @code{GNUNET_SET_accept} and @code{GNUNET_SET_prepare} do not yet have a
6673 set to operate on, thus they can not do any work yet.
6675 The client must call @code{GNUNET_SET_commit} to specify a set to use for
6676 an operation. @code{GNUNET_SET_commit} may only be called once per set
6679 @node The Result Callback
6680 @subsubsection The Result Callback
6684 Clients must specify both a result mode and a result callback with
6685 @code{GNUNET_SET_accept} and @code{GNUNET_SET_prepare}. The result
6686 callback with a status indicating either that an element was received, or
6687 the operation failed or succeeded.
6688 The interpretation of the received element depends on the result mode.
6689 The callback needs to know which result mode it is used in, as the
6690 arguments do not indicate if an element is part of the full result set,
6691 or if it is in the difference between the original set and the final set.
6693 @node The SET Client-Service Protocol
6694 @subsection The SET Client-Service Protocol
6701 * Initiating Operations::
6703 * Results and Operation Status::
6708 @subsubsection Creating Sets
6712 For each set of a client, there exists a client connection to the service.
6713 Sets are created by sending the @code{GNUNET_SERVICE_SET_CREATE} message
6714 over a new client connection. Multiple operations for one set are
6715 multiplexed over one client connection, using a request id supplied by
6719 @subsubsection Listeners2
6723 Each listener also requires a seperate client connection. By sending the
6724 @code{GNUNET_SERVICE_SET_LISTEN} message, the client notifies the service
6725 of the application id and operation type it is interested in. A client
6726 rejects an incoming request by sending @code{GNUNET_SERVICE_SET_REJECT}
6727 on the listener's client connection.
6728 In contrast, when accepting an incoming request, a
6729 @code{GNUNET_SERVICE_SET_ACCEPT} message must be sent over the@ set that
6730 is supplied for the set operation.
6732 @node Initiating Operations
6733 @subsubsection Initiating Operations
6737 Operations with remote peers are initiated by sending a
6738 @code{GNUNET_SERVICE_SET_EVALUATE} message to the service. The@ client
6739 connection that this message is sent by determines the set to use.
6741 @node Modifying Sets
6742 @subsubsection Modifying Sets
6746 Sets are modified with the @code{GNUNET_SERVICE_SET_ADD} and
6747 @code{GNUNET_SERVICE_SET_REMOVE} messages.
6751 @c %* Results and Operation Status::
6752 @c %* Iterating Sets::
6755 @node Results and Operation Status
6756 @subsubsection Results and Operation Status
6759 The service notifies the client of result elements and success/failure of
6760 a set operation with the @code{GNUNET_SERVICE_SET_RESULT} message.
6762 @node Iterating Sets
6763 @subsubsection Iterating Sets
6767 All elements of a set can be requested by sending
6768 @code{GNUNET_SERVICE_SET_ITER_REQUEST}. The server responds with
6769 @code{GNUNET_SERVICE_SET_ITER_ELEMENT} and eventually terminates the
6770 iteration with @code{GNUNET_SERVICE_SET_ITER_DONE}.
6771 After each received element, the client
6772 must send @code{GNUNET_SERVICE_SET_ITER_ACK}. Note that only one set
6773 iteration may be active for a set at any given time.
6775 @node The SET Intersection Peer-to-Peer Protocol
6776 @subsection The SET Intersection Peer-to-Peer Protocol
6780 The intersection protocol operates over CADET and starts with a
6781 GNUNET_MESSAGE_TYPE_SET_P2P_OPERATION_REQUEST being sent by the peer
6782 initiating the operation to the peer listening for inbound requests.
6783 It includes the number of elements of the initiating peer, which is used
6784 to decide which side will send a Bloom filter first.
6786 The listening peer checks if the operation type and application
6787 identifier are acceptable for its current state.
6788 If not, it responds with a GNUNET_MESSAGE_TYPE_SET_RESULT and a status of
6789 GNUNET_SET_STATUS_FAILURE (and terminates the CADET channel).
6791 If the application accepts the request, the listener sends back a
6792 @code{GNUNET_MESSAGE_TYPE_SET_INTERSECTION_P2P_ELEMENT_INFO} if it has
6793 more elements in the set than the client.
6794 Otherwise, it immediately starts with the Bloom filter exchange.
6795 If the initiator receives a
6796 @code{GNUNET_MESSAGE_TYPE_SET_INTERSECTION_P2P_ELEMENT_INFO} response,
6797 it beings the Bloom filter exchange, unless the set size is indicated to
6798 be zero, in which case the intersection is considered finished after
6799 just the initial handshake.
6803 * The Bloom filter exchange::
6807 @node The Bloom filter exchange
6808 @subsubsection The Bloom filter exchange
6812 In this phase, each peer transmits a Bloom filter over the remaining
6813 keys of the local set to the other peer using a
6814 @code{GNUNET_MESSAGE_TYPE_SET_INTERSECTION_P2P_BF} message. This
6815 message additionally includes the number of elements left in the sender's
6816 set, as well as the XOR over all of the keys in that set.
6818 The number of bits 'k' set per element in the Bloom filter is calculated
6819 based on the relative size of the two sets.
6820 Furthermore, the size of the Bloom filter is calculated based on 'k' and
6821 the number of elements in the set to maximize the amount of data filtered
6822 per byte transmitted on the wire (while avoiding an excessively high
6823 number of iterations).
6825 The receiver of the message removes all elements from its local set that
6826 do not pass the Bloom filter test.
6827 It then checks if the set size of the sender and the XOR over the keys
6828 match what is left of its own set. If they do, it sends a
6829 @code{GNUNET_MESSAGE_TYPE_SET_INTERSECTION_P2P_DONE} back to indicate
6830 that the latest set is the final result.
6831 Otherwise, the receiver starts another Bloom filter exchange, except
6832 this time as the sender.
6839 Bloomfilter operations are probabilistic: With some non-zero probability
6840 the test may incorrectly say an element is in the set, even though it is
6843 To mitigate this problem, the intersection protocol iterates exchanging
6844 Bloom filters using a different random 32-bit salt in each iteration (the
6845 salt is also included in the message).
6846 With different salts, set operations may fail for different elements.
6847 Merging the results from the executions, the probability of failure drops
6850 The iterations terminate once both peers have established that they have
6851 sets of the same size, and where the XOR over all keys computes the same
6852 512-bit value (leaving a failure probability of 2-511).
6854 @node The SET Union Peer-to-Peer Protocol
6855 @subsection The SET Union Peer-to-Peer Protocol
6859 The SET union protocol is based on Eppstein's efficient set reconciliation
6860 without prior context. You should read this paper first if you want to
6861 understand the protocol.
6863 The union protocol operates over CADET and starts with a
6864 GNUNET_MESSAGE_TYPE_SET_P2P_OPERATION_REQUEST being sent by the peer
6865 initiating the operation to the peer listening for inbound requests.
6866 It includes the number of elements of the initiating peer, which is
6869 The listening peer checks if the operation type and application
6870 identifier are acceptable for its current state. If not, it responds with
6871 a @code{GNUNET_MESSAGE_TYPE_SET_RESULT} and a status of
6872 @code{GNUNET_SET_STATUS_FAILURE} (and terminates the CADET channel).
6874 If the application accepts the request, it sends back a strata estimator
6875 using a message of type GNUNET_MESSAGE_TYPE_SET_UNION_P2P_SE. The
6876 initiator evaluates the strata estimator and initiates the exchange of
6877 invertible Bloom filters, sending a GNUNET_MESSAGE_TYPE_SET_UNION_P2P_IBF.
6879 During the IBF exchange, if the receiver cannot invert the Bloom filter or
6880 detects a cycle, it sends a larger IBF in response (up to a defined
6881 maximum limit; if that limit is reached, the operation fails).
6882 Elements decoded while processing the IBF are transmitted to the other
6883 peer using GNUNET_MESSAGE_TYPE_SET_P2P_ELEMENTS, or requested from the
6884 other peer using GNUNET_MESSAGE_TYPE_SET_P2P_ELEMENT_REQUESTS messages,
6885 depending on the sign observed during decoding of the IBF.
6886 Peers respond to a GNUNET_MESSAGE_TYPE_SET_P2P_ELEMENT_REQUESTS message
6887 with the respective element in a GNUNET_MESSAGE_TYPE_SET_P2P_ELEMENTS
6888 message. If the IBF fully decodes, the peer responds with a
6889 GNUNET_MESSAGE_TYPE_SET_UNION_P2P_DONE message instead of another
6890 GNUNET_MESSAGE_TYPE_SET_UNION_P2P_IBF.
6892 All Bloom filter operations use a salt to mingle keys before hashing them
6893 into buckets, such that future iterations have a fresh chance of
6894 succeeding if they failed due to collisions before.
6896 @cindex STATISTICS Subsystem
6897 @node STATISTICS Subsystem
6898 @section STATISTICS Subsystem
6902 In GNUnet, the STATISTICS subsystem offers a central place for all
6903 subsystems to publish unsigned 64-bit integer run-time statistics.
6904 Keeping this information centrally means that there is a unified way for
6905 the user to obtain data on all subsystems, and individual subsystems do
6906 not have to always include a custom data export method for performance
6907 metrics and other statistics. For example, the TRANSPORT system uses
6908 STATISTICS to update information about the number of directly connected
6909 peers and the bandwidth that has been consumed by the various plugins.
6910 This information is valuable for diagnosing connectivity and performance
6913 Following the GNUnet service architecture, the STATISTICS subsystem is
6914 divided into an API which is exposed through the header
6915 @strong{gnunet_statistics_service.h} and the STATISTICS service
6916 @strong{gnunet-service-statistics}. The @strong{gnunet-statistics}
6917 command-line tool can be used to obtain (and change) information about
6918 the values stored by the STATISTICS service. The STATISTICS service does
6919 not communicate with other peers.
6921 Data is stored in the STATISTICS service in the form of tuples
6922 @strong{(subsystem, name, value, persistence)}. The subsystem determines
6923 to which other GNUnet's subsystem the data belongs. name is the name
6924 through which value is associated. It uniquely identifies the record
6925 from among other records belonging to the same subsystem.
6926 In some parts of the code, the pair @strong{(subsystem, name)} is called
6927 a @strong{statistic} as it identifies the values stored in the STATISTCS
6928 service.The persistence flag determines if the record has to be preserved
6929 across service restarts. A record is said to be persistent if this flag
6930 is set for it; if not, the record is treated as a non-persistent record
6931 and it is lost after service restart. Persistent records are written to
6932 and read from the file @strong{statistics.data} before shutdown
6933 and upon startup. The file is located in the HOME directory of the peer.
6935 An anomaly of the STATISTICS service is that it does not terminate
6936 immediately upon receiving a shutdown signal if it has any clients
6937 connected to it. It waits for all the clients that are not monitors to
6938 close their connections before terminating itself.
6939 This is to prevent the loss of data during peer shutdown --- delaying the
6940 STATISTICS service shutdown helps other services to store important data
6941 to STATISTICS during shutdown.
6944 * libgnunetstatistics::
6945 * The STATISTICS Client-Service Protocol::
6948 @cindex libgnunetstatistics
6949 @node libgnunetstatistics
6950 @subsection libgnunetstatistics
6954 @strong{libgnunetstatistics} is the library containing the API for the
6955 STATISTICS subsystem. Any process requiring to use STATISTICS should use
6956 this API by to open a connection to the STATISTICS service.
6957 This is done by calling the function @code{GNUNET_STATISTICS_create()}.
6958 This function takes the subsystem's name which is trying to use STATISTICS
6959 and a configuration.
6960 All values written to STATISTICS with this connection will be placed in
6961 the section corresponding to the given subsystem's name.
6962 The connection to STATISTICS can be destroyed with the function
6963 @code{GNUNET_STATISTICS_destroy()}. This function allows for the
6964 connection to be destroyed immediately or upon transferring all
6965 pending write requests to the service.
6967 Note: STATISTICS subsystem can be disabled by setting @code{DISABLE = YES}
6968 under the @code{[STATISTICS]} section in the configuration. With such a
6969 configuration all calls to @code{GNUNET_STATISTICS_create()} return
6970 @code{NULL} as the STATISTICS subsystem is unavailable and no other
6971 functions from the API can be used.
6975 * Statistics retrieval::
6976 * Setting statistics and updating them::
6980 @node Statistics retrieval
6981 @subsubsection Statistics retrieval
6985 Once a connection to the statistics service is obtained, information
6986 about any other system which uses statistics can be retrieved with the
6987 function GNUNET_STATISTICS_get().
6988 This function takes the connection handle, the name of the subsystem
6989 whose information we are interested in (a @code{NULL} value will
6990 retrieve information of all available subsystems using STATISTICS), the
6991 name of the statistic we are interested in (a @code{NULL} value will
6992 retrieve all available statistics), a continuation callback which is
6993 called when all of requested information is retrieved, an iterator
6994 callback which is called for each parameter in the retrieved information
6995 and a closure for the aforementioned callbacks. The library then invokes
6996 the iterator callback for each value matching the request.
6998 Call to @code{GNUNET_STATISTICS_get()} is asynchronous and can be
6999 canceled with the function @code{GNUNET_STATISTICS_get_cancel()}.
7000 This is helpful when retrieving statistics takes too long and especially
7001 when we want to shutdown and cleanup everything.
7003 @node Setting statistics and updating them
7004 @subsubsection Setting statistics and updating them
7008 So far we have seen how to retrieve statistics, here we will learn how we
7009 can set statistics and update them so that other subsystems can retrieve
7012 A new statistic can be set using the function
7013 @code{GNUNET_STATISTICS_set()}.
7014 This function takes the name of the statistic and its value and a flag to
7015 make the statistic persistent.
7016 The value of the statistic should be of the type @code{uint64_t}.
7017 The function does not take the name of the subsystem; it is determined
7018 from the previous @code{GNUNET_STATISTICS_create()} invocation. If
7019 the given statistic is already present, its value is overwritten.
7021 An existing statistics can be updated, i.e its value can be increased or
7022 decreased by an amount with the function
7023 @code{GNUNET_STATISTICS_update()}.
7024 The parameters to this function are similar to
7025 @code{GNUNET_STATISTICS_set()}, except that it takes the amount to be
7026 changed as a type @code{int64_t} instead of the value.
7028 The library will combine multiple set or update operations into one
7029 message if the client performs requests at a rate that is faster than the
7030 available IPC with the STATISTICS service. Thus, the client does not have
7031 to worry about sending requests too quickly.
7034 @subsubsection Watches
7038 As interesting feature of STATISTICS lies in serving notifications
7039 whenever a statistic of our interest is modified.
7040 This is achieved by registering a watch through the function
7041 @code{GNUNET_STATISTICS_watch()}.
7042 The parameters of this function are similar to those of
7043 @code{GNUNET_STATISTICS_get()}.
7044 Changes to the respective statistic's value will then cause the given
7045 iterator callback to be called.
7046 Note: A watch can only be registered for a specific statistic. Hence
7047 the subsystem name and the parameter name cannot be @code{NULL} in a
7048 call to @code{GNUNET_STATISTICS_watch()}.
7050 A registered watch will keep notifying any value changes until
7051 @code{GNUNET_STATISTICS_watch_cancel()} is called with the same
7052 parameters that are used for registering the watch.
7054 @node The STATISTICS Client-Service Protocol
7055 @subsection The STATISTICS Client-Service Protocol
7060 * Statistics retrieval2::
7061 * Setting and updating statistics::
7062 * Watching for updates::
7065 @node Statistics retrieval2
7066 @subsubsection Statistics retrieval2
7070 To retrieve statistics, the client transmits a message of type
7071 @code{GNUNET_MESSAGE_TYPE_STATISTICS_GET} containing the given subsystem
7072 name and statistic parameter to the STATISTICS service.
7073 The service responds with a message of type
7074 @code{GNUNET_MESSAGE_TYPE_STATISTICS_VALUE} for each of the statistics
7075 parameters that match the client request for the client. The end of
7076 information retrieved is signaled by the service by sending a message of
7077 type @code{GNUNET_MESSAGE_TYPE_STATISTICS_END}.
7079 @node Setting and updating statistics
7080 @subsubsection Setting and updating statistics
7084 The subsystem name, parameter name, its value and the persistence flag are
7085 communicated to the service through the message
7086 @code{GNUNET_MESSAGE_TYPE_STATISTICS_SET}.
7088 When the service receives a message of type
7089 @code{GNUNET_MESSAGE_TYPE_STATISTICS_SET}, it retrieves the subsystem
7090 name and checks for a statistic parameter with matching the name given in
7092 If a statistic parameter is found, the value is overwritten by the new
7093 value from the message; if not found then a new statistic parameter is
7094 created with the given name and value.
7096 In addition to just setting an absolute value, it is possible to perform a
7097 relative update by sending a message of type
7098 @code{GNUNET_MESSAGE_TYPE_STATISTICS_SET} with an update flag
7099 (@code{GNUNET_STATISTICS_SETFLAG_RELATIVE}) signifying that the value in
7100 the message should be treated as an update value.
7102 @node Watching for updates
7103 @subsubsection Watching for updates
7107 The function registers the watch at the service by sending a message of
7108 type @code{GNUNET_MESSAGE_TYPE_STATISTICS_WATCH}. The service then sends
7109 notifications through messages of type
7110 @code{GNUNET_MESSAGE_TYPE_STATISTICS_WATCH_VALUE} whenever the statistic
7111 parameter's value is changed.
7114 @cindex Distributed Hash Table
7115 @node Distributed Hash Table (DHT)
7116 @section Distributed Hash Table (DHT)
7120 GNUnet includes a generic distributed hash table that can be used by
7121 developers building P2P applications in the framework.
7122 This section documents high-level features and how developers are
7123 expected to use the DHT.
7124 We have a research paper detailing how the DHT works.
7125 Also, Nate's thesis includes a detailed description and performance
7126 analysis (in chapter 6).
7128 Key features of GNUnet's DHT include:
7131 @item stores key-value pairs with values up to (approximately) 63k in size
7132 @item works with many underlay network topologies (small-world, random
7133 graph), underlay does not need to be a full mesh / clique
7134 @item support for extended queries (more than just a simple 'key'),
7135 filtering duplicate replies within the network (bloomfilter) and content
7136 validation (for details, please read the subsection on the block library)
7137 @item can (optionally) return paths taken by the PUT and GET operations
7139 @item provides content replication to handle churn
7142 GNUnet's DHT is randomized and unreliable. Unreliable means that there is
7143 no strict guarantee that a value stored in the DHT is always
7144 found --- values are only found with high probability.
7145 While this is somewhat true in all P2P DHTs, GNUnet developers should be
7146 particularly wary of this fact (this will help you write secure,
7147 fault-tolerant code). Thus, when writing any application using the DHT,
7148 you should always consider the possibility that a value stored in the
7149 DHT by you or some other peer might simply not be returned, or returned
7150 with a significant delay.
7151 Your application logic must be written to tolerate this (naturally, some
7152 loss of performance or quality of service is expected in this case).
7155 * Block library and plugins::
7157 * The DHT Client-Service Protocol::
7158 * The DHT Peer-to-Peer Protocol::
7161 @node Block library and plugins
7162 @subsection Block library and plugins
7167 * What is a Block?::
7168 * The API of libgnunetblock::
7174 @node What is a Block?
7175 @subsubsection What is a Block?
7179 Blocks are small (< 63k) pieces of data stored under a key (struct
7180 GNUNET_HashCode). Blocks have a type (enum GNUNET_BlockType) which defines
7181 their data format. Blocks are used in GNUnet as units of static data
7182 exchanged between peers and stored (or cached) locally.
7183 Uses of blocks include file-sharing (the files are broken up into blocks),
7184 the VPN (DNS information is stored in blocks) and the DHT (all
7185 information in the DHT and meta-information for the maintenance of the
7186 DHT are both stored using blocks).
7187 The block subsystem provides a few common functions that must be
7188 available for any type of block.
7190 @cindex libgnunetblock API
7191 @node The API of libgnunetblock
7192 @subsubsection The API of libgnunetblock
7196 The block library requires for each (family of) block type(s) a block
7197 plugin (implementing @file{gnunet_block_plugin.h}) that provides basic
7198 functions that are needed by the DHT (and possibly other subsystems) to
7200 These block plugins are typically implemented within their respective
7202 The main block library is then used to locate, load and query the
7203 appropriate block plugin.
7204 Which plugin is appropriate is determined by the block type (which is
7205 just a 32-bit integer). Block plugins contain code that specifies which
7206 block types are supported by a given plugin. The block library loads all
7207 block plugins that are installed at the local peer and forwards the
7208 application request to the respective plugin.
7210 The central functions of the block APIs (plugin and main library) are to
7211 allow the mapping of blocks to their respective key (if possible) and the
7212 ability to check that a block is well-formed and matches a given
7213 request (again, if possible).
7214 This way, GNUnet can avoid storing invalid blocks, storing blocks under
7215 the wrong key and forwarding blocks in response to a query that they do
7218 One key function of block plugins is that it allows GNUnet to detect
7219 duplicate replies (via the Bloom filter). All plugins MUST support
7220 detecting duplicate replies (by adding the current response to the
7221 Bloom filter and rejecting it if it is encountered again).
7222 If a plugin fails to do this, responses may loop in the network.
7225 @subsubsection Queries
7228 The query format for any block in GNUnet consists of four main components.
7229 First, the type of the desired block must be specified. Second, the query
7230 must contain a hash code. The hash code is used for lookups in hash
7231 tables and databases and must not be unique for the block (however, if
7232 possible a unique hash should be used as this would be best for
7234 Third, an optional Bloom filter can be specified to exclude known results;
7235 replies that hash to the bits set in the Bloom filter are considered
7236 invalid. False-positives can be eliminated by sending the same query
7237 again with a different Bloom filter mutator value, which parameterizes
7238 the hash function that is used.
7239 Finally, an optional application-specific "eXtended query" (xquery) can
7240 be specified to further constrain the results. It is entirely up to
7241 the type-specific plugin to determine whether or not a given block
7242 matches a query (type, hash, Bloom filter, and xquery).
7243 Naturally, not all xquery's are valid and some types of blocks may not
7244 support Bloom filters either, so the plugin also needs to check if the
7245 query is valid in the first place.
7247 Depending on the results from the plugin, the DHT will then discard the
7248 (invalid) query, forward the query, discard the (invalid) reply, cache the
7249 (valid) reply, and/or forward the (valid and non-duplicate) reply.
7252 @subsubsection Sample Code
7256 The source code in @strong{plugin_block_test.c} is a good starting point
7257 for new block plugins --- it does the minimal work by implementing a
7258 plugin that performs no validation at all.
7259 The respective @strong{Makefile.am} shows how to build and install a
7263 @subsubsection Conclusion2
7267 In conclusion, GNUnet subsystems that want to use the DHT need to define a
7268 block format and write a plugin to match queries and replies. For testing,
7269 the @code{GNUNET_BLOCK_TYPE_TEST} block type can be used; it accepts
7270 any query as valid and any reply as matching any query.
7271 This type is also used for the DHT command line tools.
7272 However, it should NOT be used for normal applications due to the lack
7273 of error checking that results from this primitive implementation.
7275 @cindex libgnunetdht
7277 @subsection libgnunetdht
7281 The DHT API itself is pretty simple and offers the usual GET and PUT
7282 functions that work as expected. The specified block type refers to the
7283 block library which allows the DHT to run application-specific logic for
7284 data stored in the network.
7291 * DHT Routing Options::
7299 When using GET, the main consideration for developers (other than the
7300 block library) should be that after issuing a GET, the DHT will
7301 continuously cause (small amounts of) network traffic until the operation
7302 is explicitly canceled.
7303 So GET does not simply send out a single network request once; instead,
7304 the DHT will continue to search for data. This is needed to achieve good
7305 success rates and also handles the case where the respective PUT
7306 operation happens after the GET operation was started.
7307 Developers should not cancel an existing GET operation and then
7308 explicitly re-start it to trigger a new round of network requests;
7309 this is simply inefficient, especially as the internal automated version
7310 can be more efficient, for example by filtering results in the network
7311 that have already been returned.
7313 If an application that performs a GET request has a set of replies that it
7314 already knows and would like to filter, it can call@
7315 @code{GNUNET_DHT_get_filter_known_results} with an array of hashes over
7316 the respective blocks to tell the DHT that these results are not
7318 This way, the DHT will filter the respective blocks using the block
7319 library in the network, which may result in a significant reduction in
7320 bandwidth consumption.
7327 @c inconsistent use of ``must'' above it's written ``MUST''
7328 In contrast to GET operations, developers @strong{must} manually re-run
7329 PUT operations periodically (if they intend the content to continue to be
7330 available). Content stored in the DHT expires or might be lost due to
7332 Furthermore, GNUnet's DHT typically requires multiple rounds of PUT
7333 operations before a key-value pair is consistently available to all
7334 peers (the DHT randomizes paths and thus storage locations, and only
7335 after multiple rounds of PUTs there will be a sufficient number of
7336 replicas in large DHTs). An explicit PUT operation using the DHT API will
7337 only cause network traffic once, so in order to ensure basic availability
7338 and resistance to churn (and adversaries), PUTs must be repeated.
7339 While the exact frequency depends on the application, a rule of thumb is
7340 that there should be at least a dozen PUT operations within the content
7341 lifetime. Content in the DHT typically expires after one day, so
7342 DHT PUT operations should be repeated at least every 1-2 hours.
7345 @subsubsection MONITOR
7349 The DHT API also allows applications to monitor messages crossing the
7351 The types of messages used by the DHT are GET, PUT and RESULT messages.
7352 Using the monitoring API, applications can choose to monitor these
7353 requests, possibly limiting themselves to requests for a particular block
7356 The monitoring API is not only useful for diagnostics, it can also be
7357 used to trigger application operations based on PUT operations.
7358 For example, an application may use PUTs to distribute work requests to
7360 The workers would then monitor for PUTs that give them work, instead of
7361 looking for work using GET operations.
7362 This can be beneficial, especially if the workers have no good way to
7363 guess the keys under which work would be stored.
7364 Naturally, additional protocols might be needed to ensure that the desired
7365 number of workers will process the distributed workload.
7367 @node DHT Routing Options
7368 @subsubsection DHT Routing Options
7372 There are two important options for GET and PUT requests:
7375 @item GNUNET_DHT_RO_DEMULITPLEX_EVERYWHERE This option means that all
7376 peers should process the request, even if their peer ID is not closest to
7377 the key. For a PUT request, this means that all peers that a request
7378 traverses may make a copy of the data.
7379 Similarly for a GET request, all peers will check their local database
7380 for a result. Setting this option can thus significantly improve caching
7381 and reduce bandwidth consumption --- at the expense of a larger DHT
7382 database. If in doubt, we recommend that this option should be used.
7383 @item GNUNET_DHT_RO_RECORD_ROUTE This option instructs the DHT to record
7384 the path that a GET or a PUT request is taking through the overlay
7385 network. The resulting paths are then returned to the application with
7386 the respective result. This allows the receiver of a result to construct
7387 a path to the originator of the data, which might then be used for
7388 routing. Naturally, setting this option requires additional bandwidth
7389 and disk space, so applications should only set this if the paths are
7390 needed by the application logic.
7391 @item GNUNET_DHT_RO_FIND_PEER This option is an internal option used by
7392 the DHT's peer discovery mechanism and should not be used by applications.
7393 @item GNUNET_DHT_RO_BART This option is currently not implemented. It may
7394 in the future offer performance improvements for clique topologies.
7397 @node The DHT Client-Service Protocol
7398 @subsection The DHT Client-Service Protocol
7403 * PUTting data into the DHT::
7404 * GETting data from the DHT::
7405 * Monitoring the DHT::
7408 @node PUTting data into the DHT
7409 @subsubsection PUTting data into the DHT
7413 To store (PUT) data into the DHT, the client sends a
7414 @code{struct GNUNET_DHT_ClientPutMessage} to the service.
7415 This message specifies the block type, routing options, the desired
7416 replication level, the expiration time, key,
7417 value and a 64-bit unique ID for the operation. The service responds with
7418 a @code{struct GNUNET_DHT_ClientPutConfirmationMessage} with the same
7419 64-bit unique ID. Note that the service sends the confirmation as soon as
7420 it has locally processed the PUT request. The PUT may still be
7421 propagating through the network at this time.
7423 In the future, we may want to change this to provide (limited) feedback
7424 to the client, for example if we detect that the PUT operation had no
7425 effect because the same key-value pair was already stored in the DHT.
7426 However, changing this would also require additional state and messages
7427 in the P2P interaction.
7429 @node GETting data from the DHT
7430 @subsubsection GETting data from the DHT
7434 To retrieve (GET) data from the DHT, the client sends a
7435 @code{struct GNUNET_DHT_ClientGetMessage} to the service. The message
7436 specifies routing options, a replication level (for replicating the GET,
7437 not the content), the desired block type, the key, the (optional)
7438 extended query and unique 64-bit request ID.
7440 Additionally, the client may send any number of
7441 @code{struct GNUNET_DHT_ClientGetResultSeenMessage}s to notify the
7442 service about results that the client is already aware of.
7443 These messages consist of the key, the unique 64-bit ID of the request,
7444 and an arbitrary number of hash codes over the blocks that the client is
7445 already aware of. As messages are restricted to 64k, a client that
7446 already knows more than about a thousand blocks may need to send
7447 several of these messages. Naturally, the client should transmit these
7448 messages as quickly as possible after the original GET request such that
7449 the DHT can filter those results in the network early on. Naturally, as
7450 these messages are sent after the original request, it is conceivable
7451 that the DHT service may return blocks that match those already known
7452 to the client anyway.
7454 In response to a GET request, the service will send @code{struct
7455 GNUNET_DHT_ClientResultMessage}s to the client. These messages contain the
7456 block type, expiration, key, unique ID of the request and of course the
7457 value (a block). Depending on the options set for the respective
7458 operations, the replies may also contain the path the GET and/or the PUT
7459 took through the network.
7461 A client can stop receiving replies either by disconnecting or by sending
7462 a @code{struct GNUNET_DHT_ClientGetStopMessage} which must contain the
7463 key and the 64-bit unique ID of the original request. Using an
7464 explicit "stop" message is more common as this allows a client to run
7465 many concurrent GET operations over the same connection with the DHT
7466 service --- and to stop them individually.
7468 @node Monitoring the DHT
7469 @subsubsection Monitoring the DHT
7473 To begin monitoring, the client sends a
7474 @code{struct GNUNET_DHT_MonitorStartStop} message to the DHT service.
7475 In this message, flags can be set to enable (or disable) monitoring of
7476 GET, PUT and RESULT messages that pass through a peer. The message can
7477 also restrict monitoring to a particular block type or a particular key.
7478 Once monitoring is enabled, the DHT service will notify the client about
7479 any matching event using @code{struct GNUNET_DHT_MonitorGetMessage}s for
7480 GET events, @code{struct GNUNET_DHT_MonitorPutMessage} for PUT events
7481 and @code{struct GNUNET_DHT_MonitorGetRespMessage} for RESULTs. Each of
7482 these messages contains all of the information about the event.
7484 @node The DHT Peer-to-Peer Protocol
7485 @subsection The DHT Peer-to-Peer Protocol
7490 * Routing GETs or PUTs::
7491 * PUTting data into the DHT2::
7492 * GETting data from the DHT2::
7495 @node Routing GETs or PUTs
7496 @subsubsection Routing GETs or PUTs
7500 When routing GETs or PUTs, the DHT service selects a suitable subset of
7501 neighbours for forwarding. The exact number of neighbours can be zero or
7502 more and depends on the hop counter of the query (initially zero) in
7503 relation to the (log of) the network size estimate, the desired
7504 replication level and the peer's connectivity.
7505 Depending on the hop counter and our network size estimate, the selection
7506 of the peers maybe randomized or by proximity to the key.
7507 Furthermore, requests include a set of peers that a request has already
7508 traversed; those peers are also excluded from the selection.
7510 @node PUTting data into the DHT2
7511 @subsubsection PUTting data into the DHT2
7515 To PUT data into the DHT, the service sends a @code{struct PeerPutMessage}
7516 of type @code{GNUNET_MESSAGE_TYPE_DHT_P2P_PUT} to the respective
7518 In addition to the usual information about the content (type, routing
7519 options, desired replication level for the content, expiration time, key
7520 and value), the message contains a fixed-size Bloom filter with
7521 information about which peers (may) have already seen this request.
7522 This Bloom filter is used to ensure that DHT messages never loop back to
7523 a peer that has already processed the request.
7524 Additionally, the message includes the current hop counter and, depending
7525 on the routing options, the message may include the full path that the
7526 message has taken so far.
7527 The Bloom filter should already contain the identity of the previous hop;
7528 however, the path should not include the identity of the previous hop and
7529 the receiver should append the identity of the sender to the path, not
7530 its own identity (this is done to reduce bandwidth).
7532 @node GETting data from the DHT2
7533 @subsubsection GETting data from the DHT2
7537 A peer can search the DHT by sending @code{struct PeerGetMessage}s of type
7538 @code{GNUNET_MESSAGE_TYPE_DHT_P2P_GET} to other peers. In addition to the
7539 usual information about the request (type, routing options, desired
7540 replication level for the request, the key and the extended query), a GET
7541 request also contains a hop counter, a Bloom filter over the peers
7542 that have processed the request already and depending on the routing
7543 options the full path traversed by the GET.
7544 Finally, a GET request includes a variable-size second Bloom filter and a
7545 so-called Bloom filter mutator value which together indicate which
7546 replies the sender has already seen. During the lookup, each block that
7547 matches they block type, key and extended query is additionally subjected
7548 to a test against this Bloom filter.
7549 The block plugin is expected to take the hash of the block and combine it
7550 with the mutator value and check if the result is not yet in the Bloom
7551 filter. The originator of the query will from time to time modify the
7552 mutator to (eventually) allow false-positives filtered by the Bloom filter
7555 Peers that receive a GET request perform a local lookup (depending on
7556 their proximity to the key and the query options) and forward the request
7558 They then remember the request (including the Bloom filter for blocking
7559 duplicate results) and when they obtain a matching, non-filtered response
7560 a @code{struct PeerResultMessage} of type
7561 @code{GNUNET_MESSAGE_TYPE_DHT_P2P_RESULT} is forwarded to the previous
7563 Whenever a result is forwarded, the block plugin is used to update the
7564 Bloom filter accordingly, to ensure that the same result is never
7565 forwarded more than once.
7566 The DHT service may also cache forwarded results locally if the
7567 "CACHE_RESULTS" option is set to "YES" in the configuration.
7570 @cindex GNU Name System
7571 @node GNU Name System (GNS)
7572 @section GNU Name System (GNS)
7576 The GNU Name System (GNS) is a decentralized database that enables users
7577 to securely resolve names to values.
7578 Names can be used to identify other users (for example, in social
7579 networking), or network services (for example, VPN services running at a
7580 peer in GNUnet, or purely IP-based services on the Internet).
7581 Users interact with GNS by typing in a hostname that ends in a
7582 top-level domain that is configured in the ``GNS'' section, matches
7583 an identity of the user or ends in a Base32-encoded public key.
7585 Videos giving an overview of most of the GNS and the motivations behind
7586 it is available here and here.
7587 The remainder of this chapter targets developers that are familiar with
7588 high level concepts of GNS as presented in these talks.
7589 @c TODO: Add links to here and here and to these.
7591 GNS-aware applications should use the GNS resolver to obtain the
7592 respective records that are stored under that name in GNS.
7593 Each record consists of a type, value, expiration time and flags.
7595 The type specifies the format of the value. Types below 65536 correspond
7596 to DNS record types, larger values are used for GNS-specific records.
7597 Applications can define new GNS record types by reserving a number and
7598 implementing a plugin (which mostly needs to convert the binary value
7599 representation to a human-readable text format and vice-versa).
7600 The expiration time specifies how long the record is to be valid.
7601 The GNS API ensures that applications are only given non-expired values.
7602 The flags are typically irrelevant for applications, as GNS uses them
7603 internally to control visibility and validity of records.
7605 Records are stored along with a signature.
7606 The signature is generated using the private key of the authoritative
7607 zone. This allows any GNS resolver to verify the correctness of a
7610 Internally, GNS uses the NAMECACHE to cache information obtained from
7611 other users, the NAMESTORE to store information specific to the local
7612 users, and the DHT to exchange data between users.
7613 A plugin API is used to enable applications to define new GNS
7618 * libgnunetgnsrecord::
7620 * The GNS Client-Service Protocol::
7621 * Hijacking the DNS-Traffic using gnunet-service-dns::
7622 * Serving DNS lookups via GNS on W32::
7623 * Importing DNS Zones into GNS::
7627 @subsection libgnunetgns
7631 The GNS API itself is extremely simple. Clients first connect to the
7632 GNS service using @code{GNUNET_GNS_connect}.
7633 They can then perform lookups using @code{GNUNET_GNS_lookup} or cancel
7634 pending lookups using @code{GNUNET_GNS_lookup_cancel}.
7635 Once finished, clients disconnect using @code{GNUNET_GNS_disconnect}.
7638 * Looking up records::
7639 * Accessing the records::
7640 * Creating records::
7644 @node Looking up records
7645 @subsubsection Looking up records
7649 @code{GNUNET_GNS_lookup} takes a number of arguments:
7652 @item handle This is simply the GNS connection handle from
7653 @code{GNUNET_GNS_connect}.
7654 @item name The client needs to specify the name to
7655 be resolved. This can be any valid DNS or GNS hostname.
7656 @item zone The client
7657 needs to specify the public key of the GNS zone against which the
7658 resolution should be done.
7659 Note that a key must be provided, the client should
7660 look up plausible values using its configuration,
7661 the identity service and by attempting to interpret the
7662 TLD as a base32-encoded public key.
7663 @item type This is the desired GNS or DNS record type
7664 to look for. While all records for the given name will be returned, this
7665 can be important if the client wants to resolve record types that
7666 themselves delegate resolution, such as CNAME, PKEY or GNS2DNS.
7667 Resolving a record of any of these types will only work if the respective
7668 record type is specified in the request, as the GNS resolver will
7669 otherwise follow the delegation and return the records from the
7670 respective destination, instead of the delegating record.
7671 @item only_cached This argument should typically be set to
7672 @code{GNUNET_NO}. Setting it to @code{GNUNET_YES} disables resolution via
7673 the overlay network.
7674 @item shorten_zone_key If GNS encounters new names during resolution,
7675 their respective zones can automatically be learned and added to the
7676 "shorten zone". If this is desired, clients must pass the private key of
7677 the shorten zone. If NULL is passed, shortening is disabled.
7678 @item proc This argument identifies
7679 the function to call with the result. It is given proc_cls, the number of
7680 records found (possibly zero) and the array of the records as arguments.
7681 proc will only be called once. After proc,> has been called, the lookup
7682 must no longer be canceled.
7683 @item proc_cls The closure for proc.
7686 @node Accessing the records
7687 @subsubsection Accessing the records
7691 The @code{libgnunetgnsrecord} library provides an API to manipulate the
7692 GNS record array that is given to proc. In particular, it offers
7693 functions such as converting record values to human-readable
7694 strings (and back). However, most @code{libgnunetgnsrecord} functions are
7695 not interesting to GNS client applications.
7697 For DNS records, the @code{libgnunetdnsparser} library provides
7698 functions for parsing (and serializing) common types of DNS records.
7700 @node Creating records
7701 @subsubsection Creating records
7705 Creating GNS records is typically done by building the respective record
7706 information (possibly with the help of @code{libgnunetgnsrecord} and
7707 @code{libgnunetdnsparser}) and then using the @code{libgnunetnamestore} to
7708 publish the information. The GNS API is not involved in this
7712 @subsubsection Future work
7716 In the future, we want to expand @code{libgnunetgns} to allow
7717 applications to observe shortening operations performed during GNS
7718 resolution, for example so that users can receive visual feedback when
7721 @node libgnunetgnsrecord
7722 @subsection libgnunetgnsrecord
7726 The @code{libgnunetgnsrecord} library is used to manipulate GNS
7727 records (in plaintext or in their encrypted format).
7728 Applications mostly interact with @code{libgnunetgnsrecord} by using the
7729 functions to convert GNS record values to strings or vice-versa, or to
7730 lookup a GNS record type number by name (or vice-versa).
7731 The library also provides various other functions that are mostly
7732 used internally within GNS, such as converting keys to names, checking for
7733 expiration, encrypting GNS records to GNS blocks, verifying GNS block
7734 signatures and decrypting GNS records from GNS blocks.
7736 We will now discuss the four commonly used functions of the API.@
7737 @code{libgnunetgnsrecord} does not perform these operations itself,
7738 but instead uses plugins to perform the operation.
7739 GNUnet includes plugins to support common DNS record types as well as
7740 standard GNS record types.
7747 @node Value handling
7748 @subsubsection Value handling
7752 @code{GNUNET_GNSRECORD_value_to_string} can be used to convert
7753 the (binary) representation of a GNS record value to a human readable,
7754 0-terminated UTF-8 string.
7755 NULL is returned if the specified record type is not supported by any
7758 @code{GNUNET_GNSRECORD_string_to_value} can be used to try to convert a
7759 human readable string to the respective (binary) representation of
7763 @subsubsection Type handling
7767 @code{GNUNET_GNSRECORD_typename_to_number} can be used to obtain the
7768 numeric value associated with a given typename. For example, given the
7769 typename "A" (for DNS A reocrds), the function will return the number 1.
7770 A list of common DNS record types is
7771 @uref{http://en.wikipedia.org/wiki/List_of_DNS_record_types, here}.
7772 Note that not all DNS record types are supported by GNUnet GNSRECORD
7773 plugins at this time.
7775 @code{GNUNET_GNSRECORD_number_to_typename} can be used to obtain the
7776 typename associated with a given numeric value.
7777 For example, given the type number 1, the function will return the
7781 @subsection GNS plugins
7785 Adding a new GNS record type typically involves writing (or extending) a
7786 GNSRECORD plugin. The plugin needs to implement the
7787 @code{gnunet_gnsrecord_plugin.h} API which provides basic functions that
7788 are needed by GNSRECORD to convert typenames and values of the respective
7789 record type to strings (and back).
7790 These gnsrecord plugins are typically implemented within their respective
7792 Examples for such plugins can be found in the GNSRECORD, GNS and
7793 CONVERSATION subsystems.
7795 The @code{libgnunetgnsrecord} library is then used to locate, load and
7796 query the appropriate gnsrecord plugin.
7797 Which plugin is appropriate is determined by the record type (which is
7798 just a 32-bit integer). The @code{libgnunetgnsrecord} library loads all
7799 block plugins that are installed at the local peer and forwards the
7800 application request to the plugins. If the record type is not
7801 supported by the plugin, it should simply return an error code.
7803 The central functions of the block APIs (plugin and main library) are the
7804 same four functions for converting between values and strings, and
7805 typenames and numbers documented in the previous subsection.
7807 @node The GNS Client-Service Protocol
7808 @subsection The GNS Client-Service Protocol
7811 The GNS client-service protocol consists of two simple messages, the
7812 @code{LOOKUP} message and the @code{LOOKUP_RESULT}. Each @code{LOOKUP}
7813 message contains a unique 32-bit identifier, which will be included in the
7814 corresponding response. Thus, clients can send many lookup requests in
7815 parallel and receive responses out-of-order.
7816 A @code{LOOKUP} request also includes the public key of the GNS zone,
7817 the desired record type and fields specifying whether shortening is
7818 enabled or networking is disabled. Finally, the @code{LOOKUP} message
7819 includes the name to be resolved.
7821 The response includes the number of records and the records themselves
7822 in the format created by @code{GNUNET_GNSRECORD_records_serialize}.
7823 They can thus be deserialized using
7824 @code{GNUNET_GNSRECORD_records_deserialize}.
7826 @node Hijacking the DNS-Traffic using gnunet-service-dns
7827 @subsection Hijacking the DNS-Traffic using gnunet-service-dns
7831 This section documents how the gnunet-service-dns (and the
7832 gnunet-helper-dns) intercepts DNS queries from the local system.
7833 This is merely one method for how we can obtain GNS queries.
7834 It is also possible to change @code{resolv.conf} to point to a machine
7835 running @code{gnunet-dns2gns} or to modify libc's name system switch
7836 (NSS) configuration to include a GNS resolution plugin.
7837 The method described in this chapter is more of a last-ditch catch-all
7840 @code{gnunet-service-dns} enables intercepting DNS traffic using policy
7842 We MARK every outgoing DNS-packet if it was not sent by our application.
7843 Using a second routing table in the Linux kernel these marked packets are
7844 then routed through our virtual network interface and can thus be
7847 Our application then reads the query and decides how to handle it.
7848 If the query can be addressed via GNS, it is passed to
7849 @code{gnunet-service-gns} and resolved internally using GNS.
7850 In the future, a reverse query for an address of the configured virtual
7851 network could be answered with records kept about previous forward
7853 Queries that are not hijacked by some application using the DNS service
7854 will be sent to the original recipient.
7855 The answer to the query will always be sent back through the virtual
7856 interface with the original nameserver as source address.
7860 * Network Setup Details::
7863 @node Network Setup Details
7864 @subsubsection Network Setup Details
7868 The DNS interceptor adds the following rules to the Linux kernel:
7870 iptables -t mangle -I OUTPUT 1 -p udp --sport $LOCALPORT --dport 53 \
7871 -j ACCEPT iptables -t mangle -I OUTPUT 2 -p udp --dport 53 -j MARK \
7872 --set-mark 3 ip rule add fwmark 3 table2 ip route add default via \
7876 @c FIXME: Rewrite to reflect display which is no longer content by line
7877 @c FIXME: due to the < 74 characters limit.
7878 Line 1 makes sure that all packets coming from a port our application
7879 opened beforehand (@code{$LOCALPORT}) will be routed normally.
7880 Line 2 marks every other packet to a DNS-Server with mark 3 (chosen
7881 arbitrarily). The third line adds a routing policy based on this mark
7882 3 via the routing table.
7884 @node Serving DNS lookups via GNS on W32
7885 @subsection Serving DNS lookups via GNS on W32
7889 This section documents how the libw32nsp (and
7890 gnunet-gns-helper-service-w32) do DNS resolutions of DNS queries on the
7891 local system. This only applies to GNUnet running on W32.
7893 W32 has a concept of "Namespaces" and "Namespace providers".
7894 These are used to present various name systems to applications in a
7896 Namespaces include DNS, mDNS, NLA and others. For each namespace any
7897 number of providers could be registered, and they are queried in an order
7898 of priority (which is adjustable).
7900 Applications can resolve names by using WSALookupService*() family of
7903 However, these are WSA-only facilities. Common BSD socket functions for
7904 namespace resolutions are gethostbyname and getaddrinfo (among others).
7905 These functions are implemented internally (by default - by mswsock,
7906 which also implements the default DNS provider) as wrappers around
7907 WSALookupService*() functions (see "Sample Code for a Service Provider"
7910 On W32 GNUnet builds a libw32nsp - a namespace provider, which can then be
7911 installed into the system by using w32nsp-install (and uninstalled by
7912 w32nsp-uninstall), as described in "Installation Handbook".
7914 libw32nsp is very simple and has almost no dependencies. As a response to
7915 NSPLookupServiceBegin(), it only checks that the provider GUID passed to
7916 it by the caller matches GNUnet DNS Provider GUID,
7918 gnunet-gns-helper-service-w32 at 127.0.0.1:5353 (hardcoded) and sends the
7919 name resolution request there, returning the connected socket to the
7922 When the caller invokes NSPLookupServiceNext(), libw32nsp reads a
7923 completely formed reply from that socket, unmarshalls it, then gives
7924 it back to the caller.
7926 At the moment gnunet-gns-helper-service-w32 is implemented to ever give
7927 only one reply, and subsequent calls to NSPLookupServiceNext() will fail
7928 with WSA_NODATA (first call to NSPLookupServiceNext() might also fail if
7929 GNS failed to find the name, or there was an error connecting to it).
7931 gnunet-gns-helper-service-w32 does most of the processing:
7934 @item Maintains a connection to GNS.
7935 @item Reads GNS config and loads appropriate keys.
7936 @item Checks service GUID and decides on the type of record to look up,
7937 refusing to make a lookup outright when unsupported service GUID is
7939 @item Launches the lookup
7942 When lookup result arrives, gnunet-gns-helper-service-w32 forms a complete
7943 reply (including filling a WSAQUERYSETW structure and, possibly, a binary
7944 blob with a hostent structure for gethostbyname() client), marshalls it,
7945 and sends it back to libw32nsp. If no records were found, it sends an
7948 This works for most normal applications that use gethostbyname() or
7949 getaddrinfo() to resolve names, but fails to do anything with
7950 applications that use alternative means of resolving names (such as
7951 sending queries to a DNS server directly by themselves).
7952 This includes some of well known utilities, like "ping" and "nslookup".
7954 @node Importing DNS Zones into GNS
7955 @subsection Importing DNS Zones into GNS
7957 This section discusses the challenges and problems faced when writing the
7958 Ascension tool. It also takes a look at possible improvements in the
7961 Consider the following diagram that shows the workflow of Ascension:
7963 @image{images/ascension_ssd,6in,,Ascensions workflow}
7965 Further the interaction between components of GNUnet are shown in the diagram
7967 @center @image{images/ascension_interaction,,6in,Ascensions workflow}
7970 * Conversions between DNS and GNS::
7975 @cindex DNS Conversion
7976 @node Conversions between DNS and GNS
7977 @subsubsection Conversions between DNS and GNS
7979 The differences between the two name systems lies in the details and is not
7980 always transparent. For instance an SRV record is converted to a BOX record
7981 which is unique to GNS.
7983 This is done by converting to a BOX record from an existing SRV record:
7987 # _service._proto.name. TTL class SRV priority weight port target
7988 _sip._tcp.example.com. 14000 IN SRV 0 0 5060 www.example.com.
7990 # TTL BOX flags port protocol recordtype priority weight port target
7991 14000 BOX n 5060 6 33 0 0 5060 www.example.com
7994 Other records that need to undergo such transformation is the MX record type,
7995 as well as the SOA record type.
7997 Transformation of a SOA record into GNS works as described in the
7998 following example. Very important to note are the rname and mname keys.
8001 # BIND syntax for a clean SOA record
8002 @ IN SOA master.example.com. hostmaster.example.com. (
8008 # Recordline for adding the record
8009 $ gnunet-namestore -z example.com -a -n @ -t SOA -V \
8010 rname=master.example.com mname=hostmaster.example.com \
8011 2017030300,3600,1800,604800,600 -e 7200s
8014 The transformation of MX records is done in a simple way.
8016 # mail.example.com. 3600 IN MX 10 mail.example.com.
8017 $ gnunet-namestore -z example.com -n mail -R 3600 MX n 10,mail
8020 Finally, one of the biggest struggling points were the NS records that are
8021 found in top level domain zones. The intended behaviour for those is to add
8022 GNS2DNS records for those so that gnunet-gns can resolve records for those
8023 domains on its own. Those require the values from DNS GLUE records, provided
8024 they are within the same zone.
8026 The following two examples show one record with a GLUE record and the other one
8027 does not have a GLUE record. This takes place in the 'com' TLD.
8030 # ns1.example.com 86400 IN A 127.0.0.1
8031 # example.com 86400 IN NS ns1.example.com.
8032 $ gnunet-namestore -z com -n example -R 86400 GNS2DNS n \
8033 example.com@@127.0.0.1
8035 # example.com 86400 IN NS ns1.example.org.
8036 $ gnunet-namestore -z com -n example -R 86400 GNS2DNS n \
8037 example.com@@ns1.example.org
8040 As you can see, one of the GNS2DNS records has an IP address listed and the
8041 other one a DNS name. For the first one there is a GLUE record to do the
8042 translation directly and the second one will issue another DNS query to figure
8043 out the IP of ns1.example.org.
8045 A solution was found by creating a hierarchical zone structure in GNS and linking
8046 the zones using PKEY records to one another. This allows the resolution of the
8047 name servers to work within GNS while not taking control over unwanted zones.
8049 Currently the following record types are supported:
8060 This is not due to technical limitations but rather a practical ones. The
8061 problem occurs with DNSSEC enabled DNS zones. As records within those zones are
8062 signed periodically, and every new signature is an update to the zone, there are
8063 many revisions of zones. This results in a problem with bigger zones as there
8064 are lots of records that have been signed again but no major changes. Also
8065 trying to add records that are unknown that require a different format take time
8066 as they cause a CLI call of the namestore. Furthermore certain record types
8067 need transformation into a GNS compatible format which, depending on the record
8068 type, takes more time.
8070 Further a blacklist was added to drop for instance DNSSEC related records. Also
8071 if a record type is neither in the white list nor the blacklist it is considered
8072 as a loss of data and a message is shown to the user. This helps with
8073 transparency and also with contributing, as the not supported record types can
8074 then be added accordingly.
8077 @subsubsection DNS Zone Size
8078 Another very big problem exists with very large zones. When migrating a small
8079 zone the delay between adding of records and their expiry is negligible. However
8080 when working with big zones that easily have more than a few million records
8081 this delay becomes a problem.
8083 Records will start to expire well before the zone has finished migrating. This
8084 is usually not a problem but can cause a high CPU load when a peer is restarted
8085 and the records have expired.
8087 A good solution has not been found yet. One of the idea that floated around was
8088 that the records should be added with the s (shadow) flag to keep the records
8089 resolvable even if they expired. However this would introduce the problem of how
8090 to detect if a record has been removed from the zone and would require deletion
8093 Another problem that still persists is how to refresh records. Expired records
8094 are still displayed when calling gnunet-namestore but do not resolve with
8095 gnunet-gns. Zonemaster will sign the expired records again and make sure that
8096 the records are still valid. With a recent change this was fixed as gnunet-gns
8097 to improve the suffix lookup which allows for a fast lookup even with thousands
8100 Currently the pace of adding records in general is around 10 records per second.
8101 Crypto is the upper limit for adding of records. The performance of your machine
8102 can be tested with the perf_crypto_* tools. There is still a big discrepancy
8103 between the pace of Ascension and the theoretical limit.
8105 A performance metric for measuring improvements has not yet been implemented in
8109 @subsubsection Performance
8110 The performance when migrating a zone using the Ascension tool is limited by a
8111 handful of factors. First of all ascension is written in Python3 and calls the
8112 CLI tools of GNUnet. This is comparable to a fork and exec call which costs a
8113 few CPU cycles. Furthermore all the records that are added to the same
8114 label are signed using the zones private key. This signing operation is very
8115 resource heavy and was optimized during development by adding the '-R'
8116 (Recordline) option to gnunet-namestore which allows to specify multiple records
8117 using the CLI tool. Assuming that in a TLD zone every domain has at least two
8118 name servers this halves the amount of signatures needed.
8120 Another improvement that could be made is with the addition of multiple threads
8121 or using asynchronous subprocesses when opening the GNUnet CLI tools. This could
8122 be implemented by simply creating more workers in the program but performance
8123 improvements were not tested.
8125 Ascension was tested using different hardware and database backends. Performance
8126 differences between SQLite and postgresql are marginal and almost non existent.
8127 What did make a huge impact on record adding performance was the storage medium.
8128 On a traditional mechanical hard drive adding of records were slow compared to a
8131 In conclusion there are many bottlenecks still around in the program, namely the
8132 single threaded implementation and inefficient, sequential calls of
8133 gnunet-namestore. In the future a solution that uses the C API would be cleaner
8136 @cindex GNS Namecache
8138 @section GNS Namecache
8140 The NAMECACHE subsystem is responsible for caching (encrypted) resolution
8141 results of the GNU Name System (GNS). GNS makes zone information available
8142 to other users via the DHT. However, as accessing the DHT for every
8143 lookup is expensive (and as the DHT's local cache is lost whenever the
8144 peer is restarted), GNS uses the NAMECACHE as a more persistent cache for
8146 Thus, instead of always looking up every name in the DHT, GNS first
8147 checks if the result is already available locally in the NAMECACHE.
8148 Only if there is no result in the NAMECACHE, GNS queries the DHT.
8149 The NAMECACHE stores data in the same (encrypted) format as the DHT.
8150 It thus makes no sense to iterate over all items in the
8151 NAMECACHE --- the NAMECACHE does not have a way to provide the keys
8152 required to decrypt the entries.
8154 Blocks in the NAMECACHE share the same expiration mechanism as blocks in
8155 the DHT --- the block expires wheneever any of the records in
8156 the (encrypted) block expires.
8157 The expiration time of the block is the only information stored in
8158 plaintext. The NAMECACHE service internally performs all of the required
8159 work to expire blocks, clients do not have to worry about this.
8160 Also, given that NAMECACHE stores only GNS blocks that local users
8161 requested, there is no configuration option to limit the size of the
8162 NAMECACHE. It is assumed to be always small enough (a few MB) to fit on
8165 The NAMECACHE supports the use of different database backends via a
8169 * libgnunetnamecache::
8170 * The NAMECACHE Client-Service Protocol::
8171 * The NAMECACHE Plugin API::
8174 @node libgnunetnamecache
8175 @subsection libgnunetnamecache
8179 The NAMECACHE API consists of five simple functions. First, there is
8180 @code{GNUNET_NAMECACHE_connect} to connect to the NAMECACHE service.
8181 This returns the handle required for all other operations on the
8182 NAMECACHE. Using @code{GNUNET_NAMECACHE_block_cache} clients can insert a
8183 block into the cache.
8184 @code{GNUNET_NAMECACHE_lookup_block} can be used to lookup blocks that
8185 were stored in the NAMECACHE. Both operations can be canceled using
8186 @code{GNUNET_NAMECACHE_cancel}. Note that canceling a
8187 @code{GNUNET_NAMECACHE_block_cache} operation can result in the block
8188 being stored in the NAMECACHE --- or not. Cancellation primarily ensures
8189 that the continuation function with the result of the operation will no
8191 Finally, @code{GNUNET_NAMECACHE_disconnect} closes the connection to the
8194 The maximum size of a block that can be stored in the NAMECACHE is
8195 @code{GNUNET_NAMECACHE_MAX_VALUE_SIZE}, which is defined to be 63 kB.
8197 @node The NAMECACHE Client-Service Protocol
8198 @subsection The NAMECACHE Client-Service Protocol
8202 All messages in the NAMECACHE IPC protocol start with the
8203 @code{struct GNUNET_NAMECACHE_Header} which adds a request
8204 ID (32-bit integer) to the standard message header.
8205 The request ID is used to match requests with the
8206 respective responses from the NAMECACHE, as they are allowed to happen
8216 @subsubsection Lookup
8220 The @code{struct LookupBlockMessage} is used to lookup a block stored in
8222 It contains the query hash. The NAMECACHE always responds with a
8223 @code{struct LookupBlockResponseMessage}. If the NAMECACHE has no
8224 response, it sets the expiration time in the response to zero.
8225 Otherwise, the response is expected to contain the expiration time, the
8226 ECDSA signature, the derived key and the (variable-size) encrypted data
8230 @subsubsection Store
8234 The @code{struct BlockCacheMessage} is used to cache a block in the
8236 It has the same structure as the @code{struct LookupBlockResponseMessage}.
8237 The service responds with a @code{struct BlockCacheResponseMessage} which
8238 contains the result of the operation (success or failure).
8239 In the future, we might want to make it possible to provide an error
8242 @node The NAMECACHE Plugin API
8243 @subsection The NAMECACHE Plugin API
8246 The NAMECACHE plugin API consists of two functions, @code{cache_block} to
8247 store a block in the database, and @code{lookup_block} to lookup a block
8257 @subsubsection Lookup2
8261 The @code{lookup_block} function is expected to return at most one block
8262 to the iterator, and return @code{GNUNET_NO} if there were no non-expired
8264 If there are multiple non-expired results in the cache, the lookup is
8265 supposed to return the result with the largest expiration time.
8268 @subsubsection Store2
8272 The @code{cache_block} function is expected to try to store the block in
8273 the database, and return @code{GNUNET_SYSERR} if this was not possible
8275 Furthermore, @code{cache_block} is expected to implicitly perform cache
8276 maintenance and purge blocks from the cache that have expired. Note that
8277 @code{cache_block} might encounter the case where the database already has
8278 another block stored under the same key. In this case, the plugin must
8279 ensure that the block with the larger expiration time is preserved.
8280 Obviously, this can done either by simply adding new blocks and selecting
8281 for the most recent expiration time during lookup, or by checking which
8282 block is more recent during the store operation.
8284 @cindex REVOCATION Subsystem
8285 @node REVOCATION Subsystem
8286 @section REVOCATION Subsystem
8289 The REVOCATION subsystem is responsible for key revocation of Egos.
8290 If a user learns that theis private key has been compromised or has lost
8291 it, they can use the REVOCATION system to inform all of the other users
8292 that their private key is no longer valid.
8293 The subsystem thus includes ways to query for the validity of keys and to
8294 propagate revocation messages.
8298 * Revocation Message Design Requirements::
8299 * libgnunetrevocation::
8300 * The REVOCATION Client-Service Protocol::
8301 * The REVOCATION Peer-to-Peer Protocol::
8305 @subsection Dissemination
8309 When a revocation is performed, the revocation is first of all
8310 disseminated by flooding the overlay network.
8311 The goal is to reach every peer, so that when a peer needs to check if a
8312 key has been revoked, this will be purely a local operation where the
8313 peer looks at its local revocation list. Flooding the network is also the
8314 most robust form of key revocation --- an adversary would have to control
8315 a separator of the overlay graph to restrict the propagation of the
8316 revocation message. Flooding is also very easy to implement --- peers that
8317 receive a revocation message for a key that they have never seen before
8318 simply pass the message to all of their neighbours.
8320 Flooding can only distribute the revocation message to peers that are
8322 In order to notify peers that join the network later, the revocation
8323 service performs efficient set reconciliation over the sets of known
8324 revocation messages whenever two peers (that both support REVOCATION
8325 dissemination) connect.
8326 The SET service is used to perform this operation efficiently.
8328 @node Revocation Message Design Requirements
8329 @subsection Revocation Message Design Requirements
8333 However, flooding is also quite costly, creating O(|E|) messages on a
8334 network with |E| edges.
8335 Thus, revocation messages are required to contain a proof-of-work, the
8336 result of an expensive computation (which, however, is cheap to verify).
8337 Only peers that have expended the CPU time necessary to provide
8338 this proof will be able to flood the network with the revocation message.
8339 This ensures that an attacker cannot simply flood the network with
8340 millions of revocation messages. The proof-of-work required by GNUnet is
8341 set to take days on a typical PC to compute; if the ability to quickly
8342 revoke a key is needed, users have the option to pre-compute revocation
8343 messages to store off-line and use instantly after their key has expired.
8345 Revocation messages must also be signed by the private key that is being
8346 revoked. Thus, they can only be created while the private key is in the
8347 possession of the respective user. This is another reason to create a
8348 revocation message ahead of time and store it in a secure location.
8350 @node libgnunetrevocation
8351 @subsection libgnunetrevocation
8355 The REVOCATION API consists of two parts, to query and to issue
8360 * Querying for revoked keys::
8361 * Preparing revocations::
8362 * Issuing revocations::
8365 @node Querying for revoked keys
8366 @subsubsection Querying for revoked keys
8370 @code{GNUNET_REVOCATION_query} is used to check if a given ECDSA public
8371 key has been revoked.
8372 The given callback will be invoked with the result of the check.
8373 The query can be canceled using @code{GNUNET_REVOCATION_query_cancel} on
8376 @node Preparing revocations
8377 @subsubsection Preparing revocations
8381 It is often desirable to create a revocation record ahead-of-time and
8382 store it in an off-line location to be used later in an emergency.
8383 This is particularly true for GNUnet revocations, where performing the
8384 revocation operation itself is computationally expensive and thus is
8385 likely to take some time.
8386 Thus, if users want the ability to perform revocations quickly in an
8387 emergency, they must pre-compute the revocation message.
8388 The revocation API enables this with two functions that are used to
8389 compute the revocation message, but not trigger the actual revocation
8392 @code{GNUNET_REVOCATION_check_pow} should be used to calculate the
8393 proof-of-work required in the revocation message. This function takes the
8394 public key, the required number of bits for the proof of work (which in
8395 GNUnet is a network-wide constant) and finally a proof-of-work number as
8397 The function then checks if the given proof-of-work number is a valid
8398 proof of work for the given public key. Clients preparing a revocation
8399 are expected to call this function repeatedly (typically with a
8400 monotonically increasing sequence of numbers of the proof-of-work number)
8401 until a given number satisfies the check.
8402 That number should then be saved for later use in the revocation
8405 @code{GNUNET_REVOCATION_sign_revocation} is used to generate the
8406 signature that is required in a revocation message.
8407 It takes the private key that (possibly in the future) is to be revoked
8408 and returns the signature.
8409 The signature can again be saved to disk for later use, which will then
8410 allow performing a revocation even without access to the private key.
8412 @node Issuing revocations
8413 @subsubsection Issuing revocations
8416 Given a ECDSA public key, the signature from @code{GNUNET_REVOCATION_sign}
8417 and the proof-of-work,
8418 @code{GNUNET_REVOCATION_revoke} can be used to perform the
8419 actual revocation. The given callback is called upon completion of the
8420 operation. @code{GNUNET_REVOCATION_revoke_cancel} can be used to stop the
8421 library from calling the continuation; however, in that case it is
8422 undefined whether or not the revocation operation will be executed.
8424 @node The REVOCATION Client-Service Protocol
8425 @subsection The REVOCATION Client-Service Protocol
8428 The REVOCATION protocol consists of four simple messages.
8430 A @code{QueryMessage} containing a public ECDSA key is used to check if a
8431 particular key has been revoked. The service responds with a
8432 @code{QueryResponseMessage} which simply contains a bit that says if the
8433 given public key is still valid, or if it has been revoked.
8435 The second possible interaction is for a client to revoke a key by
8436 passing a @code{RevokeMessage} to the service. The @code{RevokeMessage}
8437 contains the ECDSA public key to be revoked, a signature by the
8438 corresponding private key and the proof-of-work, The service responds
8439 with a @code{RevocationResponseMessage} which can be used to indicate
8440 that the @code{RevokeMessage} was invalid (i.e. proof of work incorrect),
8441 or otherwise indicates that the revocation has been processed
8444 @node The REVOCATION Peer-to-Peer Protocol
8445 @subsection The REVOCATION Peer-to-Peer Protocol
8449 Revocation uses two disjoint ways to spread revocation information among
8451 First of all, P2P gossip exchanged via CORE-level neighbours is used to
8452 quickly spread revocations to all connected peers.
8453 Second, whenever two peers (that both support revocations) connect,
8454 the SET service is used to compute the union of the respective revocation
8457 In both cases, the exchanged messages are @code{RevokeMessage}s which
8458 contain the public key that is being revoked, a matching ECDSA signature,
8459 and a proof-of-work.
8460 Whenever a peer learns about a new revocation this way, it first
8461 validates the signature and the proof-of-work, then stores it to disk
8462 (typically to a file $GNUNET_DATA_HOME/revocation.dat) and finally
8463 spreads the information to all directly connected neighbours.
8465 For computing the union using the SET service, the peer with the smaller
8466 hashed peer identity will connect (as a "client" in the two-party set
8467 protocol) to the other peer after one second (to reduce traffic spikes
8468 on connect) and initiate the computation of the set union.
8469 All revocation services use a common hash to identify the SET operation
8470 over revocation sets.
8472 The current implementation accepts revocation set union operations from
8473 all peers at any time; however, well-behaved peers should only initiate
8474 this operation once after establishing a connection to a peer with a
8475 larger hashed peer identity.
8478 @cindex FS Subsystem
8479 @node File-sharing (FS) Subsystem
8480 @section File-sharing (FS) Subsystem
8484 This chapter describes the details of how the file-sharing service works.
8485 As with all services, it is split into an API (libgnunetfs), the service
8486 process (gnunet-service-fs) and user interface(s).
8487 The file-sharing service uses the datastore service to store blocks and
8488 the DHT (and indirectly datacache) for lookups for non-anonymous
8490 Furthermore, the file-sharing service uses the block library (and the
8491 block fs plugin) for validation of DHT operations.
8493 In contrast to many other services, libgnunetfs is rather complex since
8494 the client library includes a large number of high-level abstractions;
8495 this is necessary since the Fs service itself largely only operates on
8497 The FS library is responsible for providing a file-based abstraction to
8498 applications, including directories, meta data, keyword search,
8499 verification, and so on.
8501 The method used by GNUnet to break large files into blocks and to use
8502 keyword search is called the
8503 "Encoding for Censorship Resistant Sharing" (ECRS).
8504 ECRS is largely implemented in the fs library; block validation is also
8505 reflected in the block FS plugin and the FS service.
8506 ECRS on-demand encoding is implemented in the FS service.
8508 NOTE: The documentation in this chapter is quite incomplete.
8511 * Encoding for Censorship-Resistant Sharing (ECRS)::
8512 * File-sharing persistence directory structure::
8516 @cindex Encoding for Censorship-Resistant Sharing
8517 @node Encoding for Censorship-Resistant Sharing (ECRS)
8518 @subsection Encoding for Censorship-Resistant Sharing (ECRS)
8522 When GNUnet shares files, it uses a content encoding that is called ECRS,
8523 the Encoding for Censorship-Resistant Sharing.
8524 Most of ECRS is described in the (so far unpublished) research paper
8525 attached to this page. ECRS obsoletes the previous ESED and ESED II
8526 encodings which were used in GNUnet before version 0.7.0.
8527 The rest of this page assumes that the reader is familiar with the
8528 attached paper. What follows is a description of some minor extensions
8529 that GNUnet makes over what is described in the paper.
8530 The reason why these extensions are not in the paper is that we felt
8531 that they were obvious or trivial extensions to the original scheme and
8532 thus did not warrant space in the research report.
8535 * Namespace Advertisements::
8539 @node Namespace Advertisements
8540 @subsubsection Namespace Advertisements
8543 @c %**FIXME: all zeroses -> ?
8545 An @code{SBlock} with identifier all zeros is a signed
8546 advertisement for a namespace. This special @code{SBlock} contains
8547 metadata describing the content of the namespace.
8548 Instead of the name of the identifier for a potential update, it contains
8549 the identifier for the root of the namespace.
8550 The URI should always be empty. The @code{SBlock} is signed with the
8551 content provider's RSA private key (just like any other SBlock). Peers
8552 can search for @code{SBlock}s in order to find out more about a namespace.
8555 @subsubsection KSBlocks
8559 GNUnet implements @code{KSBlocks} which are @code{KBlocks} that, instead
8560 of encrypting a CHK and metadata, encrypt an @code{SBlock} instead.
8561 In other words, @code{KSBlocks} enable GNUnet to find @code{SBlocks}
8562 using the global keyword search.
8563 Usually the encrypted @code{SBlock} is a namespace advertisement.
8564 The rationale behind @code{KSBlock}s and @code{SBlock}s is to enable
8565 peers to discover namespaces via keyword searches, and, to associate
8566 useful information with namespaces. When GNUnet finds @code{KSBlocks}
8567 during a normal keyword search, it adds the information to an internal
8568 list of discovered namespaces. Users looking for interesting namespaces
8569 can then inspect this list, reducing the need for out-of-band discovery
8571 Naturally, namespaces (or more specifically, namespace advertisements) can
8572 also be referenced from directories, but @code{KSBlock}s should make it
8573 easier to advertise namespaces for the owner of the pseudonym since they
8574 eliminate the need to first create a directory.
8576 Collections are also advertised using @code{KSBlock}s.
8578 @c https://old.gnunet.org/sites/default/files/ecrs.pdf
8580 @node File-sharing persistence directory structure
8581 @subsection File-sharing persistence directory structure
8585 This section documents how the file-sharing library implements
8586 persistence of file-sharing operations and specifically the resulting
8587 directory structure.
8588 This code is only active if the @code{GNUNET_FS_FLAGS_PERSISTENCE} flag
8589 was set when calling @code{GNUNET_FS_start}.
8590 In this case, the file-sharing library will try hard to ensure that all
8591 major operations (searching, downloading, publishing, unindexing) are
8592 persistent, that is, can live longer than the process itself.
8593 More specifically, an operation is supposed to live until it is
8596 If @code{GNUNET_FS_stop} is called before an operation has been stopped, a
8597 @code{SUSPEND} event is generated and then when the process calls
8598 @code{GNUNET_FS_start} next time, a @code{RESUME} event is generated.
8599 Additionally, even if an application crashes (segfault, SIGKILL, system
8600 crash) and hence @code{GNUNET_FS_stop} is never called and no
8601 @code{SUSPEND} events are generated, operations are still resumed (with
8602 @code{RESUME} events).
8603 This is implemented by constantly writing the current state of the
8604 file-sharing operations to disk.
8605 Specifically, the current state is always written to disk whenever
8606 anything significant changes (the exception are block-wise progress in
8607 publishing and unindexing, since those operations would be slowed down
8608 significantly and can be resumed cheaply even without detailed
8610 Note that if the process crashes (or is killed) during a serialization
8611 operation, FS does not guarantee that this specific operation is
8612 recoverable (no strict transactional semantics, again for performance
8613 reasons). However, all other unrelated operations should resume nicely.
8615 Since we need to serialize the state continuously and want to recover as
8616 much as possible even after crashing during a serialization operation,
8617 we do not use one large file for serialization.
8618 Instead, several directories are used for the various operations.
8619 When @code{GNUNET_FS_start} executes, the master directories are scanned
8620 for files describing operations to resume.
8621 Sometimes, these operations can refer to related operations in child
8622 directories which may also be resumed at this point.
8623 Note that corrupted files are cleaned up automatically.
8624 However, dangling files in child directories (those that are not
8625 referenced by files from the master directories) are not automatically
8628 Persistence data is kept in a directory that begins with the "STATE_DIR"
8629 prefix from the configuration file
8630 (by default, "$SERVICEHOME/persistence/") followed by the name of the
8631 client as given to @code{GNUNET_FS_start} (for example, "gnunet-gtk")
8632 followed by the actual name of the master or child directory.
8634 The names for the master directories follow the names of the operations:
8643 Each of the master directories contains names (chosen at random) for each
8644 active top-level (master) operation.
8645 Note that a download that is associated with a search result is not a
8646 top-level operation.
8648 In contrast to the master directories, the child directories are only
8649 consulted when another operation refers to them.
8650 For each search, a subdirectory (named after the master search
8651 synchronization file) contains the search results.
8652 Search results can have an associated download, which is then stored in
8653 the general "download-child" directory.
8654 Downloads can be recursive, in which case children are stored in
8655 subdirectories mirroring the structure of the recursive download
8656 (either starting in the master "download" directory or in the
8657 "download-child" directory depending on how the download was initiated).
8658 For publishing operations, the "publish-file" directory contains
8659 information about the individual files and directories that are part of
8661 However, this directory structure is flat and does not mirror the
8662 structure of the publishing operation.
8663 Note that unindex operations cannot have associated child operations.
8665 @cindex REGEX subsystem
8666 @node REGEX Subsystem
8667 @section REGEX Subsystem
8671 Using the REGEX subsystem, you can discover peers that offer a particular
8672 service using regular expressions.
8673 The peers that offer a service specify it using a regular expressions.
8674 Peers that want to patronize a service search using a string.
8675 The REGEX subsystem will then use the DHT to return a set of matching
8676 offerers to the patrons.
8678 For the technical details, we have Max's defense talk and Max's Master's
8681 @c An additional publication is under preparation and available to
8682 @c team members (in Git).
8683 @c FIXME: Where is the file? Point to it. Assuming that it's szengel2012ms
8686 * How to run the regex profiler::
8689 @node How to run the regex profiler
8690 @subsection How to run the regex profiler
8694 The gnunet-regex-profiler can be used to profile the usage of mesh/regex
8695 for a given set of regular expressions and strings.
8696 Mesh/regex allows you to announce your peer ID under a certain regex and
8697 search for peers matching a particular regex using a string.
8698 See @uref{https://old.gnunet.org/szengel2012ms, szengel2012ms} for a full
8701 First of all, the regex profiler uses GNUnet testbed, thus all the
8702 implications for testbed also apply to the regex profiler
8703 (for example you need password-less ssh login to the machines listed in
8706 @strong{Configuration}
8708 Moreover, an appropriate configuration file is needed.
8709 Generally you can refer to the
8710 @file{contrib/regex_profiler_infiniband.conf} file in the sourcecode
8711 of GNUnet for an example configuration.
8712 In the following paragraph the important details are highlighted.
8714 Announcing of the regular expressions is done by the
8715 gnunet-daemon-regexprofiler, therefore you have to make sure it is
8716 started, by adding it to the START_ON_DEMAND set of ARM:
8720 START_ON_DEMAND = YES
8724 Furthermore you have to specify the location of the binary:
8728 # Location of the gnunet-daemon-regexprofiler binary.
8729 BINARY = /home/szengel/gnunet/src/mesh/.libs/gnunet-daemon-regexprofiler
8730 # Regex prefix that will be applied to all regular expressions and
8732 REGEX_PREFIX = "GNVPN-0001-PAD"
8736 When running the profiler with a large scale deployment, you probably
8737 want to reduce the workload of each peer.
8738 Use the following options to do this.
8742 # Force network size estimation
8747 # Disable RC-file for Bloom filter? (for benchmarking with limited IO
8750 # Disable Bloom filter entirely
8754 # Minimize proof-of-work CPU consumption by NSE
8761 To finally run the profiler some options and the input data need to be
8762 specified on the command line.
8765 gnunet-regex-profiler -c config-file -d log-file -n num-links \
8766 -p path-compression-length -s search-delay -t matching-timeout \
8767 -a num-search-strings hosts-file policy-dir search-strings-file
8774 @item ... @code{config-file} means the configuration file created earlier.
8775 @item ... @code{log-file} is the file where to write statistics output.
8776 @item ... @code{num-links} indicates the number of random links between
8778 @item ... @code{path-compression-length} is the maximum path compression
8780 @item ... @code{search-delay} time to wait between peers finished linking
8781 and starting to match strings.
8782 @item ... @code{matching-timeout} timeout after which to cancel the
8784 @item ... @code{num-search-strings} number of strings in the
8785 search-strings-file.
8786 @item ... the @code{hosts-file} should contain a list of hosts for the
8787 testbed, one per line in the following format:
8790 @item @code{user@@host_ip:port}
8792 @item ... the @code{policy-dir} is a folder containing text files
8793 containing one or more regular expressions. A peer is started for each
8794 file in that folder and the regular expressions in the corresponding file
8795 are announced by this peer.
8796 @item ... the @code{search-strings-file} is a text file containing search
8797 strings, one in each line.
8801 You can create regular expressions and search strings for every AS in the
8802 Internet using the attached scripts. You need one of the
8803 @uref{http://data.caida.org/datasets/routing/routeviews-prefix2as/, CAIDA routeviews prefix2as}
8804 data files for this. Run
8807 create_regex.py <filename> <output path>
8811 to create the regular expressions and
8814 create_strings.py <input path> <outfile>
8818 to create a search strings file from the previously created
8819 regular expressions.
8821 @cindex REST subsystem
8822 @node REST Subsystem
8823 @section REST Subsystem
8827 Using the REST subsystem, you can expose REST-based APIs or services.
8828 The REST service is designed as a pluggable architecture.
8829 To create a new REST endpoint, simply add a library in the form
8831 The REST service will automatically load all REST plugins on startup.
8833 @strong{Configuration}
8835 The REST service can be configured in various ways.
8836 The reference config file can be found in
8837 @file{src/rest/rest.conf}:
8841 REST_ALLOW_HEADERS=Authorization,Accept,Content-Type
8843 REST_ALLOW_CREDENTIALS=true
8847 @deffn{cross-origin resource sharing} (CORS)
8849 headers that are supposed to be advertised by the rest service are
8853 * Namespace considerations::
8854 * Endpoint documentation::
8857 @node Namespace considerations
8858 @subsection Namespace considerations
8860 The @command{gnunet-rest-service} will load all plugins that are installed.
8861 As such it is important that the endpoint namespaces do not clash.
8863 For example, plugin X might expose the endpoint ``/xxx'' while plugin Y
8864 exposes endpoint ``/xxx/yyy''.
8865 This is a problem if plugin X is also supposed to handle a call
8867 Currently the REST service will not complain or warn about such clashes,
8868 so please make sure that endpoints are unambiguous.
8870 @node Endpoint documentation
8871 @subsection Endpoint documentation
8873 This is WIP. Endpoints should be documented appropriately.
8874 Preferably using annotations.
8877 @cindex RPS Subsystem
8879 @section RPS Subsystem
8881 In literature, Random Peer Sampling (RPS) refers to the problem of
8882 reliably drawing random samples from an unstructured p2p network.
8884 Doing so in a reliable manner is not only hard because of inherent
8885 problems but also because of possible malicious peers that could try to
8888 It is useful for all kind of gossip protocols that require the selection
8889 of random peers in the whole network like gathering statistics,
8890 spreading and aggregating information in the network, load balancing and
8891 overlay topology management.
8893 The approach chosen in the rps implementation in GNUnet follows the
8894 Brahms@uref{https://bib.gnunet.org/full/date.html\#2009_5f0} design.
8896 The current state is "work in progress". There are a lot of things that
8897 need to be done, primarily finishing the experimental evaluation and a
8898 re-design of the API.
8900 The abstract idea is to subscribe to connect to/start the rps service
8901 and request random peers that will be returned when they represent a
8902 random selection from the whole network with high probability.
8904 An additional feature to the original Brahms-design is the selection of
8905 sub-groups: The GNUnet implementation of rps enables clients to ask for
8906 random peers from a group that is defined by a common shared secret.
8907 (The secret could of course also be public, depending on the use-case.)
8909 Another addition to the original protocol was made: The sampler
8910 mechanism that was introduced in Brahms was slightly adapted and used to
8911 actually sample the peers and returned to the client.
8912 This is necessary as the original design only keeps peers connected to
8913 random other peers in the network. In order to return random peers to
8914 client requests independently random, they cannot be drawn from the
8916 The adapted sampler makes sure that each request for random peers is
8917 independent from the others.
8921 The high-level concept of Brahms is two-fold: Combining push-pull gossip
8922 with locally fixing a assumed bias using cryptographic min-wise
8924 The central data structure is the view - a peer's current local sample.
8925 This view is used to select peers to push to and pull from.
8926 This simple mechanism can be biased easily. For this reason Brahms
8927 'fixes' the bias by using the so-called sampler. A data structure that
8928 takes a list of elements as input and outputs a random one of them
8929 independently of the frequency in the input set. Both an element that
8930 was put into the sampler a single time and an element that was put into
8931 it a million times have the same probability of being the output.
8932 This is achieved this is achieved with exploiting min-wise independent
8933 permutations. In rps we use HMACs: On the initialisation of a sampler
8934 element, a key is chosen at random. On each input the HMAC with the
8935 random key is computed. The sampler element keeps the element with the
8938 In order to fix the bias in the view, a fraction of the elements in the
8939 view are sampled through the sampler from the random stream of peer IDs.
8941 According to the theoretical analysis of Bortnikov et al. this suffices
8942 to keep the network connected and having random peers in the view.