This book is intended to be an introduction for programmers that want to
extend the GNUnet framework. GNUnet is more than a simple peer-to-peer
-application. For developers, GNUnet is:
+application.
+
+For developers, GNUnet is:
@itemize @bullet
-@item Free software under the GNU General Public License, with a community
-that believes in the GNU philosophy
+@item developed by a community that believes in the GNU philosophy
+@item Free Software (Free as in Freedom), licensed under the
+@xref{GNU General Public License}
@item A set of standards, including coding conventions and
architectural rules
@item A set of layered protocols, both specifying the communication
In particular, the architecture specifies that a peer consists of many
processes communicating via protocols. Processes can be written in almost
-any language. C and Java APIs exist for accessing existing services and
-for writing extensions. It is possible to write extensions in other
-languages by implementing the necessary IPC protocols.
+any language.
+C and Java APIs exist for accessing existing services and for writing
+extensions. It is possible to write extensions in other languages by
+implementing the necessary IPC protocols.
GNUnet can be extended and improved along many possible dimensions, and
anyone interested in Free Software and Freedom-enhancing Networking is
welcome to join the effort. This Developer Handbook attempts to provide
an initial introduction to some of the key design choices and central
-components of the system. This part of the GNUNet documentation
-is far from complete, and we welcome informed contributions,
-be it in the form of new chapters or insightful comments.
+components of the system.
+This part of the GNUNet documentation is far from complete,
+and we welcome informed contributions, be it in the form of
+new chapters, sections or insightful comments.
@menu
* Developer Introduction::
@item GNUnet Java tutorial
@end itemize
-In addition to this book, the GNUnet server contains various resources for
-GNUnet developers. They are all conveniently reachable via the "Developer"
+In addition to the GNUnet Reference Documentation you are reading,
+the GNUnet server contains various resources for GNUnet
+developers and those who aspire to become regular contributors.
+They are all conveniently reachable via the "Developer"
entry in the navigation menu. Some additional tools (such as static
analysis reports) require a special developer access to perform certain
-operations. If you feel you need access, you should contact
+operations. If you want (or require) access, you should contact
@uref{http://grothoff.org/christian/, Christian Grothoff},
GNUnet's maintainer.
@itemize @bullet
-@item The Version control system (git) keeps our code and enables
+@item The version control system (git) keeps our code and enables
distributed development.
+It is pubclicly accessible at @uref{https://gnunet.org/git/}.
Only developers with write access can commit code, everyone else is
encouraged to submit patches to the
-@uref{https://lists.gnu.org/mailman/listinfo/gnunet-developers, GNUnet-developers mailinglist}
-.
-
-@item The GNUnet bugtracking system (Mantis) is used to track
-feature requests, open bug reports and their resolutions.
-Anyone can report bugs, only developers can claim to have fixed them.
-
-@item A site installation of the CI system "Buildbot" is used to check
-GNUnet builds automatically on a range of platforms.
-Builds are triggered automatically after 30 minutes of no changes to Git.
+@uref{https://lists.gnu.org/mailman/listinfo/gnunet-developers, GNUnet-developers mailinglist}.
+
+@item The bugtracking system (Mantis).
+We use it to track feature requests, open bug reports and their
+resolutions.
+It can be accessed at @uref{https://gnunet.org/bugs/}.
+Anyone can report bugs, but only developers can claim to have fixed them.
+
+@item Our site installation of the
+CI@footnote{Continuous Integration} system "@code{Buildbot}" is used
+to check GNUnet builds automatically on a range of platforms.
+The web interface of this CI is exposed at
+@uref{https://gnunet.org/buildbot/}.
+Builds are triggered automatically 30 minutes after the last commit to
+our repository was made.
@item The current quality of our automated test suite is assessed using
Code coverage analysis. This analysis is run daily; however the webpage
@table @asis
-@item gnunet
+@item @command{gnunet}
Core of the P2P framework, including file-sharing, VPN and
-chat applications; this is what the developer handbook covers mostly
-@item gnunet-gtk Gtk+-based user interfaces, including gnunet-fs-gtk
-(file-sharing), gnunet-statistics-gtk (statistics over time),
-gnunet-peerinfo-gtk (information about current connections and known
-peers), gnunet-chat-gtk (chat GUI) and gnunet-setup (setup tool for
-"everything")
-@item gnunet-fuse
+chat applications; this is what the Developer Handbook covers mostly
+@item @command{gnunet-gtk}
+Gtk+-based user interfaces, including:
+
+@itemize @bullet
+@item @command{gnunet-fs-gtk} (file-sharing),
+@item @command{gnunet-statistics-gtk} (statistics over time),
+@item @command{gnunet-peerinfo-gtk}
+(information about current connections and known peers),
+@item @command{gnunet-chat-gtk} (chat GUI) and
+@item @command{gnunet-setup} (setup tool for "everything")
+@end itemize
+
+@item @command{gnunet-fuse}
Mounting directories shared via GNUnet's file-sharing
-on Linux
-@item gnunet-update
+on GNU/Linux distributions
+@item @command{gnunet-update}
Installation and update tool
-@item gnunet-ext
+@item @command{gnunet-ext}
Template for starting 'external' GNUnet projects
-@item gnunet-java
+@item @command{gnunet-java}
Java APIs for writing GNUnet services and applications
@c ** FIXME: Point to new website repository once we have it:
@c ** @item svn/gnunet-www/ Code and media helping drive the GNUnet
@c website
-@item eclectic
+@item @command{eclectic}
Code to run GNUnet nodes on testbeds for research, development,
testing and evaluation
@c ** FIXME: Solve the status and location of gnunet-qt
-@item gnunet-qt
-Qt-based GNUnet GUI (dead?)
-@item gnunet-cocoa
-cocoa-based GNUnet GUI (dead?)
+@item @command{gnunet-qt}
+Qt-based GNUnet GUI (is it depreacated?)
+@item @command{gnunet-cocoa}
+cocoa-based GNUnet GUI (is it depreacated?)
@end table
@c ** FIXME: What about gauger, and what about libmwmodem?
@table @asis
-@item libextractor
+@item @command{libextractor}
GNU libextractor (meta data extraction)
-@item libmicrohttpd
+@item @command{libmicrohttpd}
GNU libmicrohttpd (embedded HTTP(S) server library)
-@item gauger
+@item @command{gauger}
Tool for performance regression analysis
-@item monkey
+@item @command{monkey}
Tool for automated debugging of distributed systems
-@item libmwmodem
+@item @command{libmwmodem}
Library for accessing satellite connection quality
reports
-@item libgnurl
-gnURL (feature restricted variant of cURL/libcurl)
+@item @command{libgnurl}
+gnURL (feature-restricted variant of cURL/libcurl)
@end table
Finally, there are various external projects (see links for a list of
linked to a hash code (the key for the DHT and for databases). The block
library is a wapper around block plugins which provide the necessary
functions for each block type.
-@item @file{statistics/}
+@item @file{statistics/} --- statistics service
The statistics service enables associating
values (of type uint64_t) with a componenet name and a string. The main
uses is debugging (counting events), performance tracking and user
is the GNUnet master service. Its role is to start gnunet-services, to
re-start them when they crashed and finally to shut down the system when
requested.
-@item @file{peerinfo/}
+@item @file{peerinfo/} --- peerinfo service
The peerinfo service keeps track of which peers are known
to the local peer and also tracks the validated addresses for each peer
(in the form of a HELLO message) for each of those peers. The peer is not
Existing plugins can store blocks in Sqlite, Postgres or MySQL databases.
All data stored in the cache is lost when the peer is stopped or
restarted (datacache uses temporary tables).
-@item @file{datastore/}
+@item @file{datastore/} --- datastore service
The datastore service stores file-sharing blocks in
databases for extended periods of time. In contrast to the datacache, data
is not lost when peers restart. However, quota restrictions may still
cause old, expired or low-priority data to be eventually discarded.
Existing plugins can store blocks in Sqlite, Postgres or MySQL databases.
-@item @file{template/}
+@item @file{template/} --- service template
Template for writing a new service. Does nothing.
@item @file{ats/} --- Automatic Transport Selection
-The automatic transport
-selection (ATS) service is responsible for deciding which address (i.e.
+The automatic transport selection (ATS) service
+is responsible for deciding which address (i.e.
which transport plugin) should be used for communication with other peers,
and at what bandwidth.
@item @file{nat/} --- libgnunetnat
break larger packets into chunks of at most 1k and transmit the resulting
fragments reliabily (with acknowledgement, retransmission, timeouts,
etc.).
-@item @file{transport/}
+@item @file{transport/} --- transport service
The transport service is responsible for managing the
basic P2P communication. It uses plugins to support P2P communication
over TCP, UDP, HTTP, HTTPS and other protocols.The transport service
validates peer addresses, enforces bandwidth restrictions, limits the
total number of connections and enforces connectivity restrictions (i.e.
friends-only).
-@item @file{peerinfo-tool/}
+@item @file{peerinfo-tool/} --- gnunet-peerinfo
This directory contains the gnunet-peerinfo binary which can be used to
inspect the peers and HELLOs known to the peerinfo service.
@item @file{core/}
It also supports automatic generation of configurations for peers
ensuring that the ports and paths are disjoint. libgnunettesting is also
the foundation for the testbed service
-@item @file{testbed/}
+@item @file{testbed/} --- testbed service
The testbed service is used for creating small or large scale deployments
of GNUnet peers for evaluation of protocols.
It facilitates peer depolyments on multiple
The distributed hash table (DHT) service provides a
distributed implementation of a hash table to store blocks under hash
keys in the P2P network.
-@item @file{hostlist/}
+@item @file{hostlist/} --- hostlist service
The hostlist service allows learning about
other peers in the network by downloading HELLO messages from an HTTP
server, can be configured to run such an HTTP server and also implements
a P2P protocol to advertise and automatically learn about other peers
that offer a public hostlist server.
-@item @file{topology/}
+@item @file{topology/} --- topology service
The topology service is responsible for
maintaining the mesh topology. It tries to maintain connections to friends
(depending on the configuration) and also tries to ensure that the peer
The file-sharing (FS) service implements GNUnet's
file-sharing application. Both anonymous file-sharing (using gap) and
non-anonymous file-sharing (using dht) are supported.
-@item @file{cadet/}
+@item @file{cadet/} --- cadet service
The CADET service provides a general-purpose routing abstraction to create
end-to-end encrypted tunnels in mesh networks. We wrote a paper
documenting key aspects of the design.
the local machine. Currently used for IPv4-IPv6 protocol translation
(DNS-ALG) as implemented by "pt/" and for the GNUnet naming system. The
service can also be configured to offer an exit service for DNS traffic.
-@item @file{vpn/}
+@item @file{vpn/} --- VPN service
The virtual public network (VPN) service provides a virtual
tunnel interface (VTUN) for IP routing over GNUnet.
Needs some other peers to run an "exit" service to work.
@node Coding style
@subsection Coding style
+@c XXX: Adjust examples to GNU Standards!
@itemize @bullet
@item We follow the GNU Coding Standards (@pxref{Top, The GNU Coding Standards,, standards, The GNU Coding Standards});
@item Indentation is done with spaces, two per level, no tabs;
@example
export GNUNET_TESTBED_RSH_CMD="ssh -o BatchMode=yes \
--o NoHostAuthenticationForLocalhost=yes %h"@
+-o NoHostAuthenticationForLocalhost=yes %h"
@end example
-Substitutions are allowed int the above command string also allows for
-substitions. through placemarks which begin with a `%'. At present the
-following substitutions are supported
+Substitutions are allowed in the command string above,
+this allows for substitutions through placemarks which begin with a `%'.
+At present the following substitutions are supported
@itemize @bullet
-@item
-%h: hostname
-@item
-%u: username
-@item
-%p: port
+@item %h: hostname
+@item %u: username
+@item %p: port
@end itemize
Note that the substitution placemark is replaced only when the
corresponding field is available and only once. Specifying
+
@example
%u@atchar{}%h
@end example
-doesn't work either.
-If you want to user username substitutions for SSH
-use the argument @code{-l} before the username substitution.
-For exmaple:
+
+doesn't work either. If you want to user username substitutions for
+@command{SSH}, use the argument @code{-l} before the
+username substitution.
+
+For example:
@example
ssh -l %u -p %p %h
@end example
@noindent
and rerun the @code{gpg --verify-files} command.
+@b{Note:}@
+@b{The pub key to sign the 0.10.1 release has been
+revoked}. You will get an error message stating that
+@i{there is no known public key or that it has been revoked}.
+The next release of GNUnet will have a valid signature
+again. We are sorry for the inconvenience this causes.
+Another possible source you could use is our
+"gnunet" git repository which has mandatory signed commits
+by every developer.
+
Now you can extract the tarball and rename the resulting
directory to @i{gnunet} which we will be using in the
remainder of this document.
@node Installing Build Tool Chain and Dependencies
@section Installing Build Tool Chain and Dependencies
-To successfully compile GNUnet you need the tools to build GNUnet and
-the required dependencies. Please have a look at
-@uref{https://gnunet.org/dependencies} for a list of required dependencies
-and @uref{https://gnunet.org/generic_installation} for specific
-instructions for your operating system. Please check the notes at
-the end of the configure process about required dependencies.
-
-For GNUnet bootstrapping support and the http(s) plugin you should
+To successfully compile GNUnet, you need the tools to build GNUnet and
+the required dependencies. Please take a look at the
+GNUnet Reference Documentation
+(@pxref{Dependencies, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation})
+for a list of required dependencies
+and
+(@pxref{Generic installation instructions, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation})
+read its Installation chapter for specific instructions for
+your operating system.
+Please check the notes at the end of the configure process about
+required dependencies.
+
+For GNUnet bootstrapping support and the HTTP(S) plugin you should
install @uref{https://gnunet.org/gnurl, libgnurl}.
For the filesharing service you should install at least one of the
datastore backends. MySQL, SQlite and PostgreSQL are supported.
@section Obtaining the latest version from Git
The latest development version can obtained from our Git repository.
-To obtain the code you need Git installed and checkout the repository
-using:
+To obtain the code you need to have @code{Git} installed, which is
+required for obtaining the repository via:
@example
$ git clone https://gnunet.org/git/gnunet
@noindent
After cloning the repository you have to execute the @file{bootstrap}
-script in the directory:
+script in the new directory:
@example
-$ cd gnunet ; ./bootstrap
+$ cd gnunet
+$ ./bootstrap
@end example
@noindent
-The remainder of this tutorial assumes that you have the Git branch
-``master'' checked out.
+The remainder of this tutorial will assume that you have the
+Git branch ``master'' checked out.
@node Compiling and Installing GNUnet
@section Compiling and Installing GNUnet
-First, you need to install at least libgnupgerror 1.27 and
-libgcrypt 1.7.6.
+Note: This section is a duplication of the more in depth
+@pxref{GNUnet Installation Handbook, The GNUnet Reference Documentation,, gnunet, The GNUnet Reference Documentation}.
+
+First, you need to install libgnupgerror @geq{} 1.27 and
+libgcrypt @geq{} 1.7.6:
@example
$ export GNUPGFTP="https://www.gnupg.org/ftp/gcrypt"
projects / oweals / gnunet.git / commitdiff