@c ***********************************************************************
@node Client - Establish connection
@subsubsection Client - Establish connection
-@c %**end of header
+
At first, on the client side, the underlying API is employed to create a
@c ***********************************************************************
@node Client - Initialize request message
@subsubsection Client - Initialize request message
-@c %**end of header
+
When the connection is ready, we initialize the message. In this step,
all the fields of the message should be properly initialized, namely the
@c ***********************************************************************
@node Client - Send request and receive response
@subsubsection Client - Send request and receive response
-@c %**end of header
+
@b{FIXME: This is very outdated, see the tutorial for the current API!}
@c ***********************************************************************
@node Server - Add new handles for specified messages
@subsubsection Server - Add new handles for specified messages
-@c %**end of header
+
in the function above the argument @code{run} is used to initiate
transport service,and defined like this:
@c ***********************************************************************
@node Server - Process request message
@subsubsection Server - Process request message
-@c %**end of header
+
After the initialization of transport service, the request message would
be processed. Before handling the main message data, the validity of this
@c ***********************************************************************
@node Server - Response to client
@subsubsection Server - Response to client
-@c %**end of header
+
Once the processing of current request is done, the server should give the
response to the client. A new @code{struct AddressLookupMessage} would be
@c ***********************************************************************
@node Server - Notification of clients
@subsubsection Server - Notification of clients
-@c %**end of header
+
Often a service needs to (repeatedly) transmit notifications to a client
or a group of clients. In these cases, the client typically has once
@node Conversion between Network Byte Order (Big Endian) and Host Byte Order
@subsubsection Conversion between Network Byte Order (Big Endian) and Host Byte Order
@c %** subsub? it's a referenced page on the ipc document.
-@c %**end of header
+
Here we can simply comprehend big endian and little endian as Network Byte
Order and Host Byte Order respectively. What is the difference between
@cindex Cryptography API
@node Cryptography API
@subsection Cryptography API
-@c %**end of header
+
The gnunetutil APIs provides the cryptographic primitives used in GNUnet.
GNUnet uses 2048 bit RSA keys for the session key exchange and for signing
@cindex Message Queue API
@node Message Queue API
@subsection Message Queue API
-@c %**end of header
+
@strong{ Introduction }@
Often, applications need to queue messages that
@cindex Service API
@node Service API
@subsection Service API
-@c %**end of header
+
Most GNUnet code lives in the form of services. Services are processes
that offer an API for other components of the system to build on. Those
@c ***********************************************************************
@node Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps
@subsection Optimizing Memory Consumption of GNUnet's (Multi-) Hash Maps
-@c %**end of header
+
A commonly used data structure in GNUnet is a (multi-)hash map. It is most
often used to map a peer identity to some data structure, but also to map
@node Analysis
@subsubsection Analysis
-@c %**end of header
+
The main reason for the "excessive" memory consumption by the hash map is
that GNUnet uses 512-bit cryptographic hash codes --- and the
@c ***********************************************************************
@node Solution
@subsubsection Solution
-@c %**end of header
+
The solution that has now been implemented is to @strong{optionally}
allow the hash map to not make a (deep) copy of the hash but instead have
@c ***********************************************************************
@node Migration
@subsubsection Migration
-@c %**end of header
+
To use the new feature, first check that the values contain the respective
key (and never modify it). Then, all calls to
@c ***********************************************************************
@node Conclusion
@subsubsection Conclusion
-@c %**end of header
+
The new optimization can is often applicable and can result in a
reduction in memory consumption of up to 30% in practice. However, it
@c ***********************************************************************
@node Availability
@subsubsection Availability
-@c %**end of header
+
The new multi hash map code was committed in SVN 24319 (which made its
way into GNUnet version 0.9.4).
@cindex CONTAINER_MDLL API
@node CONTAINER_MDLL API
@subsection CONTAINER_MDLL API
-@c %**end of header
+
This text documents the GNUNET_CONTAINER_MDLL API. The
GNUNET_CONTAINER_MDLL API is similar to the GNUNET_CONTAINER_DLL API in
@cindex ARM
@node Automatic Restart Manager (ARM)
@section Automatic Restart Manager (ARM)
-@c %**end of header
+
GNUnet's Automated Restart Manager (ARM) is the GNUnet service responsible
for system initialization and service babysitting. ARM starts and halts
@c ***********************************************************************
@node Basic functionality
@subsection Basic functionality
-@c %**end of header
+
@itemize @bullet
@item ARM source code can be found under "src/arm".@ Service processes are
@c ***********************************************************************
@node Key configuration options
@subsection Key configuration options
-@c %**end of header
+
Configurations for ARM and services should be available in a .conf file
(As an example, see test_arm_api_data.conf). When running ARM, the
@c ***********************************************************************
@node ARM - Availability
@subsection ARM - Availability
-@c %**end of header
+
As mentioned before, one of the features provided by ARM is starting
services on demand. Consider the example of one service "client" that
@cindex TRANSPORT Subsystem
@node TRANSPORT Subsystem
@section TRANSPORT Subsystem
-@c %**end of header
+
This chapter documents how the GNUnet transport subsystem works. The
GNUnet transport subsystem consists of three main components: the
@node Address validation protocol
@subsection Address validation protocol
-@c %**end of header
+
This section documents how the GNUnet transport service validates
connections with other peers. It is a high-level description of the
@cindex NAT library
@node NAT library
@section NAT library
-@c %**end of header
+
The goal of the GNUnet NAT library is to provide a general-purpose API for
NAT traversal @strong{without} third-party support. So protocols that
@node Distance-Vector plugin
@section Distance-Vector plugin
-@c %**end of header
+
The Distance Vector (DV) transport is a transport mechanism that allows
peers to act as relays for each other, thereby connecting peers that would
@cindex SMTP plugin
@node SMTP plugin
@section SMTP plugin
-@c %**end of header
+
@c TODO: Update!
This section describes the new SMTP transport plugin for GNUnet as it
@node Why use SMTP for a peer-to-peer transport?
@subsection Why use SMTP for a peer-to-peer transport?
-@c %**end of header
+
There are many reasons why one would not want to use SMTP:
@node How does it work?
@subsection How does it work?
-@c %**end of header
+
When a GNUnet peer needs to send a message to another GNUnet peer that has
advertised (only) an SMTP transport address, GNUnet base64-encodes the
@node How do I configure my peer?
@subsection How do I configure my peer?
-@c %**end of header
+
First, you need to configure @code{procmail} to filter your inbound E-mail
for GNUnet traffic. The GNUnet messages must be delivered into a pipe, for
@node How do I test if it works?
@subsection How do I test if it works?
-@c %**end of header
+
Any transport can be subjected to some rudimentary tests using the
@code{gnunet-transport-check} tool. The tool sends a message to the local
@node How fast is it?
@subsection How fast is it?
-@c %**end of header
+
We have measured the performance of the UDP, TCP and SMTP transport layer
directly and when used from an application using the GNUnet core.
@cindex Bluetooth plugin
@node Bluetooth plugin
@section Bluetooth plugin
-@c %**end of header
+
This page describes the new Bluetooth transport plugin for GNUnet. The
plugin is still in the testing stage so don't expect it to work
@node What do I need to use the Bluetooth plugin transport?
@subsection What do I need to use the Bluetooth plugin transport?
-@c %**end of header
+
If you are a GNU/Linux user and you want to use the Bluetooth
transport plugin you should install the
@c FIXME: Change to unique title
@node How does it work2?
@subsection How does it work2?
-@c %**end of header
+
The Bluetooth transport plugin uses virtually the same code as the WLAN
plugin and only the helper binary is different. The helper takes a single
@node What possible errors should I be aware of?
@subsection What possible errors should I be aware of?
-@c %**end of header
+
@emph{This section is dedicated for GNU/Linux users}
@c FIXME: A more unique name
@node How do I configure my peer2?
@subsection How do I configure my peer2?
-@c %**end of header
+
On GNU/Linux, you just have to be sure that the interface name
corresponds to the one that you want to use.
@node How can I test it?
@subsection How can I test it?
-@c %**end of header
+
If you have two Bluetooth devices on the same machine and you are using
GNU/Linux you must:
@node The implementation of the Bluetooth transport plugin
@subsection The implementation of the Bluetooth transport plugin
-@c %**end of header
+
This page describes the implementation of the Bluetooth transport plugin.
@node Linux functionality
@subsubsection Linux functionality
-@c %**end of header
+
In order to implement the plugin functionality on GNU/Linux I
used the BlueZ stack.
@node Details about the broadcast implementation
@subsubsection Details about the broadcast implementation
-@c %**end of header
+
First I want to point out that the broadcast functionality for the CONTROL
messages is not implemented in a conventional way. Since the inquiry scan
@node Windows functionality
@subsubsection Windows functionality
-@c %**end of header
+
For Windows I decided to use the Microsoft Bluetooth stack which has the
advantage of coming standard from Windows XP SP2. The main disadvantage is
@node Pending features
@subsubsection Pending features
-@c %**end of header
+
@itemize @bullet
@item Implement the broadcast functionality on Windows @emph{(currently
@node WLAN plugin
@section WLAN plugin
-@c %**end of header
+
This section documents how the wlan transport plugin works. Parts which
are not implemented yet or could be better implemented are described at
@cindex ATS Subsystem
@node ATS Subsystem
@section ATS Subsystem
-@c %**end of header
+
ATS stands for "automatic transport selection", and the function of ATS in
GNUnet is to decide on which address (and thus transport plugin) should
@cindex CORE Subsystem
@node CORE Subsystem
@section CORE Subsystem
-@c %**end of header
+
The CORE subsystem in GNUnet is responsible for securing link-layer
communications between nodes in the GNUnet overlay network. CORE builds
@cindex core subsystem limitations
@node Limitations
@subsection Limitations
-@c %**end of header
+
CORE does not perform
@uref{http://en.wikipedia.org/wiki/Routing, routing}; using CORE it is
@cindex when is a peer connected
@node When is a peer "connected"?
@subsection When is a peer "connected"?
-@c %**end of header
+
In addition to the security features mentioned above, CORE also provides
one additional key feature to applications using it, and that is a
@cindex libgnunetcore
@node libgnunetcore
@subsection libgnunetcore
-@c %**end of header
+
The CORE API (defined in @file{gnunet_core_service.h}) is the basic
messaging API used by P2P applications built using GNUnet. It provides
@cindex core clinet-service protocol
@node The CORE Client-Service Protocol
@subsection The CORE Client-Service Protocol
-@c %**end of header
+
This section describes the protocol between an application using the CORE
service (the client) and the CORE service process itself.
@node Setup2
@subsubsection Setup2
-@c %**end of header
+
When a client connects to the CORE service, it first sends a
@code{InitMessage} which specifies options for the connection and a set of
@node Notifications
@subsubsection Notifications
-@c %**end of header
+
The CORE will send @code{ConnectNotifyMessage}s and
@code{DisconnectNotifyMessage}s whenever peers connect or disconnect from
@node Sending
@subsubsection Sending
-@c %**end of header
+
When a client wants to transmit a message, it first requests a
transmission slot by sending a @code{SendMessageRequest} which specifies
@cindex CORE Peer-to-Peer Protocol
@node The CORE Peer-to-Peer Protocol
@subsection The CORE Peer-to-Peer Protocol
-@c %**end of header
+
@menu
@cindex EphemeralKeyMessage creation
@node Creating the EphemeralKeyMessage
@subsubsection Creating the EphemeralKeyMessage
-@c %**end of header
+
When the CORE service starts, each peer creates a fresh ephemeral (ECC)
public-private key pair and signs the corresponding
@node Establishing a connection
@subsubsection Establishing a connection
-@c %**end of header
+
Peers begin their interaction by sending a @code{EphemeralKeyMessage} to
the other peer once the TRANSPORT service notifies the CORE service about
@node Encryption and Decryption
@subsubsection Encryption and Decryption
-@c %**end of header
+
All functions related to the key exchange and encryption/decryption of
messages can be found in @file{gnunet-service-core_kx.c} (except for the
@node Type maps
@subsubsection Type maps
-@c %**end of header
+
Once an encrypted connection has been established, peers begin to exchange
type maps. Type maps are used to allow the CORE service to determine which
@node Target value
@subsubsection Target value
-@c %**end of header
+
The target value itself is generated by hashing the current time, rounded
down to an agreed value. If the rounding amount is 1h (default) and the
@node Timing
@subsubsection Timing
-@c %**end of header
+
The NSE subsystem has some timing control to avoid everybody broadcasting
its ID all at one. Once each peer has the target random value, it
@node Controlled Flooding
@subsubsection Controlled Flooding
-@c %**end of header
+
When a peer receives a value, first it verifies that it is closer than the
closest value it had so far, otherwise it answers the incoming message
@node Calculating the estimate
@subsubsection Calculating the estimate
-@c %**end of header
+
Once the closest ID has been spread across the network each peer gets the
exact distance between this ID and the target value of the round and
@node libgnunetnse
@subsection libgnunetnse
-@c %**end of header
+
The NSE subsystem has the simplest API of all services, with only two
calls: @code{GNUNET_NSE_connect} and @code{GNUNET_NSE_disconnect}.
@node Results
@subsubsection Results
-@c %**end of header
+
The callback provides two values: the average and the
@uref{http://en.wikipedia.org/wiki/Standard_deviation, standard deviation}
@node libgnunetnse - Examples
@subsubsection libgnunetnse -Examples
-@c %**end of header
+
Let's close with a couple examples.
@node The NSE Client-Service Protocol
@subsection The NSE Client-Service Protocol
-@c %**end of header
+
As with the API, the client-service protocol is very simple, only has 2
different messages, defined in @code{src/nse/nse.h}:
@node The NSE Peer-to-Peer Protocol
@subsection The NSE Peer-to-Peer Protocol
-@c %**end of header
+
The NSE subsystem only has one message in the P2P protocol, the
@code{GNUNET_MESSAGE_TYPE_NSE_P2P_FLOOD} message.
@node HOSTLIST Subsystem
@section HOSTLIST Subsystem
-@c %**end of header
+
Peers in the GNUnet overlay network need address information so that they
can connect with other peers. GNUnet uses so called HELLO messages to
@node HELLOs
@subsection HELLOs
-@c %**end of header
+
The basic information peers require to connect to other peers are
contained in so called HELLO messages you can think of as a business card.
@node Overview for the HOSTLIST subsystem
@subsection Overview for the HOSTLIST subsystem
-@c %**end of header
+
The HOSTLIST subsystem provides a way to distribute and obtain contact
information to connect to other peers using a simple HTTP GET request.
@node Features
@subsubsection Features
-@c %**end of header
+
The HOSTLIST daemon can:
@node HOSTLIST - Limitations
@subsubsection HOSTLIST - Limitations
-@c %**end of header
+
The HOSTLIST daemon does not:
@node Interacting with the HOSTLIST daemon
@subsection Interacting with the HOSTLIST daemon
-@c %**end of header
+
The HOSTLIST subsystem is currently implemented as a daemon, so there is
no need for the user to interact with it and therefore there is no
@node Hostlist security address validation
@subsection Hostlist security address validation
-@c %**end of header
+
Since information obtained from other parties cannot be trusted without
validation, we have to distinguish between @emph{validated} and
@node The HOSTLIST daemon
@subsection The HOSTLIST daemon
-@c %**end of header
+
The hostlist daemon is the main component of the HOSTLIST subsystem. It is
started by the ARM service and (if configured) starts the HOSTLIST client
@node The HOSTLIST server
@subsection The HOSTLIST server
-@c %**end of header
+
The server provides a way for other peers to obtain HELLOs. Basically it
is a small web server other peers can connect to and download a list of
@node The HTTP Server
@subsubsection The HTTP Server
-@c %**end of header
+
During startup, the server starts a web server listening on the port
specified with the HTTPPORT value (default 8080). In addition it connects
@node Advertising the URL
@subsubsection Advertising the URL
-@c %**end of header
+
The server also advertises the URL to download the hostlist to other peers
if hostlist advertisement is enabled.
@node The HOSTLIST client
@subsection The HOSTLIST client
-@c %**end of header
+
The client provides the functionality to download the list of HELLOs from
a set of URLs.
@node Bootstrapping
@subsubsection Bootstrapping
-@c %**end of header
+
For bootstrapping, it schedules a task to download the hostlist from the
set of known URLs.
@node Learning
@subsubsection Learning
-@c %**end of header
+
The client also manages hostlist advertisements from other peers. The
HOSTLIST daemon forwards @code{GNUNET_MESSAGE_TYPE_HOSTLIST_ADVERTISEMENT}
@node Usage
@subsection Usage
-@c %**end of header
+
To start HOSTLIST by default, it has to be added to the DEFAULTSERVICES
section for the ARM services. This is done in the default configuration.
@node IDENTITY Subsystem
@section IDENTITY Subsystem
-@c %**end of header
+
Identities of "users" in GNUnet are called egos.
Egos can be used as pseudonyms ("fake names") or be tied to an
@cindex libgnunetidentity
@node libgnunetidentity
@subsection libgnunetidentity
-@c %**end of header
+
@menu
@node Connecting to the service
@subsubsection Connecting to the service
-@c %**end of header
+
First, typical clients connect to the identity service using
@code{GNUNET_IDENTITY_connect}. This function takes a callback as a
@node Operations on Egos
@subsubsection Operations on Egos
-@c %**end of header
+
Given an ego handle, the main operations are to get its associated private
key using @code{GNUNET_IDENTITY_ego_get_private_key} or its associated
@node The anonymous Ego
@subsubsection The anonymous Ego
-@c %**end of header
+
A special way to obtain an ego handle is to call
@code{GNUNET_IDENTITY_ego_get_anonymous}, which returns an ego for the
@node The IDENTITY Client-Service Protocol
@subsection The IDENTITY Client-Service Protocol
-@c %**end of header
+
A client connecting to the identity service first sends a message with
type
@node Editing Zone Information
@subsubsection Editing Zone Information
-@c %**end of header
+
NAMESTORE provides functions to lookup records stored under a label in a
zone and to store records under a label in a zone.
@node Iterating Zone Information
@subsubsection Iterating Zone Information
-@c %**end of header
+
A client can iterate over all information in a zone or all zones managed
by NAMESTORE.
@node Monitoring Zone Information
@subsubsection Monitoring Zone Information
-@c %**end of header
+
Clients can also monitor zones to be notified about changes. Here the
clients uses the @code{GNUNET_NAMESTORE_zone_monitor_start} function and
@node PEERINFO Subsystem
@section PEERINFO Subsystem
-@c %**end of header
+
The PEERINFO subsystem is used to store verified (validated) information
about known peers in a persistent way. It obtains these addresses for
@node PEERINFO - Features
@subsection PEERINFO - Features
-@c %**end of header
+
@itemize @bullet
@item Persistent storage
@node DeveloperPeer Information
@subsection DeveloperPeer Information
-@c %**end of header
+
The PEERINFO subsystem stores these information in the form of HELLO
messages you can think of as business cards.
@node Startup
@subsection Startup
-@c %**end of header
+
During startup the PEERINFO services loads persistent HELLOs from disk.
First PEERINFO parses the directory configured in the HOSTS value of the
@node Managing Information
@subsection Managing Information
-@c %**end of header
+
The PEERINFO services stores information about known PEERS and a single
HELLO message for every peer.
@node Obtaining Information
@subsection Obtaining Information
-@c %**end of header
+
When a client requests information from PEERINFO, PEERINFO performs a
lookup for the respective peer or all peers if desired and transmits this
@node The PEERINFO Client-Service Protocol
@subsection The PEERINFO Client-Service Protocol
-@c %**end of header
+
To connect and disconnect to and from the PEERINFO Service PEERINFO
utilizes the util client/server infrastructure, so no special messages
@node libgnunetpeerinfo
@subsection libgnunetpeerinfo
-@c %**end of header
+
The PEERINFO API consists mainly of three different functionalities:
@node Connecting to the PEERINFO Service
@subsubsection Connecting to the PEERINFO Service
-@c %**end of header
+
To connect to the PEERINFO service the function
@code{GNUNET_PEERINFO_connect} is used, taking a configuration handle as
@node Adding Information to the PEERINFO Service
@subsubsection Adding Information to the PEERINFO Service
-@c %**end of header
+
@code{GNUNET_PEERINFO_add_peer} adds a new peer to the PEERINFO subsystem
storage. This function takes the PEERINFO handle as an argument, the HELLO
@node Obtaining Information from the PEERINFO Service
@subsubsection Obtaining Information from the PEERINFO Service
-@c %**end of header
+
To iterate over information in PEERINFO you use
@code{GNUNET_PEERINFO_iterate}.
@node PEERSTORE Subsystem
@section PEERSTORE Subsystem
-@c %**end of header
+
GNUnet's PEERSTORE subsystem offers persistent per-peer storage for other
GNUnet subsystems. GNUnet subsystems can use PEERSTORE to persistently
@node Functionality
@subsection Functionality
-@c %**end of header
+
Subsystems can store any type of value under a (subsystem, peerid, key)
combination. A "replace" flag set during store operations forces the
@node Architecture
@subsection Architecture
-@c %**end of header
+
PEERSTORE implements the following components:
@node libgnunetpeerstore
@subsection libgnunetpeerstore
-@c %**end of header
+
libgnunetpeerstore is the library containing the PEERSTORE API. Subsystems
wishing to communicate with the PEERSTORE service use this API to open a
@node SET Subsystem
@section SET Subsystem
-@c %**end of header
+
The SET service implements efficient set operations between two peers
over a mesh tunnel.
@node Local Sets
@subsection Local Sets
-@c %**end of header
+
Sets created by a local client can be modified and reused for multiple
operations. As each set operation requires potentially expensive special
@node Set Modifications
@subsection Set Modifications
-@c %**end of header
+
Even when set operations are active, one can add to and remove elements
from a set.
@node Set Operations
@subsection Set Operations
-@c %**end of header
+
Set operations can be started in two ways: Either by accepting an
operation request from a remote peer, or by requesting a set operation
@node Result Elements
@subsection Result Elements
-@c %**end of header
+
The SET service has three @emph{result modes} that determine how an
operation's result set is delivered to the client:
@node libgnunetset
@subsection libgnunetset
-@c %**end of header
+
@menu
* Sets::
@node Sets
@subsubsection Sets
-@c %**end of header
+
New sets are created with @code{GNUNET_SET_create}. Both the local peer's
configuration (as each set has its own client connection) and the
@node Listeners
@subsubsection Listeners
-@c %**end of header
+
Listeners are created with @code{GNUNET_SET_listen}. Each time time a
remote peer suggests a set operation with an application id and operation
@node Operations
@subsubsection Operations
-@c %**end of header
+
Operations to be initiated by the local peer are created with
@code{GNUNET_SET_prepare}. Note that the operation will not be started
@node Supplying a Set
@subsubsection Supplying a Set
-@c %**end of header
+
To create symmetry between the two ways of starting a set operation
(accepting and initiating it), the operation handles returned by
@node The Result Callback
@subsubsection The Result Callback
-@c %**end of header
+
Clients must specify both a result mode and a result callback with
@code{GNUNET_SET_accept} and @code{GNUNET_SET_prepare}. The result
@node The SET Client-Service Protocol
@subsection The SET Client-Service Protocol
-@c %**end of header
+
@menu
* Creating Sets::
@node Creating Sets
@subsubsection Creating Sets
-@c %**end of header
+
For each set of a client, there exists a client connection to the service.
Sets are created by sending the @code{GNUNET_SERVICE_SET_CREATE} message
@node Listeners2
@subsubsection Listeners2
-@c %**end of header
+
Each listener also requires a seperate client connection. By sending the
@code{GNUNET_SERVICE_SET_LISTEN} message, the client notifies the service
@node Initiating Operations
@subsubsection Initiating Operations
-@c %**end of header
+
Operations with remote peers are initiated by sending a
@code{GNUNET_SERVICE_SET_EVALUATE} message to the service. The@ client
@node Modifying Sets
@subsubsection Modifying Sets
-@c %**end of header
+
Sets are modified with the @code{GNUNET_SERVICE_SET_ADD} and
@code{GNUNET_SERVICE_SET_REMOVE} messages.
@node Results and Operation Status
@subsubsection Results and Operation Status
-@c %**end of header
+
The service notifies the client of result elements and success/failure of
a set operation with the @code{GNUNET_SERVICE_SET_RESULT} message.
@node Iterating Sets
@subsubsection Iterating Sets
-@c %**end of header
+
All elements of a set can be requested by sending
@code{GNUNET_SERVICE_SET_ITER_REQUEST}. The server responds with
@node The SET Intersection Peer-to-Peer Protocol
@subsection The SET Intersection Peer-to-Peer Protocol
-@c %**end of header
+
The intersection protocol operates over CADET and starts with a
GNUNET_MESSAGE_TYPE_SET_P2P_OPERATION_REQUEST being sent by the peer
@node The Bloom filter exchange
@subsubsection The Bloom filter exchange
-@c %**end of header
+
In this phase, each peer transmits a Bloom filter over the remaining
keys of the local set to the other peer using a
@node Salt
@subsubsection Salt
-@c %**end of header
+
Bloomfilter operations are probabilistic: With some non-zero probability
the test may incorrectly say an element is in the set, even though it is
@node The SET Union Peer-to-Peer Protocol
@subsection The SET Union Peer-to-Peer Protocol
-@c %**end of header
+
The SET union protocol is based on Eppstein's efficient set reconciliation
without prior context. You should read this paper first if you want to
@node STATISTICS Subsystem
@section STATISTICS Subsystem
-@c %**end of header
+
In GNUnet, the STATISTICS subsystem offers a central place for all
subsystems to publish unsigned 64-bit integer run-time statistics.
@node libgnunetstatistics
@subsection libgnunetstatistics
-@c %**end of header
+
@strong{libgnunetstatistics} is the library containing the API for the
STATISTICS subsystem. Any process requiring to use STATISTICS should use
@node Statistics retrieval
@subsubsection Statistics retrieval
-@c %**end of header
+
Once a connection to the statistics service is obtained, information
about any other system which uses statistics can be retrieved with the
@node Setting statistics and updating them
@subsubsection Setting statistics and updating them
-@c %**end of header
+
So far we have seen how to retrieve statistics, here we will learn how we
can set statistics and update them so that other subsystems can retrieve
@node Watches
@subsubsection Watches
-@c %**end of header
+
As interesting feature of STATISTICS lies in serving notifications
whenever a statistic of our interest is modified.
@node The STATISTICS Client-Service Protocol
@subsection The STATISTICS Client-Service Protocol
-@c %**end of header
+
@menu
@node Statistics retrieval2
@subsubsection Statistics retrieval2
-@c %**end of header
+
To retrieve statistics, the client transmits a message of type
@code{GNUNET_MESSAGE_TYPE_STATISTICS_GET} containing the given subsystem
@node Setting and updating statistics
@subsubsection Setting and updating statistics
-@c %**end of header
+
The subsystem name, parameter name, its value and the persistence flag are
communicated to the service through the message
@node Watching for updates
@subsubsection Watching for updates
-@c %**end of header
+
The function registers the watch at the service by sending a message of
type @code{GNUNET_MESSAGE_TYPE_STATISTICS_WATCH}. The service then sends
@node Distributed Hash Table (DHT)
@section Distributed Hash Table (DHT)
-@c %**end of header
+
GNUnet includes a generic distributed hash table that can be used by
developers building P2P applications in the framework.
@node Block library and plugins
@subsection Block library and plugins
-@c %**end of header
+
@menu
* What is a Block?::
@node What is a Block?
@subsubsection What is a Block?
-@c %**end of header
+
Blocks are small (< 63k) pieces of data stored under a key (struct
GNUNET_HashCode). Blocks have a type (enum GNUNET_BlockType) which defines
@node The API of libgnunetblock
@subsubsection The API of libgnunetblock
-@c %**end of header
+
The block library requires for each (family of) block type(s) a block
plugin (implementing @file{gnunet_block_plugin.h}) that provides basic
@node Queries
@subsubsection Queries
-@c %**end of header
+
The query format for any block in GNUnet consists of four main components.
First, the type of the desired block must be specified. Second, the query
@node Sample Code
@subsubsection Sample Code
-@c %**end of header
+
The source code in @strong{plugin_block_test.c} is a good starting point
for new block plugins --- it does the minimal work by implementing a
@node Conclusion2
@subsubsection Conclusion2
-@c %**end of header
+
In conclusion, GNUnet subsystems that want to use the DHT need to define a
block format and write a plugin to match queries and replies. For testing,
@node libgnunetdht
@subsection libgnunetdht
-@c %**end of header
+
The DHT API itself is pretty simple and offers the usual GET and PUT
functions that work as expected. The specified block type refers to the
@node GET
@subsubsection GET
-@c %**end of header
+
When using GET, the main consideration for developers (other than the
block library) should be that after issuing a GET, the DHT will
@node PUT
@subsubsection PUT
-@c %**end of header
+
@c inconsistent use of ``must'' above it's written ``MUST''
In contrast to GET operations, developers @strong{must} manually re-run
@node MONITOR
@subsubsection MONITOR
-@c %**end of header
+
The DHT API also allows applications to monitor messages crossing the
local DHT service.
@node DHT Routing Options
@subsubsection DHT Routing Options
-@c %**end of header
+
There are two important options for GET and PUT requests:
@node The DHT Client-Service Protocol
@subsection The DHT Client-Service Protocol
-@c %**end of header
+
@menu
* PUTting data into the DHT::
@node PUTting data into the DHT
@subsubsection PUTting data into the DHT
-@c %**end of header
+
To store (PUT) data into the DHT, the client sends a
@code{struct GNUNET_DHT_ClientPutMessage} to the service.
@node GETting data from the DHT
@subsubsection GETting data from the DHT
-@c %**end of header
+
To retrieve (GET) data from the DHT, the client sends a
@code{struct GNUNET_DHT_ClientGetMessage} to the service. The message
@node Monitoring the DHT
@subsubsection Monitoring the DHT
-@c %**end of header
+
To begin monitoring, the client sends a
@code{struct GNUNET_DHT_MonitorStartStop} message to the DHT service.
@node The DHT Peer-to-Peer Protocol
@subsection The DHT Peer-to-Peer Protocol
-@c %**end of header
+
@menu
@node Routing GETs or PUTs
@subsubsection Routing GETs or PUTs
-@c %**end of header
+
When routing GETs or PUTs, the DHT service selects a suitable subset of
neighbours for forwarding. The exact number of neighbours can be zero or
@node PUTting data into the DHT2
@subsubsection PUTting data into the DHT2
-@c %**end of header
+
To PUT data into the DHT, the service sends a @code{struct PeerPutMessage}
of type @code{GNUNET_MESSAGE_TYPE_DHT_P2P_PUT} to the respective
@node GETting data from the DHT2
@subsubsection GETting data from the DHT2
-@c %**end of header
+
A peer can search the DHT by sending @code{struct PeerGetMessage}s of type
@code{GNUNET_MESSAGE_TYPE_DHT_P2P_GET} to other peers. In addition to the
@node GNU Name System (GNS)
@section GNU Name System (GNS)
-@c %**end of header
+
The GNU Name System (GNS) is a decentralized database that enables users
to securely resolve names to values.
@node libgnunetgns
@subsection libgnunetgns
-@c %**end of header
+
The GNS API itself is extremely simple. Clients first connect to the
GNS service using @code{GNUNET_GNS_connect}.
@node Looking up records
@subsubsection Looking up records
-@c %**end of header
+
@code{GNUNET_GNS_lookup} takes a number of arguments:
@node Accessing the records
@subsubsection Accessing the records
-@c %**end of header
+
The @code{libgnunetgnsrecord} library provides an API to manipulate the
GNS record array that is given to proc. In particular, it offers
@node Creating records
@subsubsection Creating records
-@c %**end of header
+
Creating GNS records is typically done by building the respective record
information (possibly with the help of @code{libgnunetgnsrecord} and
@node Future work
@subsubsection Future work
-@c %**end of header
+
In the future, we want to expand @code{libgnunetgns} to allow
applications to observe shortening operations performed during GNS
@node libgnunetgnsrecord
@subsection libgnunetgnsrecord
-@c %**end of header
+
The @code{libgnunetgnsrecord} library is used to manipulate GNS
records (in plaintext or in their encrypted format).
@node Value handling
@subsubsection Value handling
-@c %**end of header
+
@code{GNUNET_GNSRECORD_value_to_string} can be used to convert
the (binary) representation of a GNS record value to a human readable,
@node Type handling
@subsubsection Type handling
-@c %**end of header
+
@code{GNUNET_GNSRECORD_typename_to_number} can be used to obtain the
numeric value associated with a given typename. For example, given the
@node GNS plugins
@subsection GNS plugins
-@c %**end of header
+
Adding a new GNS record type typically involves writing (or extending) a
GNSRECORD plugin. The plugin needs to implement the
@node The GNS Client-Service Protocol
@subsection The GNS Client-Service Protocol
-@c %**end of header
+
The GNS client-service protocol consists of two simple messages, the
@code{LOOKUP} message and the @code{LOOKUP_RESULT}. Each @code{LOOKUP}
@node Hijacking the DNS-Traffic using gnunet-service-dns
@subsection Hijacking the DNS-Traffic using gnunet-service-dns
-@c %**end of header
+
This section documents how the gnunet-service-dns (and the
gnunet-helper-dns) intercepts DNS queries from the local system.
@node Network Setup Details
@subsubsection Network Setup Details
-@c %**end of header
+
The DNS interceptor adds the following rules to the Linux kernel:
@example
@node Serving DNS lookups via GNS on W32
@subsection Serving DNS lookups via GNS on W32
-@c %**end of header
+
This section documents how the libw32nsp (and
gnunet-gns-helper-service-w32) do DNS resolutions of DNS queries on the
@node Importing DNS Zones into GNS
@subsection Importing DNS Zones into GNS
-@c %**end of header
+
This section discusses the challenges and problems faced when writing the
Ascension tool. It also takes a look at possible improvements in the future.
@node GNS Namecache
@section GNS Namecache
-@c %**end of header
+
The NAMECACHE subsystem is responsible for caching (encrypted) resolution
results of the GNU Name System (GNS). GNS makes zone information available
@node libgnunetnamecache
@subsection libgnunetnamecache
-@c %**end of header
+
The NAMECACHE API consists of five simple functions. First, there is
@code{GNUNET_NAMECACHE_connect} to connect to the NAMECACHE service.
@node The NAMECACHE Client-Service Protocol
@subsection The NAMECACHE Client-Service Protocol
-@c %**end of header
+
All messages in the NAMECACHE IPC protocol start with the
@code{struct GNUNET_NAMECACHE_Header} which adds a request
@node Lookup
@subsubsection Lookup
-@c %**end of header
+
The @code{struct LookupBlockMessage} is used to lookup a block stored in
the cache.
@node Store
@subsubsection Store
-@c %**end of header
+
The @code{struct BlockCacheMessage} is used to cache a block in the
NAMECACHE.
@node The NAMECACHE Plugin API
@subsection The NAMECACHE Plugin API
-@c %**end of header
+
The NAMECACHE plugin API consists of two functions, @code{cache_block} to
store a block in the database, and @code{lookup_block} to lookup a block
@node Lookup2
@subsubsection Lookup2
-@c %**end of header
+
The @code{lookup_block} function is expected to return at most one block
to the iterator, and return @code{GNUNET_NO} if there were no non-expired
@node Store2
@subsubsection Store2
-@c %**end of header
+
The @code{cache_block} function is expected to try to store the block in
the database, and return @code{GNUNET_SYSERR} if this was not possible
@cindex REVOCATION Subsystem
@node REVOCATION Subsystem
@section REVOCATION Subsystem
-@c %**end of header
+
The REVOCATION subsystem is responsible for key revocation of Egos.
If a user learns that theis private key has been compromised or has lost
@node Dissemination
@subsection Dissemination
-@c %**end of header
+
When a revocation is performed, the revocation is first of all
disseminated by flooding the overlay network.
@node Revocation Message Design Requirements
@subsection Revocation Message Design Requirements
-@c %**end of header
+
However, flooding is also quite costly, creating O(|E|) messages on a
network with |E| edges.
@node libgnunetrevocation
@subsection libgnunetrevocation
-@c %**end of header
+
The REVOCATION API consists of two parts, to query and to issue
revocations.
@node Querying for revoked keys
@subsubsection Querying for revoked keys
-@c %**end of header
+
@code{GNUNET_REVOCATION_query} is used to check if a given ECDSA public
key has been revoked.
@node Preparing revocations
@subsubsection Preparing revocations
-@c %**end of header
+
It is often desirable to create a revocation record ahead-of-time and
store it in an off-line location to be used later in an emergency.
@node The REVOCATION Peer-to-Peer Protocol
@subsection The REVOCATION Peer-to-Peer Protocol
-@c %**end of header
+
Revocation uses two disjoint ways to spread revocation information among
peers.
@node File-sharing (FS) Subsystem
@section File-sharing (FS) Subsystem
-@c %**end of header
+
This chapter describes the details of how the file-sharing service works.
As with all services, it is split into an API (libgnunetfs), the service
@node Encoding for Censorship-Resistant Sharing (ECRS)
@subsection Encoding for Censorship-Resistant Sharing (ECRS)
-@c %**end of header
+
When GNUnet shares files, it uses a content encoding that is called ECRS,
the Encoding for Censorship-Resistant Sharing.
@node Namespace Advertisements
@subsubsection Namespace Advertisements
-@c %**end of header
+
@c %**FIXME: all zeroses -> ?
An @code{SBlock} with identifier all zeros is a signed
@node KSBlocks
@subsubsection KSBlocks
-@c %**end of header
+
GNUnet implements @code{KSBlocks} which are @code{KBlocks} that, instead
of encrypting a CHK and metadata, encrypt an @code{SBlock} instead.
@node File-sharing persistence directory structure
@subsection File-sharing persistence directory structure
-@c %**end of header
+
This section documents how the file-sharing library implements
persistence of file-sharing operations and specifically the resulting
@node REGEX Subsystem
@section REGEX Subsystem
-@c %**end of header
+
Using the REGEX subsystem, you can discover peers that offer a particular
service using regular expressions.
@node How to run the regex profiler
@subsection How to run the regex profiler
-@c %**end of header
+
The gnunet-regex-profiler can be used to profile the usage of mesh/regex
for a given set of regular expressions and strings.
@node REST Subsystem
@section REST Subsystem
-@c %**end of header
+
Using the REST subsystem, you can expose REST-based APIs or services.
The REST service is designed as a pluggable architecture.
@node Using GNUnet
@chapter Using GNUnet
-@c %**end of header
+
This tutorial is supposed to give a first introduction for users
trying to do something real with GNUnet. Installation and
@node First steps - Using the GNU Name System
@section First steps - Using the GNU Name System
-@c %**end of header
+
@menu
* Preliminaries::
@node Preliminaries
@subsection Preliminaries
-@c %**end of header
+
``.pin'' is a default zone which points to a zone managed by gnunet.org.
Use @code{gnunet-config -s gns} to view the GNS configuration, including
@node The GNS Tab
@subsection The GNS Tab
-@c %**end of header
+
Maintaing your zones is through the NAMESTORE service and is discussed
here. You can manage your zone using @command{gnunet-identity} and
@node Creating a Record
@subsection Creating a Record
-@c %**end of header
+
We will begin by creating a simple record in your master zone.
To do this, click on the text "<new name>" in the table. The field is
@node Resolving GNS records
@subsection Resolving GNS records
-@c %**end of header
+
Next, you should try resolving your own GNS records. The method we
found to be the most uncomplicated is to do this by explicitly
@node Integration with Browsers
@subsection Integration with Browsers
-@c %**end of header
+
While we recommend integrating GNS using the NSS module in the
GNU libc Name Service Switch, you can also integrate GNS
@node Be Social
@subsection Be Social
-@c %**end of header
+
Next, you should print out your business card and be social.
Find a friend, help them install GNUnet and exchange business cards with
@node What's Next?
@subsection What's Next?
-@c %**end of header
+
This may seem not like much of an application yet, but you have
just been one of the first to perform a decentralized secure name
@pindex gnunet-conservation-gtk
@node First steps - Using GNUnet Conversation
@section First steps - Using GNUnet Conversation
-@c %**end of header
+
First, you should launch the graphical user interface. You can do
this from the command-line by typing
@node Testing your Audio Equipment
@subsection Testing your Audio Equipment
-@c %**end of header
+
First, you should use @code{gnunet-conversation-test} to check that your
microphone and speaker are working correctly. You will be prompted to
@node GNS Zones
@subsection GNS Zones
-@c %**end of header
+
@code{gnunet-conversation} uses GNS for addressing. This means that
you need to have a GNS zone created before using it. Information
@node Picking an Identity
@subsubsection Picking an Identity
-@c %**end of header
+
To make a call with @code{gnunet-conversation}, you first
need to choose an identity. This identity is both the caller ID
@node Calling somebody
@subsubsection Calling somebody
-@c %**end of header
+
Now you can call a buddy. Obviously, your buddy will have to have GNUnet
installed and must have performed the same steps. Also, you must have
@node First steps - Using the GNUnet VPN
@section First steps - Using the GNUnet VPN
-@c %**end of header
+
@menu
@node VPN Preliminaries
@subsection VPN Preliminaries
-@c %**end of header
+
To test the GNUnet VPN, we should first run a web server.
The easiest way to do this is to just start @code{gnunet-bcd},
@node GNUnet-Exit configuration
@subsection GNUnet-Exit configuration
-@c %**end of header
+
Stop your peer (as user @code{gnunet}, run @command{gnunet-arm -e}) and
run @command{gnunet-setup}. In @command{gnunet-setup}, make sure to
@node GNS configuration
@subsection GNS configuration
-@c %**end of header
+
Now, using your normal user (not the @code{gnunet} system user), run
@command{gnunet-namestore-gtk}. Add a new label www in your
@node Accessing the service
@subsection Accessing the service
-@c %**end of header
+
You should now be able to access your webserver. Type in:
@node Using a Browser
@subsection Using a Browser
-@c %**end of header
+
Sadly, modern browsers tend to bypass the Name Services Switch and
attempt DNS resolution directly. You can either run
@node File-sharing
@section File-sharing
-@c %**end of header
+
This chapter documents the GNUnet file-sharing application. The original
file-sharing implementation for GNUnet was designed to provide
@node fs-Searching
@subsection Searching
-@c %**end of header
+
The command @command{gnunet-search} can be used to search
for content on GNUnet. The format is:
@node fs-Downloading
@subsection Downloading
-@c %**end of header
+
In order to download a file, you need the whole line returned by
@command{gnunet-search}.
@node fs-Publishing
@subsection Publishing
-@c %**end of header
+
The command @command{gnunet-publish} can be used to add content
to the network. The basic format of the command is
@node Important command-line options
@subsubsection Important command-line options
-@c %**end of header
+
The option @code{-k} is used to specify keywords for the file that
should be inserted. You can supply any number of keywords,
@node Indexing vs. Inserting
@subsubsection Indexing vs Inserting
-@c %**end of header
+
By default, GNUnet indexes a file instead of making a full copy.
This is much more efficient, but requires the file to stay unaltered
@node fs-Concepts
@subsection Concepts
-@c %**end of header
+
For better results with filesharing it is useful to understand the
following concepts.
@node Files
@subsubsection Files
-@c %**end of header
+
A file in GNUnet is just a sequence of bytes. Any file-format is allowed
and the maximum file size is theoretically @math{2^64 - 1} bytes, except
@node Keywords
@subsubsection Keywords
-@c %**end of header
+
Keywords are the most simple mechanism to find files on GNUnet.
Keywords are @strong{case-sensitive} and the search string
@node Directories
@subsubsection Directories
-@c %**end of header
+
A directory in GNUnet is a list of file identifiers with meta data.
The file identifiers provide sufficient information about the files
@node Pseudonyms
@subsubsection Pseudonyms
-@c %**end of header
+
@b{Please note that the text in this subsection is outdated and needs}
@b{to be rewritten for version 0.10!}
@node Namespaces
@subsubsection Namespaces
-@c %**end of header
+
@b{Please note that the text in this subsection is outdated and needs}
@b{to be rewritten for version 0.10!}
@node Advertisements
@subsubsection Advertisements
-@c %**end of header
+
@b{Please note that the text in this subsection is outdated and needs}
@b{to be rewritten for version 0.10!}
@node Anonymity level
@subsubsection Anonymity level
-@c %**end of header
+
The anonymity level determines how hard it should be for an adversary to
determine the identity of the publisher or the searcher/downloader. An
@node Content Priority
@subsubsection Content Priority
-@c %**end of header
+
Depending on the peer's configuration, GNUnet peers migrate content
between peers. Content in this sense are individual blocks of a file,
@node Replication
@subsubsection Replication
-@c %**end of header
+
When peers migrate content to other systems, the replication level
of a block is used to decide which blocks need to be migrated most
@node Namespace Management
@subsection Namespace Management
-@c %**end of header
+
@b{Please note that the text in this subsection is outdated and needs}
@b{to be rewritten for version 0.10!}
@node Creating Pseudonyms
@subsubsection Creating Pseudonyms
-@c %**end of header
+
@b{Please note that the text in this subsection is outdated and needs}
@b{to be rewritten for version 0.10!}
@node Deleting Pseudonyms
@subsubsection Deleting Pseudonyms
-@c %**end of header
+
@b{Please note that the text in this subsection is outdated and needs}
@b{to be rewritten for version 0.10!}
@node Advertising namespaces
@subsubsection Advertising namespaces
-@c %**end of header
+
@b{Please note that the text in this subsection is outdated and needs}
@b{to be rewritten for version 0.10!}
@node Namespace names
@subsubsection Namespace names
-@c %**end of header
+
@b{Please note that the text in this subsection is outdated and needs}
@b{to be rewritten for version 0.10!}
@node Namespace root
@subsubsection Namespace root
-@c %**end of header
+
@b{Please note that the text in this subsection is outdated and needs}
@b{to be rewritten for version 0.10!}
@node File-Sharing URIs
@subsection File-Sharing URIs
-@c %**end of header
+
GNUnet (currently) uses four different types of URIs for
file-sharing. They all begin with "gnunet://fs/".
@node Encoding of hash values in URIs
@subsubsection Encoding of hash values in URIs
-@c %**end of header
+
Most URIs include some hash values. Hashes are encoded using
base32hex (RFC 2938).
@cindex chk-uri
@node Content Hash Key (chk)
@subsubsection Content Hash Key (chk)
-@c %**end of header
+
A chk-URI is used to (uniquely) identify a file or directory
and to allow peers to download the file. Files are stored in
@cindex loc-uri
@node Location identifiers (loc)
@subsubsection Location identifiers (loc)
-@c %**end of header
+
For non-anonymous file-sharing, loc-URIs are used to specify which
peer is offering the data (in addition to specifying all of the
@cindex ksk-uri
@node Keyword queries (ksk)
@subsubsection Keyword queries (ksk)
-@c %**end of header
+
A keyword-URI is used to specify that the desired operation
is the search using a particular keyword. The format is simply
@cindex sks-uri
@node Namespace content (sks)
@subsubsection Namespace content (sks)
-@c %**end of header
+
@b{Please note that the text in this subsection is outdated and needs}
@b{to be rewritten for version 0.10!}
@node gtk-Publishing
@subsubsection Publishing
-@c %**end of header
+
To publish a file, select "File Sharing" in the menu bar just below the
"Statistics" icon, and then select "Publish" from the menu.
@node gtk-Searching
@subsubsection Searching
-@c %**end of header
+
Below the menu bar, there are four entry widges labeled "Namespace",
"Keywords", "Anonymity" and "Mime-type" (from left to right). These
@node gtk-Downloading
@subsubsection Downloading
-@c %**end of header
+
In the downloading area, you can select the target directory (default is
"Downloads") and specify the desired filename (by default the filename it
@node The GNU Name System
@section The GNU Name System
-@c %**end of header
+
The GNU Name System (GNS) is secure and decentralized naming system.
projects / oweals / gnunet.git / commitdiff