-This document is a summary of why we're moving to GNUnet NG and what
-this major redesign tries to address.
+This document is a summary of the changes made to GNUnet for version
+0.9.x (from 0.8.x) and what this major redesign tries to address.
First of all, the redesign does not (intentionally) change anything
fundamental about the application-level protocols or how files are
encoded and shared. However, it is not protocol-compatible due to
other changes that do not relate to the essence of the application
-protocols.
-
+protocols. This choice was made since productive development and
+readable code were considered more important than compatibility at
+this point.
The redesign tries to address the following major problem groups
describing isssues that apply more or less to all GNUnet versions
mutexes and almost 1000 lines of lock/unlock operations.
It is challenging for even good programmers to program or
maintain good multi-threaded code with this complexity.
- The excessive locking essentially prevents GNUnet from
+ The excessive locking essentially prevents GNUnet 0.8 from
actually doing much in parallel on multicores.
* Despite efforts like Freeway, it was virtually
- impossible to contribute code to GNUnet that was not
+ impossible to contribute code to GNUnet 0.8 that was not
writen in C/C++.
* Changes to the configuration almost always required restarts
of gnunetd; the existence of change-notifications does not
days, result in really nasty and hard-to-find crashes.
* structs of function pointers in service APIs were
needlessly adding complexity, especially since in
- most cases there was no polymorphism
+ most cases there was no actual polymorphism
SOLUTION:
* Use multiple, lously-coupled processes and one big select
- loop in each (supported by a powerful library to eliminate
+ loop in each (supported by a powerful util library to eliminate
code duplication for each process).
* Eliminate all threads, manage the processes with a
master-process (gnunet-arm, for automatic restart manager)
=> Process priorities can be used to schedule the CPU better
Note that we can not just use one process with a big
select loop because we have blocking operations (and the
- blocking is outside of our control, thanks MySQL,
+ blocking is outside of our control, thanks to MySQL,
sqlite, gethostbyaddr, etc.). So in order to perform
reasonably well, we need some construct for parallel
- execution.
+ execution.
RULE: If your service contains blocking functions, it
- MUST be a process by itself.
+ MUST be a process by itself. If your service
+ is sufficiently complex, you MAY choose to make
+ it a separate process.
* Eliminate structs with function pointers for service APIs;
instead, provide a library (still ending in _service.h) API
that transmits the requests nicely to the respective
thing given the potential for bugs.
* There is no more TIME API function to do anything
with 32-bit seconds
+* There is now a bandwidth API to handle
+ non-trivial bandwidth utilization calculations
PROBLEM GROUP 3 (statistics):
* FILENAME metadata is killed by ECRS/FSUI to avoid
exposing HOME, but what if the user set it manually?
-SOLUTION (draft, not done yet, details missing...):
+SOLUTION:
* Eliminate threads from FS-APIs
- => Open question: how to best write the APIs to
- allow integration with diverse event loops
- of GUI libraries?
-* Store FS-state always also on disk
- => Open question: how to do this without
- compromising state/scalability?
+* Incrementally store FS-state always also on disk using many
+ small files instead of one big file
* Have API to manipulate sharing tree before
upload; have auto-construction modify FILENAME
but allow user-modifications afterwards
+
PROBLEM GROUP 7 (User experience):
* Searches often do not return a sufficient / significant number of
results
- network traffic monitors (goal: eliminate)
- IPC semaphores (goal: d-bus? / eliminate?)
- second timers
- - DNS lookup (goal: have async service; issue: still need synchronous
- resolution in places, current code may not be portable)
- => code shrunk from 61 files to 34, 22k LOC to 15k LOC, 470k to 330k (with symbols)
* New features in util:
- scheduler
- service and program boot-strap code
+ - bandwidth and time APIs
+ - buffered IO API
+ - HKDF implementation (crypto)
* Major changes in util:
- more expressive server (replaces selector)
+ - DNS lookup replaced by async service
GNUnet is peer-to-peer framework focusing on security. The first and
primary application for GNUnet is anonymous file-sharing. GNUnet is
currently developed by a worldwide group of independent free software
-developers. GNUnet is a part of the GNU project (http://www.gnu.org/).
+developers. GNUnet is a GNU package (http://www.gnu.org/).
This is an ALPHA release. There are known and significant bugs as
well as many missing features in this release.
-For a longer description of the GNUnet System see our webpages
-http://www.gnu.org/software/gnunet/ and https://ng.gnunet.org/.
+Additional documentation about GNUnet can be found at
+https://gnunet.org/.
Dependencies:
The fastest way is to use a binary package if it is available for your
system. For a more detailed description, read the installation
-instructions on the webpage at https://ng.gnunet.org/installation.
+instructions on the webpage at https://gnunet.org/installation.
+
+GNUnet requires the GNU MP library (http://www.gnu.org/software/gmp/)
+and libgcrypt (http://www.gnupg.org/). You can specify the path to
+libgcrypt by passing "--with-gcrypt=PATH" to configure. You will also
+need either sqlite (http://www.sqlite.org/) or MySQL
+(http://www.mysql.org/).
If you install from source, you need to install GNU libextractor first
(download from http://www.gnu.org/software/libextractor/). We also
-recommend installing GNU libmicrohttpd. Then you can start the actual
-GNUnet compilation process with:
+recommend installing GNU libmicrohttpd (download from
+http://www.gnu.org/software/libmicrohttpd/). Then you can start the
+actual GNUnet compilation process with:
$ ./configure --prefix=$HOME --with-extractor=$HOME
$ make
# make install
-# sudo -u gnunet gnunet-setup # (note: does not yet exist!)
+# sudo -u gnunet mkdir ~/.gnunet/
+# sudo -u gnunet touch ~/.gnunet/gnunet.conf
# sudo -u gnunet gnunet-arm -s
-This will compile and install GNUnet to ~/bin/, ~/lib/ and ~/man/.
-gnunet-setup will create the configuration; this step is interactive.
-You can run gnunet-setup as root for a system-wide installation or as
-a particular user to create a personal installation. It is
-recommended that you add a user "gnunet" to run "gnunet-arm". You can
-then still run the end-user applications as another user. If you
+This will compile and install GNUnet to $HOME/bin/, $HOME/lib/ and
+$HOME/share/ and start the system with the default configuration. It
+is recommended that you add a user "gnunet" to run "gnunet-arm". You
+can then still run the end-user applications as another user. If you
create a user "gnunet", it is recommended that you edit the
configuration file slightly so that data can be stored in
"/var/lib/gnunet"; you may also want to use "/etc/gnunet.conf" for the
-configuration in this case.
+location of the configuration file in this case.
Note that additional, per-user configuration files
(~/.gnunet/gnunet.conf) need to be created by each user (for example,
by running gnunet-setup). For more general information about the GNU
build process read the INSTALL file.
-GNUnet requires the GNU MP library (http://www.gnu.org/software/gmp/)
-and libgcrypt. You can specify the path to libgcrypt by passing
-"--with-gcrypt=PATH" to configure. You will also need either sqlite
-(version 3 or higher) or MySQL (version 5.0 or higher).
-
If you are compiling the code from subversion, you have to run
". bootstrap" before ./configure. If you receive an error during the
running of ". bootstrap" that looks like "macro `AM_PATH_GTK' not
Configuration
=============
-// FIXME: update this section once we have gnunet-setup! GNUnet uses
-two types of configuration files, one for the daemon (called
-gnunetd.conf) and one for each user (gnunet.conf). You can create and
-edit these configuration files with the gnunet-setup tool. The
-defaults that are created the first time you run gnunet-setup are
-usually ok, you may want to adjust the limitations (space consumption,
-bandwidth, etc.) though. The configuration files are human-readable;
-GNUnet's configuration is typically located at
-"~/.gnunet/gnunet.conf".
+GNUnet uses two types of configuration files, one that specifies the
+system-wide defaults (typically located in /usr/share/gnunet/defaults
+.conf) and a second one that overrides default values with
+user-specific preferences. The user-specific configuration file
+should be located in "~/.gnunet/gnunet.conf" or its location can be
+specified by giving the "-c" option to the respective GNUnet
+application.
-You MUST create "~/.gnunet/gnunet.conf" explicitly before starting
-GNUnet, and the recommended way to do this is to run gnunet-setup.
-You can also create an empty configuration file, in which case
-default values will be used for everything.
-
-If you want to use a different configuration file, pass the name of
-the configuration file as an argument with the option "-c" to any
-GNUnet application.
+The defaults that are shipped with the installation are usually ok,
+you may want to adjust the limitations (space consumption, bandwidth,
+etc.) though. The configuration files are human-readable. Note that
+you MUST create "~/.gnunet/gnunet.conf" explicitly before starting
+GNUnet. You can either copy "defaults.conf" or simply create an empty
+file.
Usage
and offering 'the_file' via your web server. Alternatively, you can
run the build-in web server by adding '-p' to the OPTIONS value
-in the "hostlist" section of gnunet.conf.
+in the "hostlist" section of gnunet.conf and opening the respective
+HTTPPORT to the public.
-If the solution with the URL is not feasible for your situation, you
-can also add hosts manually. Simply copy the hostkeys to
-"$SERVICEHOME/data/hosts/" (where $SERVICEHOME is the directory
-specified in the /etc/gnunet.conf configuration file).
+If the solution with the hostlist URL is not feasible for your
+situation, you can also add hosts manually. Simply copy the hostkeys
+to "$SERVICEHOME/data/hosts/" (where $SERVICEHOME is the directory
+specified in the gnunet.conf configuration file).
Now start the local node using "gnunet-arm -s". GNUnet should run 24/7 if
you want to maximize your anonymity.
-// FIXME: gnunet-gtk is not yet supported in 0.9.x
-The GTK user interface is shipped separately from GNUnet. After
-downloading and installing gnunet-gtk, you can invoke the GUI with:
+The GTK user interface is shipped separately. After downloading and
+installing gnunet-gtk, you can invoke the GUI with:
$ gnunet-gtk
=======================
In order to hide GNUnet's HTTP traffic perfectly, you might consider
-running GNUnet's HTTP transport on port 80. However, you should not
-run GNUnet as root. Instead, forward port 80 to say 8080 with this
-command (as root, in your startup scripts):
+running GNUnet's HTTP transport on port 80. However, we do not
+recommend running GNUnet as root. Instead, forward port 80 to say
+8080 with this command (as root, in your startup scripts):
# iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080
-Then set in the HTTP section of gnunetd.conf the "ADVERTISED-PORT"
+Then set in the HTTP section of gnunet.conf the "ADVERTISED-PORT"
to "80" and "PORT" to 8080. You can do the same trick for the
TCP and UDP transports if you want to map them to a priviledged
port (from the point of view of the network).
Stay tuned
==========
-* http://www.gnu.org/software/gnunet/
-* http://gnunet.org/
+* https://gnunet.org/
* https://gnunet.org/bugs/
-* https://ng.gnunet.org/
+* https://gnunet.org/svn/
+* http://www.gnu.org/software/gnunet/
* http://mail.gnu.org/mailman/listinfo/gnunet-developers
* http://mail.gnu.org/mailman/listinfo/help-gnunet
* http://mail.gnu.org/mailman/listinfo/info-gnunet
* http://mail.gnu.org/mailman/listinfo/gnunet-svn
-
[On W32, we need to select after calling socket before doing connect etc.]
* TRANSPORT:
- HTTP backend [MW]
+ - UDP fails non-deterministically on niko; suspect that it uses hostname-derived,
+ invalid 10.0.0.x address for test-message (which creates transmission failure) [CG]
- Jun 27 11:51:54 core-7670 ERROR Assertion failed at gnunet-service-core.c:3616.
(transport notified us that we connected to ourselves!!!)
* CORE:
disconnect in a timely fashion (waits for connection timeout);
need to figure a way to make it near-instant in those cases
(i.e. rapid reduction in timeout on transport-level disconnect) [CG]
+ => may have been fixed with instant-notification of disconnect
+ to core on session-oriented connection hick-up; not perfect but
+ likely good enough until we get ATS going; still should be tested...
- derived key generation [Nils]
* DHT: [Nate]
- implement DHT service
- - implement testcases
- implement performance tests
* FS: [CG]
- service:
+ trust: do not charge when "idle" / load considerations (migration, routing)
+ artificial delays
- + active reply route caching design & implementation of service; gap extension!
+ + active reply route caching design & implementation; gap extension!
- non-anonymous FS service (needs DHT)
+ DHT integration for search
+ CS-DHT-functions (DHT-put of LOC)
+ P2P-functions (DHT-get)
- - implement FS performance tests
- + insert
- + download
- + search
- + unindex
* GNUNET-GTK: [CG]
- implement publish dialog details:
+ open (import directory structure)
- right-clicking on NS list in search dialog should open menu that allows
+ viewing full meta data
+ deletion of namespace info
- - extend peer dialog with green-yellow-red connectivity status lights
- - extend peer dialog with country flags and names
- - implement statistics tabs
- do meaningful update to status line (starting up, peer running, #connections, shutdown, ...)
- implement unindex operation (use dialog with all indexed files for selection)
- implement download by URI dialog; figure out where to display those downloads!
0.9.0pre3:
+* Determine RC bugs and fix those!
* TRACEKIT: [MW]
- good to have for DV/DHT evaluation!
-* Determine RC bugs and fix those!
-* DATASTORE:
+* DATASTORE: [NN]
- GNUNET_DATASTORE_cancel method not tested
* TESTING: [Nate]
- test basic peer re-configure
- reconstruct IBLOCKS from DBLOCKS if possible (during download; see FIXME in fs_download)
- add support for pushing "already seen" search results to FS service for bloomfilter (can wait)
- use different 'priority' for probe downloads vs. normal downloads
+ - implement FS performance tests
+ + insert
+ + download
+ + search
+ + unindex
* ARM: [CG/Safey]
- better tracking of which config changes actually need to cause process restarts by ARM.
- handle gnunet-arm -k in combination with auto-start magic (what is the right thing here?)
- discover dependencies between services
-* STATISTICS:
+* STATISTICS: [NN]
- test notification-based statistics API
+ - implement statistics GUI
+* PEERINFO: [NN]
+ - move peerinfo to new GUI?
+ - extend peer dialog with green-yellow-red connectivity status lights
+ - extend peer dialog with country flags and names
* SETUP:
- design & implement new setup tool
+* GNUNET-GTK:
+ - add tool bar
+* MONKEY: [Safey]
+ - '-m EMAIL' option for alternative e-mail TO
+ - '-f FILENAME' option to write report to file instead of e-mail (for testing!)
0.9.0:
* new webpage:
(Note: build library always, build service when libxml2/etc. are available)
* FS: [CG]
- Remove KBlocks in gnunet-unindex (see discussion with Kenneth Almquist on gnunet-devs in 9/2009)
-* PEERINFO: [CG]
+* PEERINFO: [NN]
- expire 'ancient' HELLOs (those without valid addresses AND that
we have not 'used' (for their public keys) in a while; need a way
to track actual 'use')
- make sue we also trigger notifications whenever HELLOs expire
* VPN
-* UTIL:
+* UTIL: [CG]
- allow limiting UNIX socket access by UID/GID
WHICH=type
fi
-echo "On some systems, you may need to change /bin/sh to point to bash"
-echo
echo "Please submit the following information with your bug report: "
echo "--------------------------------------------------------------"
OS=`uname -s 2>/dev/null`
TEST=`$WHICH gnunetd 2>/dev/null`
if test -n "$TEST"; then
gnunetd -v | sed -e "s/v//" 2>/dev/null |\
- awk '{print "GNUnet : "$2}'
+ awk '{print "GNUnet 0.8 : "$2 (may conflict!)}'
else
- echo "GNUnet : Not Found"
+ echo "GNUnet 0.8 : Not Found (good)"
+fi
+
+TEST=`$WHICH gnunet-arm 2>/dev/null`
+if test -n "$TEST"; then
+ gnunet-arm -v | sed -e "s/v//" 2>/dev/null |\
+ awk '{print "GNUnet 0.9 : "$2}'
+else
+ echo "GNUnet 0.9 : Not Found"
fi
TEST=`$WHICH libgcrypt-config 2> /dev/null`
fi
-TEST=`grep "Id:" /usr/include/adns.h 2> /dev/null`
-if test -n "$TEST"; then
- echo "$TEST" | awk '{print "GNU adns : "$4}'
-else
- echo "GNU adns : Not found"
-fi
-
TEST=`which qmake 2> /dev/null`
if test -x "$TEST"; then
qmake --version | tail -n 1 | awk '{print "Qt : "$4}'
#! /bin/sh
# mkinstalldirs --- make directory hierarchy
-scriptversion=2006-05-11.19
+scriptversion=2009-04-28.21; # UTC
# Original author: Noah Friedman <friedman@prep.ai.mit.edu>
# Created: 1993-05-16
# eval: (add-hook 'write-file-hooks 'time-stamp)
# time-stamp-start: "scriptversion="
# time-stamp-format: "%:y-%02m-%02d.%02H"
-# time-stamp-end: "$"
+# time-stamp-time-zone: "UTC"
+# time-stamp-end: "; # UTC"
# End:
projects / oweals / gnunet.git / commitdiff