-PHASE #1: (Goal: settle key design questions)
-
-Util:
-* Windows: use events instead of pipes to signal select()s [Nils]
-* only connect() sockets that are ready (select()) [Nils]
- [On W32, we need to select after calling socket before
- doing connect etc.]
-
-PHASE #2: (Goal: recover basic file-sharing functionality)
-
-
-TESTCASES WANTED:
-
-For these functions, it would be nice if we had testcases ("make check")
-that would cause them to be executed and check that they are working:
-* gnunet-service-peerinfo:
- - change_host_trust / flush_trust
- - remove_garbage /
- - discard_hosts_helper / cron_clean_data_hosts
-* gnunet-service-transport:
- - try_unvalidated_addresses
- - lookup_address_callback
- - lookup_hello_callback
- - plugin_env_lookup_address
- - notify_clients_disconnect
- - list_validated_addresses
- - cleanup_validation
- - disconnect_neighbour
- - handle_set_quota
-* plugin_transport_tcp.c:
- - tcp_plugin_cancel
- - tcp_plugin_address_pretty_printer / append_port
- - tcp_plugin_set_receive_quota
- - delayed_done
-* transport_api:
- - GNUNET_TRANSPORT_set_qutoa / send_set_quota
- - hello_wait_timeout
- - transmit_ready
- - transmit_timeout
- - remove_from_any_list / remove_neighbour
- - GNUNET_TRANSPORT_notify_transmit_ready_cancel
- - Testcases for set_quota, timeouts, disconnects, transmit_ready_cancel
- - gnunet-service-transport HELLO validation (how good is our coverage?)
- - direct test of plugins compliance to plugin API
-* core_api:
- - timeout_request
- - solicit_traffic / copy_and_free
- - GNUNET_CORE_peer_configure / produce_configure_message
-* gnunet-service-core:
- - update_window
- - find_client
- - handle_client_request_configure
- - set_key_retry_task
- - align_and_deliver
- - handle_transport_notify_disconnect
-* hostlist (everything)
-* topology (everything)
-
-
-Module features to implement:
-* datacache (needed for DHT)
- - design plugin API
- - implement sqlite-based dstore plugin
- - implement dstore API
- - implement testcases for dstore API
- - implement testcases for dstore plugin
- - implement performance tests
-* TESTING (needed for DV, DHT, Topology)
- - review TESTING API
- - implement library for local testing
- - implement testcases for library
- - implement library for distributed testing
- - implement testcases for distributed testing
-* TOPOLOGY:
- - implement testcases (needs TESTING)
-* HOSTLIST:
- - implement testcases
-* DV (needed for DHT)
- - write DV API
- - implement DV service
- - implement DV library
- - implement DV transport plugin
- - implement testcases
- - implement performance tests
-* DHT (needed for FS)
- - review DHT API
- - implement DHT service (needs DV, DSTORE)
- - implement DHT library
- - implement testcases
- - implement performance tests
-* FS
- - review FS API
- - design network structs (CS)
- - implement FS library
- + URI API
- + getopt API
- + persistence mechanism
- + sharing API
- ~ file-information
- ~ insert
- ~ download
- ~ search
- ~ unindex
- ~ namespaces
- ~ collection
- + directory API
- - design network structs (P2P)
- - datastore request queueing mechanism
- - implement FS service (needs DHT)
- + insert
- + download
- + search
- + unindex
+Implementable right now (but not necessarily important), with caveats
+(unavailable components that will limit what can be implemented right
+away), in order in which they will likely be done:
+* TESTING
+* FS (DHT not available)
+* SETUP
+* DV (distributed testing not available)
+* TBENCH (distributed testing not available)
+* TRACEKIT (distributed testing not available)
+* HTTP transport
+* FRAGMENTATION
+* MySQL / Postgres plugins (datastore, datacache)
+* UPNP
+
+Urgent items (before announcing ng.gnunet.org):
+* FS (basic anonymous FS only)
+ - need way to determine network load (up/down)
+ - need *fast* way to check/update trust in peers
+ (async peerinfo would not be right)
+ - implement FS service
+ + P2P-functions (query, response, peer-disconnect)
+ + CS/P2P-functions (initiate P2P, handle P2P->CS)
- implement testcases
+ URI API
+ getopt API
- + sharing API (needs TESTING)
+ + sharing API
~ file-information
- ~ insert
+ ~ publish (insert, index)
~ download
~ search
~ unindex
- ~ namespaces
- ~ collection
+ directory API
- - implement performance tests (needs TESTING)
+* TESTING (needed for DV, DHT, Topology)
+ - implement library for local testing
+ + modify configuration to allow controlling connections for non-local starts
+ + CORE service does not start with valid peer ID (all zeros) -- testcase fails!
+ + consider changing API for peer-group termination to call continuation when done
+ - implement testcases for library
+ + get test for basic peer start to work!
+ + test basic peer connect
+ + test group start
+* TEST:
+ - topology (needs TESTING)
+ - hostlist (maybe easier with TESTING?)
+* new webpage
+ - prevent?
+ - migrate Mantis!?
+ - run peer => have a 0.9.x hostlist
+ - improve basic documentation (configure, dependencies, what works, etc.)
+ - write minimal welcome/introduction to 0.9.x-development/webpage/status
+=> Deploy(able) development network
+
+
+0.9.0pre0:
+* Module features to implement:
+ - advanced FS API parts
+ + namespaces: fundamental namespace API
+ + search: SBlocks, KSBlocks, probes, notify FS-service of known results
+ + collection
+ + location URIs (publish, search, download)
+ + persistence support (publish, unindex, search, download)
+ + indexing: index-failure-cleanup
+ + download: management/bounding of parallel downloads (for recursive dl.)
+ + datastore reservation (publishing)
+ + gnunet-publish (URI args)
+ + gnunet-download (directory)
+ + gnunet-search (options, incl. namespace search)
+ + gnunet-download (many options)
+ + gnunet-directory (man page, options)
+ + gnunet-pseudonym (all of it)
+ + gnunet-service-fs (remove failing on-demand blocks)
+ + datastore: do active migration support here?
+ - implement adv. FS testcases
+ + insert: sblocks, loc uris
+ + download: loc uris
+ + search: sblocks, skblocks, loc uris
+ + namespaces
+ + collection
+ - implement FS performance tests (needs TESTING)
+ insert
+ download
+ search
+ unindex
-
-
-
-PHASE #3: (Goal: ready for pre-release) [completion-goal: end of 2009]
-
-Module features to implement:
-* setup
- - default generation
- - need to settle basic design; do we want to keep guile?
-* tbench
- - good to have for DV evaluation!
-* tracekit
- - good to have for DV/DHT evaluation!
-* vpn
-
-
-GUIs to implement:
-* gtk
-* qt
-* fuse
-
-
-Plugins to implement:
-* UDP backend (need LIBRARY to support (de)fragmentation!)
-* HTTP backend
-
-
-
-
-
-Minor TODO items / known bugs:
-* UTIL:
- - crypto_hash: use libgcrypt (supports SHA-512 since 2003)
- - container_bloomfilter: improve efficiency (see FIXME)
-* SERVER:
- - inefficient memmove
-* TRANSPORT:
- - transport_api: support forcing disconnects through low quotas!
- (required for working F2F support!)
- - API: consider having core provide deadline information for each message
- (likely important for DV plugin which wants to loop back!)
- - implement transport API to pretty-print transport address
- + transport_api extension (API extension!)
- + service-transport extension (protocol extension)
- - add calls to statistics in various places
- - implement gnunet-transport (transport configurator / tester)
- - UPnP-based IP detection
- (Note: build library always, build service when libxml2/etc. are available)
- - instantly filter addresses from *other* peers that
- are *equal* to our own address + port (i.e., localhost:2086). We
- no longer filter those for outgoing (helps with loopback testing
- and keeps the code clean), but we should filter strictly *impossible*
- incoming addresses! This is for efficiency, not correctness.
- - We currently are happy to take any address told to us in a WELCOME
- to our set of addresses; we should have some minimal threshold-based
- scheme, limiting both the total number of addresses that we accept
- this way as well as requiring multiple confirmations; also, we
- should possibly try to confirm that the given address works for
- us ourselves (loopback-style) before adding it to the list
- [SECURITY issue]
- + we may be able to simplify WELCOME messages (no need to add
- addresses there anymore, but may help to learn them there anyway...).
- + we probably want some kind of voting/counting for learning IP addresses
- (maybe including IP addresses in ads proportional to how often others
- report them? we at least need some protection against >64k HELLOs!),
- + provide a way to give the user a list of "learned" IP addresses and
- a way to easily "veto" addresses off the list!
- => If MiM attacker uses vetoed address, blacklist the specific IP for
- the presumed neighbour!
- - not sure current way of doing ACKs works well-enough
- with unreliable transports where the ACK maybe lost;
- the "is_new" check would then possibly prevent future
- ACKs to be delivered, all while we're happily
- receiving messages from that peer! Worse, the other
- peer won't generate another ACK since it thinks we're
- connected just fine...
- Key questions:
- + How necessary is ACKing in the first place? (alternatives?)
- + Should we transmit ACKs in response to every HELLO? (would that
- fully address the problem?)
- - latency measurements implemented in the transport
- plugins makes it only work for bi-di transports
- and results in code replication
- - should latency be included in the ReceiveCallback and
- NotifyConnect or passed on request?
- - FIXME's with latency being simply set to 0 in a few places
- - Memory leak (running valgrind --trace-children=yes on test_transport_api:
- ==28393== 16 bytes in 1 blocks are indirectly lost in loss record 1 of 5
- ==28393== at 0x4C2260E: malloc (vg_replace_malloc.c:207)
- ==28393== by 0x52343E3: GNUNET_xmalloc_unchecked_ (common_allocation.c:62)
- ==28393== by 0x5234389: GNUNET_xmalloc_ (common_allocation.c:53)
- ==28393== by 0x524458A: GNUNET_NETWORK_socket_create_from_accept (network.c:289)
- ==28393== by 0x524B2DA: ??? (server.c:332)
- ==28393== by 0x524A4C7: ??? (scheduler.c:425)
- ==28393== by 0x524A73D: GNUNET_SCHEDULER_run (scheduler.c:510)
- ==28393== by 0x524FF8C: GNUNET_SERVICE_run (service.c:1326)
- ==28393== by 0x405500: main (gnunet-service-transport.c:2645)
- And also:
- ==28393== 65,744 (65,728 direct, 16 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5
- ==28393== at 0x4C2260E: malloc (vg_replace_malloc.c:207)
- ==28393== by 0x52343E3: GNUNET_xmalloc_unchecked_ (common_allocation.c:62)
- ==28393== by 0x5234389: GNUNET_xmalloc_ (common_allocation.c:53)
- ==28393== by 0x524473E: GNUNET_NETWORK_socket_create_from_accept (network.c:323)
- (rest of trace identical)
-
-* DATASTORE:
- - mysql backend
- - postgres backend
-* SETUP:
- - auto-generate "defaults.conf" using gnunet-setup from "config.scm"
- - integrate all options into "config.scm"
- - change config-file writing to exclude options set to default values
-* ARM:
- - implement exponential back-off for service restarts
- - better tracking of which config changes actually need to cause process restarts by ARM.
- - have way to specify dependencies between services (to manage ARM restarts better)
- - client-API is inefficient since it opens a TCP connection per service that is started
- (instead of re-using connections).
-* CORE:
- - code currently notifies clients about "encrypted" connections being up well before
- we get the encrypted PONG; sometimes this may be OK (for topology killing
- unwanted connnections), but of course not in general. I suspect we want
- to signal on PONG and have topology hook directly into transport to
- kill plaintext connections before they have a chance to become encrypted
- (may require minor hack in transport API)
-* PEERINFO:
- - have gnunet-peerinfo print actual host addresses again
- - add option to gnunet-peerinfo to modify trust value
-* POSTGRES-DB:
- - finish postgres implementation; simplify other SQLs using new stats
-* HTTPS transport
- - PolariSSL for MHD?
- - https integration
-* GAP improvements:
- - active reply route caching design & implementation of service,
- gap extension!
-* HOSTLIST:
- - implement advertising of hostlist URL
- - implement learning of hostlist URLs
-
-
-
-=> PRE-RELEASE
-
-PHASE #4: [completion-goal: mid 2010]
+ - non-anonymous FS service (needs DHT)
+ + basic DHT integration
+ + CS-DHT-functions (DHT-put of LOC)
+ + P2P-functions (DHT-get)
+ - setup (RC-pre0)
+ + default generation
+ + need to settle basic design; do we want to keep guile?
+ - testing (RC-pre0)
+ + testbed creation with topology (needs working F2F topology) [Nate]
+ + testbed with churn [Nate]
+ + implement library for distributed testing [Nate]
+ + implement testcases for distributed testing [Nate]
+ + test basic peer re-configure [Nate]
+ + test topology creation [Nate]
+ + test churn generation [Nate]
+
+0.9.0pre1:
+* Module features to implement:
+ - tbench (RC-pre1)
+ + good to have for DV evaluation!
+ - DV (RC-pre1)
+ + write DV API
+ + implement DV service [Nate & CG]
+ + implement DV library [Nate]
+ + implement DV transport plugin [Nate & CG]
+ + implement testcases [Nate]
+ + implement performance tests [Nate]
+* GUIs to implement:
+ - gtk (RC-pre1)
+ + how to integrate scheduler with GTK event loop!
+
+0.9.0pre2:
+* Module features to implement:
+ - tracekit (RC-pre2)
+ + good to have for DV/DHT evaluation!
+ - DHT (RC-pre2)
+ + implement DHT service (needs DV, DATACACHE)
+ + implement DHT library
+ + implement testcases
+ + implement performance tests
+* GUIs to implement:
+ - fuse (RC-pre2)
+* Plugins to implement:
+ - UDP backend (RC-pre2)
+ + Fragmentation library
+ + actual plugin
+ - HTTP backend (RC-pre2)
+
+0.9.0pre3:
+* GUIs to implement:
+ - qt (RC-pre3)
+ + see discussions @ FISL about integration with event loop!
+* Determine RC bugs and fix those!
+
+0.9.0:
* Documentation
- - Doxygen generation
- - lcov generation
- update man pages
- update webpage documentation
* new webpage:
- expand bibliography
- convert documentation pages to books
- update books (especially for developers)
- - add content type for links and view
- create good Drupal theme for GNUnet
- make a NICE download page and figure out how to
enable developers to publish TGZs nicely
- port "contact" page; add impressum
- add content type for "todo" items?
- DNS activation
-
-
-Stuff to remember:
-* Features eliminated from util:
- - threading (goal: good riddance!)
- - complex logging features [ectx-passing, target-kinds] (goal: good riddance!)
- - complex configuration features [defaults, notifications] (goal: good riddance!)
- - network traffic monitors (goal: eliminate)
- - IPC semaphores (goal: d-bus? / eliminate?)
- - second timers
- - DNS lookup (goal: have async service; issue: still need synchronous resolution in places, current code may not be portable)
- => code shrunk from 61 files to 34, 22k LOC to 15k LOC, 470k to 330k (with symbols)
-* New features in util:
- - scheduler
- - service and program boot-strap code
-* Major changes in util:
- - more expressive server (replaces selector)
-* Open questions:
- - how to integrate scheduler with GTK event loop!
-
-
-
-Test coverage:
-* UTIL : 78.7%
-* HELLO : 93.7%
-* ARM : 69.9%
-* RESOLVER : 60.4%
-* STATISTICS: 82.8%
-* PEERINFO : 71.5%
-* TRANSPORT : 70.9%
-* CORE : 65.8%
-===================
-* TOTAL : 74.9%
-
-Not yet tested:
-* HOSTLIST : 0.0%
-* TOPOLOGY : 0.0%
+* Plugins to implement:
+ - MySQL database backends
+ + datacache
+ + datastore
+ - Postgres database backends
+ + datacache
+ + datastore
+ - vpn
+* Determine RC bugs and fix those!
+
+
+0.9.x:
+* Plugins to implement:
+ - SMTP transport backend
+ - HTTPS transport backend
+ + improved HTTPS support in MHD
+ + actual plugin
projects / oweals / gnunet.git / blobdiff