From 2798cec3ad8c31397ccedef2dfca7f957efa0fa7 Mon Sep 17 00:00:00 2001 From: rexxnor Date: Sun, 20 Jan 2019 13:01:45 +0100 Subject: [PATCH] updated ascension documentation; added developer section --- doc/handbook/chapters/developer.texi | 119 ++++++++++++++++++++++++ doc/handbook/chapters/installation.texi | 5 +- doc/handbook/chapters/user.texi | 9 +- 3 files changed, 125 insertions(+), 8 deletions(-) diff --git a/doc/handbook/chapters/developer.texi b/doc/handbook/chapters/developer.texi index 37e11cb11..af3ac0197 100644 --- a/doc/handbook/chapters/developer.texi +++ b/doc/handbook/chapters/developer.texi @@ -7743,6 +7743,7 @@ record types. * The GNS Client-Service Protocol:: * Hijacking the DNS-Traffic using gnunet-service-dns:: * Serving DNS lookups via GNS on W32:: +* Importing DNS Zones into GNS:: @end menu @node libgnunetgns @@ -8073,6 +8074,124 @@ applications that use alternative means of resolving names (such as sending queries to a DNS server directly by themselves). This includes some of well known utilities, like "ping" and "nslookup". +@node Importing DNS Zones into GNS +@subsection Importing DNS Zones into GNS + +@c %**end of header + +This section will mainly comprise of the challenges and problems faced when +writing the ascension tool. + +When considering to migrate existing into GNS there are a few things to +consider. + +@menu +* Conversions between DNS and GNS:: +* DNS Zone Size:: +* Performance:: +@end menu + +@node Conversions between DNS and GNS +@subsubsection Conversions between DNS and GNS + +The differences between the two name systems lies in the details +and is not visible from the start. For instance an SRV record is converted to a +gnunet only BOX record. + +This is done by building a BOX record from an existing SRV record: + +@example +# _service._proto.name. TTL class SRV priority weight port target +_sip._tcp.example.com. 14000 IN SRV 0 0 5060 www.example.com. +@end example + +Which can be transformed to a GNS BOX record by converting it like this: + +@example +# TTL BOX flags port protocol recordtype priority weight port target +14000 BOX n 5060 6 33 0 0 5060 www.example.com +@end example + +Other records that have such a transformation is the MX record type, as well as +the SOA record type. + +Transformation of a SOA record into GNS works as described in the following +example. Very important to note are the rname and mname keys. +@example +# BIND syntax for a clean SOA record +@ IN SOA master.example.com. hostmaster.example.com. ( + 2017030300 ; serial + 3600 ; refresh + 1800 ; retry + 604800 ; expire + 600 ) ; ttl +# Recordline for adding the record +gnunet-namestore -z example.com -a -n @ -t SOA -V rname=master.example.com \ + mname=hostmaster.example.com 2017030300,3600,1800,604800,600 -e 7200s +@end example + +The transformation of MX records is done in a simple way. +@example +# mail.example.com. 3600 IN MX 10 mail.example.com. +gnunet-namestore -z example.com -n mail -R 3600 MX n 10,mail +@end example + +Finally, one of the biggest struggling points was the NS records that are found +in top level domain zones. The inteded behaviour for those is to add GNS2DNS +records for the zone so that gnunet-gns can resolve the for those domain on it's +own. Also a very important aspect of this is, that gnunet needs to be able to +resolve the nameservers from it's own database. This requires migration of the +DNS GLUE records as well. + +This proved to be quite a challenge to implement, as in GNS every dot is a +strict zone cut. + +The issue was fixed by creating a hierarchical zone structure in GNS and linking +the zones using PKEY records to one another. This allows the resolution of the +nameservers to work within GNS. + +@node DNS Zone Size +@subsubsection DNS Zone Size + +Another very big problem exists with very large zones. When migrating a small +zone the delay between adding of records and their expiry is negligible. However +when working with a TLD zone that has more that 1 million records this delay +becomes a problem. + +Records will start to expire well before the zone has finished migrating. This +causes unwanted anomalies when trying to resolve records. + +A good solution has not been found yet. One of the idea that floated around was +that the records should be added with the s (shadow) flag to keep the records +resolvable even if they expired. However this would introduce the problem of how +to detect if a record has been removed from the zone and would require deletion +of said record(s). + +@node Performance +@subsubsection Performance +The performance when migrating a zone using the ascension tool is limited by a +handful of factors. First of all ascension is written in python3 and calls the +CLI tools of gnunet. Furthermore all the records that are added to the same +label are signed using the zones private key. This signing operation is very +resource heavy and was optimized during development by adding the '-R' +(Recordline) option to gnunet-namestore. This allows to add multiple records +at once using the CLI. + +The result of this was a much faster migration of TLD zones, as most records +with the same label have two nameservers. + +Another improvement that could be made is with the addition of multiple threads +when opening the gnunet CLI tools. This could be implemented by simply creating +more workers in the program but performance improvements were not tested. + +During the entire development of the ascension tool sqlite was used as a +database backend. Other backends need to be tested in the future. + +In conclusion there are many bottlenecks still around in the program, namely the +signing process and the single threaded implementation. In the future a solution +that uses the c api would be cleaner and better. + + @cindex GNS Namecache @node GNS Namecache @section GNS Namecache diff --git a/doc/handbook/chapters/installation.texi b/doc/handbook/chapters/installation.texi index 5ce9805ed..9a64feef7 100644 --- a/doc/handbook/chapters/installation.texi +++ b/doc/handbook/chapters/installation.texi @@ -1712,10 +1712,9 @@ Or installed globally like this (not recommended): $ sudo python3 setup.py install @end example -The advantage of using a virtual environment is, that all the dependencies can be installed separately in different versions without touching your existing python installation. +The advantage of using a virtual environment is, that all the dependencies can be installed separately in different versions without touching your existing python installation and their dependencies. -@c How to reference another section here? -Using the tool is discussed in another section. +Using the tool is discussed in the user section of the documentation. @node Configuring the GNUnet VPN diff --git a/doc/handbook/chapters/user.texi b/doc/handbook/chapters/user.texi index 79c6563a1..76d39b50e 100644 --- a/doc/handbook/chapters/user.texi +++ b/doc/handbook/chapters/user.texi @@ -1632,8 +1632,6 @@ GNS currently supports the following record types: * ABE MASTER:: * RECLAIM OIDC CLIENT:: * RECLAIM OIDC REDIRECT:: -* Synchronizing with legacy DNS:: -* Migrating an existing DNS zone into GNS:: @end menu @node NICK @@ -1897,7 +1895,7 @@ option ``DISABLE'' to ``YES'' in section ``[namecache]''. @node Migrating an existing DNS zone into GNS @subsection Migrating an existing DNS zone into GNS -After installing the tool according to the README file you have these options: +After installing the tool according to the README file you have the following options: @example Ascension @@ -1926,8 +1924,9 @@ $ ascension sy. -ns ns1.tld.sy. This will take a while but after it has finished executing the zone will have been migrated into GNS. -@c remove this once daemonized -Currently this will only snapshot the zone at it's current state and not update it. There is a plan to daemonize the migration process so you don't have to worry that a zone will stay up to date. +The program will continue to run daemon and update once the refresh time specified in the zones SOA record has elapsed. + +At this point it is trivial to write for example a systemd unit file and to enable the service, so that your zone is migrated periodically. @node re@:claim Identity Provider @section re@:claim Identity Provider -- 2.25.1