--- /dev/null
+# Simple AdBlock
+
+A simple DNSMASQ/Unbound-based AdBlocking service for OpenWrt/LEDE Project.
+
+## Features
+
+- Super-fast due to the nature of supported block lists and parallel downloading/processing of the blacklists.
+- Supports hosts files and domains lists for blocking.
+- Everything is configurable from Web UI.
+- Allows you to easily add your own domains to whitelist or blacklist.
+- Allows you to easily add URLs to your own blocked hosts or domains lists to block/whitelist (just put whitelisted domains one per line in the file you're linking).
+- Supports multiple modes of AdBlocking implementations with DNSMASQ and Unbound.
+- Doesn't stay in memory -- creates the list of blocked domains and then uses DNSMASQ/Unbound and firewall rules to serve NXDOMAIN or 127.0.0.1 reply or to reject access (depending on settings) for blocked domains.
+- As some of the default lists are using https, reliably works with either wget/libopenssl, uclient-fetch/libustream-mbedtls or curl.
+- Very lightweight and easily hackable, the whole script is just one ```/etc/init.d/simple-adblock``` file.
+- Retains the downloaded/sorted AdBlocking list on service stop and reuses it on service start (use ```dl``` command if you want to force re-download of the list).
+- Has an option to store a compressed copy of the AdBlocking list in persistent memory which survives reboots.
+- Blocks ads served over https (unlike PixelServ-derived solutions).
+- Blocks ads inside browsers with [DNS-over-HTTPS proxy](https://en.wikipedia.org/wiki/DNS_over_HTTPS) built-in, like [Mozilla Firefox](https://support.mozilla.org/en-US/kb/firefox-dns-over-https#w_about-dns-over-https) or [Google Chrome/Chromium](https://blog.chromium.org/2019/09/experimenting-with-same-provider-dns.html) -- with the ```dnsmasq.ipset``` option.
+- Proudly made in Canada, using locally-sourced electrons.
+
+If you want a more robust AdBlocking, supporting free memory detection and complex block lists, supporting IDN, check out [net/adblock](https://github.com/openwrt/packages/tree/master/net/adblock/files).
+
+## Screenshots (luci-app-simple-adblock)
+
+Service Status
+
+
+
+Configuration - Basic Configuration
+
+
+
+Configuration - Advanced Configuration
+
+
+
+Whitelist and Blocklist Management
+
+
+
+## Requirements
+
+This service requires the following packages to be installed on your router: ```dnsmasq``` or ```dnsmasq-full``` or ```unbound``` and either ```ca-certificates```, ```wget``` and ```libopenssl``` (for OpenWrt 15.05.1) or ```uclient-fetch``` and ```libustream-mbedtls``` (for LEDE Project and OpenWrt 18.06.xx or newer). Additionally installation of ```coreutils-sort``` is highly recommended as it speeds up blocklist processing.
+
+To satisfy the requirements for connect to your router via ssh and run the following commands:
+
+### How to use DNSMASQ ipset
+
+The ```dnsmasq.ipset``` option requires you to install ```ipset``` and ```dnsmasq-full``` instead of the ```dnsmasq```. To do that, connect to your router via ssh and run the following command:
+
+```sh
+opkg update; opkg remove dnsmasq; opkg install dnsmasq-full ipset;
+```
+
+### OpenWrt 15.05.1 Requirements
+
+```sh
+opkg update; opkg install ca-certificates wget libopenssl coreutils-sort dnsmasq
+```
+
+### LEDE Project 17.01.x and OpenWrt 18.xx (or newer) Requirements
+
+```sh
+opkg update; opkg install uclient-fetch libustream-mbedtls coreutils-sort dnsmasq
+```
+
+### IPv6 Support
+
+For IPv6 support additionally install ```ip6tables-mod-nat``` and ```kmod-ipt-nat6``` packages from Web UI or run the following in the command line:
+
+```sh
+opkg update; opkg install ip6tables-mod-nat kmod-ipt-nat6
+```
+
+### Speed Up Blocklist Processing
+
+The ```coreutils-sort``` is an optional, but recommended package as it speeds up sorting and removing duplicates from the merged list dramatically. If opkg complains that it can't install ```coreutils-sort``` because /usr/bin/sort is already provided by busybox, you can run ```opkg --force-overwrite install coreutils-sort```.
+
+## Unmet Dependencies
+
+If you are running a development (trunk/snapshot) build of OpenWrt/LEDE Project on your router and your build is outdated (meaning that packages of the same revision/commit hash are no longer available and when you try to satisfy the [requirements](#requirements) you get errors), please flash either current LEDE release image or current development/snapshot image.
+
+## How To Install
+
+Install ```simple-adblock``` and ```luci-app-simple-adblock``` packages from Web UI or run the following in the command line:
+
+```sh
+opkg update; opkg install simple-adblock luci-app-simple-adblock
+```
+
+If ```simple-adblock``` and ```luci-app-simple-adblock``` packages are not found in the official feed/repo for your version of OpenWrt/LEDE Project, you will need to [add a custom repo to your router](https://github.com/stangri/openwrt_packages/blob/master/README.md#on-your-router) first.
+
+## Default Settings
+
+Default configuration has service disabled (use Web UI to enable/start service or run ```uci set simple-adblock.config.enabled=1; uci commit simple-adblock;```) and selected ad/malware lists suitable for routers with 64Mb RAM.
+
+If your router has less then 64Mb RAM, edit the configuration file, located at ```/etc/config/simple-adblock```. The configuration file has lists in ascending order starting with smallest ones and each list has a preceding comment indicating its size, comment out or delete the lists you don't want or your router can't handle.
+
+## How To Customize
+
+You can use Web UI (found in Services/Simple AdBlock) to add/remove/edit links to:
+
+- [hosts files](https://en.wikipedia.org/wiki/Hosts_(file)) (127.0.0.1 or 0.0.0.0 followed by space and domain name per line) to be blocked.
+- domains lists (one domain name per line) to be blocked.
+- domains lists (one domain name per line) to be whitelisted. It is useful if you want to run ```simple-adblock``` on multiple routers and maintain one centralized whitelist which you can publish on a web-server.
+
+Please note that these lists **must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix. Some of the top block lists (both hosts files and domains lists) suitable for routers with at least 8MB RAM are used in the default ```simple-adblock``` installation.
+
+You can also use Web UI to add individual domains to be blocked or whitelisted.
+
+If you want to use CLI to customize ```simple-adblock``` config, refer to the [Customization Settings](#customization-settings) section.
+
+## How To Use
+
+Once the service is enabled in the [config file](#default-settings), run ```/etc/init.d/simple-adblock start``` to start the service. Either ```/etc/init.d/simple-adblock restart``` or ```/etc/init.d/simple-adblock reload``` will only restart the service and/or re-download the lists if there were relevant changes in the config file since the last successful start. Had the previous start resulted in any error, either ```/etc/init.d/simple-adblock start```, ```/etc/init.d/simple-adblock restart``` or ```/etc/init.d/simple-adblock reload``` will attempt to re-download the lists.
+
+If you want to force simple-adblock to re-download the lists, run ```/etc/init.d/simple-adblock dl```.
+
+If you want to check if the specific domain (or part of the domain name) is being blocked, run ```/etc/init.d/simple-adblock check test-domain.com```.
+
+## Configuration Settings
+
+In the Web UI the ```simple-adblock``` settings are split into ```basic``` and ```advanced``` settings. The full list of configuration parameters of ```simple-adblock.config``` section is:
+
+|Web UI Section|Parameter|Type|Default|Description|
+| --- | --- | --- | --- | --- |
+|Basic|enabled|boolean|0|Enable/disable the ```simple-adblock``` service.|
+|Basic|verbosity|integer|2|Can be set to 0, 1 or 2 to control the console and system log output verbosity of the ```simple-adblock``` service.|
+|Basic|force_dns|boolean|1|Force router's DNS to local devices which may have different/hardcoded DNS server settings. If enabled, creates a firewall rule to intercept DNS requests from local devices to external DNS servers and redirect them to router.|
+|Basic|led|string|none|Use one of the router LEDs to indicate the AdBlocking status.|
+|Advanced|dns|string|dnsmasq.servers|DNS resolution option. See [table below](#dns-resolution-option) for addtional information.|
+|Advanced|ipv6_enabled|boolean|0|Add IPv6 entries to block-list if ```dnsmasq.addnhosts``` is used. This option is only visible in Web UI if the ```dnsmasq.addnhosts``` is selected as the DNS resolution option.|
+|Advanced|boot_delay|integer|120|Delay service activation for that many seconds on boot up. You can shorten it to 10-30 seconds on modern fast routers. Routers with built-in modems may require longer boot delay.|
+|Advanced|download_timeout|integer|10|Time-out downloads if no reply received within that many last seconds.|
+|Advanced|curl_retry|integer|3|If ```curl``` is installed and detected, attempt that many retries for failed downloads.|
+|Advanced|parallel_downloads|boolean|1|If enabled, all downloads are completed concurrently, if disabled -- sequentioally. Concurrent downloads dramatically speed up service loading.|
+|Advanced|debug|boolean|0|If enabled, output service full debug to ```/tmp/simple-adblock.log```. Please note that the debug file may clog up the router's RAM on some devices. Use with caution.|
+|Advanced|allow_non_ascii|boolean|0|Enable support for non-ASCII characters in the final AdBlocking file. Only enable if your target service supports non-ASCII characters. If you enable this on the system where DNS resolver doesn't support non-ASCII characters, it will crash. Use with caution.|
+|Advanced|compressed_cache|boolean|0|Create compressed cache of the AdBlocking file in router's persistent memory. Only recommended to be used on routers with large ROM and/or routers with metered/flaky internet connection.|
+||whitelist_domain|list/string||List of white-listed domains.|
+||whitelist_domains_url|list/string||List of URL(s) to text files containing white-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix. Useful if you want to keep/publish a single white-list for multiple routers.|
+||blacklist_domains_url|list/string||List of URL(s) to text files containing black-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix.|
+||blacklist_hosts_url|list/string||List of URL(s) to [hosts files](https://en.wikipedia.org/wiki/Hosts_(file)) containing black-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix.|
+
+### DNS Resolution Option
+
+Currently supported options are:
+
+|Option|Explanation|
+| --- | --- |
+|```dnsmasq.addnhosts```|Creates the DNSMASQ additional hosts file ```/var/run/simple-adblock.addnhosts``` and modifies DNSMASQ settings, so that DNSMASQ resolves all blocked domains to "local machine": 127.0.0.1. This option doesn't allow block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), so it results in a much larger block-list file, but, unlike other DNSMASQ-based options, it has almost no effect on the DNS look up speed. This option also allows quick reloads of DNSMASQ on block-list updates.|
+|```dnsmasq.conf```|Creates the DNSMASQ config file ```/var/dnsmasq.d/simple-adblock``` so that DNSMASQ replies with NXDOMAIN: "domain not found". This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file. This option will slow down DNS look up speed somewhat.|
+|```dnsmasq.ipset```|Creates the DNSMASQ ipset file ```/var/dnsmasq.d/simple-adblock.ipset``` and the firewall rule to reject the matching requests. This is the only option for AdBlocking if you're using a browser with [DNS-over-HTTPS proxy](https://en.wikipedia.org/wiki/DNS_over_HTTPS) built-in, like [Mozilla Firefox](https://support.mozilla.org/en-US/kb/firefox-dns-over-https#w_about-dns-over-https) or [Google Chrome/Chromium](https://blog.chromium.org/2019/09/experimenting-with-same-provider-dns.html). This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file. This option requires you install ```dnsmasq-full``` and ```ipset``` [as described here](#how-to-use-dnsmasq-ipset).<br/>PLEASE NOTE, that unlike other options which are truly domain name based blocking, this is essentially an IP address based blocking, ie: if you try to block ```google-analytics.com``` with this option, it may also block/break things like YouTube, Hangouts and other Google services if they share IP address(es) with ```google-analytics.com```.|
+|```dnsmasq.servers```|Creates the DNSMASQ servers file ```/var/run/simple-adblock.servers``` and modifies DNSMASQ settings so that DNSMASQ replies with NXDOMAIN: "domain not found". This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file. This option will slow down DNS look up speed somewhat. This is a default setting as it results in the smaller block-file and allows quick reloads of DNSMASQ.|
+|```unbound.adb_list```|Creates the Unbound config file ```/var/lib/unbound/adb_list.simple-adblock``` so that Unbound replies with NXDOMAIN: "domain not found". This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file.|
+
+## How Does It Work
+
+This service downloads (and processes in the background, removing comments and other useless data) lists of hosts and domains to be blocked, combines those lists into one big block list, removes duplicates and sorts it and then removes your whitelisted domains from the block list before converting to to DNSMASQ/Unbound-compatible file and restarting DNSMASQ/Unbound if needed. The result of the process is that DNSMASQ/Unbound return NXDOMAIN or 127.0.0.1 (depending on settings) for the blocked domains.
+
+If you specify ```google.com``` as a domain to be whitelisted, you will have access to ```google.com```, ```www.google.com```, ```analytics.google.com```, but not fake domains like ```email-google.com``` or ```drive.google.com.verify.signin.normandeassociation.com``` for example. If you only want to allow ```www.google.com``` while blocking all other ```google.com``` subdomains, just specify ```www.google.com``` as domain to be whitelisted.
+
+In general, whatever domain is specified to be whitelisted; it, along with with its subdomains will be whitelisted, but not any fake domains containing it.
+
+## How It Does Not Work
+
+For most of the [DNS Resolution Options](#dns-resolution-option) to work, your local LAN clients need to be set to use your router's DNS (by default ```192.168.1.1```). The ```dnsmasq.addnhosts``` is the only option which can help you block ads if your local LAN clients are NOT using your router's DNS. There are multiple ways your local LAN clients can be set to NOT use your router's DNS:
+
+ 1. Hardcoded on the device. Some Android Lollipop 5.0 phones, some media-centric tablets and some streaming devices for example are known to have hardcoded DNS servers and they ignore your router's DNS settings. You can fix this by either:
+ - Rooting your device and changing it from hardcoded DNS servers to obtaining DNS servers from DHCP.
+ - Enabling ```simple-adblock```'s ```force_dns``` setting to override the hardcoded DNS on your device.
+ 2. Manually set on the device. Instead of setting your device to obtain the DNS settings via DHCP, you can set the DNS servers manually. There are some guides online which recommend manually changing the DNS servers on your computer to Google's (8.8.8.8) or Cloudflare's (1.1.1.1) or OpenDNS (208.67.222.222). You can fix this by either:
+ - Changing the on-device DNS settings from manual to obtaining DNS servers from DHCP and changing your [router's DNS settings](https://openwrt.org/docs/guide-user/base-system/dhcp#all_options) to use the DNS from Google, Cloudflare or OpenDNS respectively.
+ - Enabling ```simple-adblock```'s ```force_dns``` setting to override the hardcoded DNS on your device.
+ 3. Sent to your device from router via [DHCP Options](https://openwrt.org/docs/guide-user/base-system/dhcp_configuration#dhcp_options). You can fix this by either:
+ - Removing [DHCP Options](https://openwrt.org/docs/guide-user/base-system/dhcp_configuration#dhcp_options) 5 and 6 from your router's ```/etc/config/dhcp``` file.
+ - Enabling ```simple-adblock```'s ```force_dns``` setting to override the hardcoded DNS on your device.
+ 4. By using the DNS-over-TLS, DNS-over-HTTPS or DNSCrypt on your local device or (if supported) by browser on your local device. You can fix this only by:
+ - Stopping/removing/disabling DNS-over-TLS, DNS-over-HTTPS or DNSCrypt on your local device and using the secure DNS on your router instead. There are merits to all three of the options above, I can recommend the ```https_dns_proxy``` and ```luci-app-https_dns_proxy``` packages for enabling DNS-over-HTTPS on your router.
+
+## Documentation / Discussion
+
+Please head to [OpenWrt Forum](https://forum.openwrt.org/t/simple-adblock-fast-lean-and-fully-uci-luci-configurable-adblocking/1327/) for discussion of this package.
+
+## Thanks
+
+I'd like to thank everyone who helped create, test and troubleshoot this service. Special thanks to [@hnyman](https://github.com/hnyman) for general package/luci guidance, [@dibdot](https://github.com/dibdot) for general guidance and block-list optimization code, [@ckuethe](https://github.com/ckuethe) for the curl support, non-ASCII filtering and compressed cache code, [@EricLuehrsen](https://github.com/EricLuehrsen) for the Unbound support information, [@mushoz]( https://github.com/mushoz) for performance testing and [@phasecat](https://forum.openwrt.org/u/phasecat/summary) for submitting various bugs and testing.
--- /dev/null
+#!/bin/sh /etc/rc.common
+# Copyright 2017-2019 Stan Grishin (stangri@melmac.net)
+# shellcheck disable=SC2039
+# shellcheck disable=SC1091
+PKG_VERSION='dev-test'
+
+export START=94
+export USE_PROCD=1
+export LC_ALL=C
+
+export EXTRA_COMMANDS='check dl killcache sizes status'
+export EXTRA_HELP=' check Checks if specified domain is found in current blacklist
+ dl Force-redownloads all the list
+ sizes Shows the file-sizes of enabled block-lists (by downloading them one by one)
+ status Shows the service last-run status'
+
+readonly packageName='simple-adblock'
+readonly serviceName="$packageName $PKG_VERSION"
+readonly addnhostsFile="/var/run/${packageName}.addnhosts"
+readonly addnhostsCache="/var/run/${packageName}.addnhosts.cache"
+readonly addnhostsGzip="/etc/${packageName}.addnhosts.gz"
+readonly addnhostsOutputFilter='s|^|127.0.0.1 |;s|$||'
+readonly addnhostsOutputFilterIPv6='s|^|:: |;s|$||'
+readonly dnsmasqFile="/var/dnsmasq.d/${packageName}"
+readonly dnsmasqCache="/var/run/${packageName}.dnsmasq.cache"
+readonly dnsmasqGzip="/etc/${packageName}.dnsmasq.gz"
+readonly dnsmasqOutputFilter='s|^|local=/|;s|$|/|'
+readonly ipsetFile="/var/dnsmasq.d/${packageName}.ipset"
+readonly ipsetCache="/var/run/${packageName}.ipset.cache"
+readonly ipsetGzip="/etc/${packageName}.ipset.gz"
+readonly ipsetOutputFilter='s|^|ipset=/|;s|$|/adb|'
+readonly serversFile="/var/run/${packageName}.servers"
+readonly serversCache="/var/run/${packageName}.servers.cache"
+readonly serversGzip="/etc/${packageName}.servers.gz"
+readonly serversOutputFilter='s|^|server=/|;s|$|/|'
+readonly unboundFile="/var/lib/unbound/adb_list.${packageName}"
+readonly unboundCache="/var/run/${packageName}.unbound.cache"
+readonly unboundGzip="/etc/${packageName}.unbound.gz"
+readonly unboundOutputFilter='s|^|local-zone: "|;s|$|" static|'
+readonly A_TMP="/var/${packageName}.hosts.a.tmp"
+readonly B_TMP="/var/${packageName}.hosts.b.tmp"
+readonly PIDFile="/var/run/${packageName}.pid"
+readonly jsonFile="/var/run/${packageName}.json"
+readonly hostsFilter='/localhost/d;/^#/d;/^[^0-9]/d;s/^0\.0\.0\.0.//;s/^127\.0\.0\.1.//;s/[[:space:]]*#.*$//;s/[[:cntrl:]]$//;s/[[:space:]]//g;/[`~!@#\$%\^&\*()=+;:"'\'',<>?/\|[{}]/d;/]/d;/\./!d;/^$/d;/[^[:alnum:]_.-]/d;'
+readonly domainsFilter='/^#/d;s/[[:space:]]*#.*$//;s/[[:space:]]*$//;s/[[:cntrl:]]$//;/[[:space:]]/d;/[`~!@#\$%\^&\*()=+;:"'\'',<>?/\|[{}]/d;/]/d;/\./!d;/^$/d;/[^[:alnum:]_.-]/d;'
+readonly checkmark='\xe2\x9c\x93'
+readonly xmark='\xe2\x9c\x97'
+readonly _OK_='\033[0;32m\xe2\x9c\x93\033[0m'
+readonly _FAIL_='\033[0;31m\xe2\x9c\x97\033[0m'
+readonly __OK__='\033[0;32m[\xe2\x9c\x93]\033[0m'
+readonly __FAIL__='\033[0;31m[\xe2\x9c\x97]\033[0m'
+readonly _ERROR_='\033[0;31mERROR\033[0m'
+readonly statusSuccess='Success'
+readonly statusFail='Fail'
+readonly statusDownloading='Downloading'
+readonly statusReloading='Reloading'
+readonly statusRestarting='Restarting'
+readonly statusStarting='Starting'
+readonly statusForceReloading='Force-Reloading'
+readonly statusProcessing='Processing'
+readonly statusStopped='Stopped'
+readonly sharedMemoryError="/dev/shm/$packageName-error"
+readonly sharedMemoryOutput="/dev/shm/$packageName-output"
+
+create_lock() { [ -e "$PIDFile" ] && return 1; touch "$PIDFile"; }
+remove_lock() { [ -e "$PIDFile" ] && rm -f "$PIDFile"; }
+trap remove_lock EXIT
+output_ok() { output 1 "$_OK_"; output 2 "$__OK__\\n"; }
+output_okn() { output 1 "$_OK_\\n"; output 2 "$__OK__\\n"; }
+output_fail() { output 1 "$_FAIL_"; output 2 "$__FAIL__\\n"; }
+output_failn() { output 1 "$_FAIL_\\n"; output 2 "$__FAIL__\\n"; }
+str_replace() { printf "%b" "$1" | sed -e "s/$(printf "%b" "$2")/$(printf "%b" "$3")/g"; }
+str_contains() { test "$1" != "$(str_replace "$1" "$2" '')"; }
+compare_versions() { test "$(printf '%s\n' "$@" | sort -V | head -n 1)" != "$1"; }
+is_chaos_calmer() { ubus -S call system board | grep -q 'Chaos Calmer'; }
+is_ipset_procd() { compare_versions "$(sed -ne 's/^Version: //p' /usr/lib/opkg/info/firewall.control)" "2019-09-18"; }
+led_on(){ if [ -n "${1}" ] && [ -e "${1}/trigger" ]; then echo 'default-on' > "${1}/trigger" 2>&1; fi; }
+led_off(){ if [ -n "${1}" ] && [ -e "${1}/trigger" ]; then echo 'none' > "${1}/trigger" 2>&1; fi; }
+dnsmasq_hup() { killall -q -HUP dnsmasq; }
+dnsmasq_kill() { killall -q -KILL dnsmasq; }
+dnsmasq_restart() { /etc/init.d/dnsmasq restart >/dev/null 2>&1; }
+unbound_restart() { /etc/init.d/unbound restart >/dev/null 2>&1; }
+
+output() {
+# Can take a single parameter (text) to be output at any verbosity
+# Or target verbosity level and text to be output at specifc verbosity
+ local msg memmsg
+ if [ $# -ne 1 ]; then
+ if [ $((verbosity & $1)) -gt 0 ] || [ "$verbosity" = "$1" ]; then shift; else return 0; fi
+ fi
+ [ -t 1 ] && printf "%b" "$1"
+ msg="$(printf "%s" "$(str_replace "$1" "$serviceName " "service ")" | sed 's|\\033\[[0-9]\?;\?[0-9]\?[0-9]\?m||g')";
+ if [ "$(printf "%b" "$msg" | wc -l)" -gt 0 ]; then
+ [ -s "$sharedMemoryOutput" ] && memmsg="$(cat "$sharedMemoryOutput")"
+ logger -t "${packageName:-service} [$$]" "$(printf "%b" "${memmsg}${msg}")"
+ rm -f "$sharedMemoryOutput"
+ else
+ printf "%b" "$msg" >> "$sharedMemoryOutput"
+ fi
+}
+
+export serviceEnabled forceDNS parallelDL debug allowIDN compressedCache
+export targetDNS bootDelay dlTimeout curlRetry verbosity led dnsInstance
+export whitelist_domains blacklist_domains
+export whitelist_domains_urls blacklist_domains_urls blacklist_hosts_urls
+export wan_if wan_gw wanphysdev dl_command serviceStatus dl_flag
+export outputFilter outputFilterIPv6 outputFile outputGzip outputCache ipv6Enabled
+
+load_package_config() {
+ config_load "$packageName"
+ config_get_bool serviceEnabled 'config' 'enabled' 1
+ config_get_bool forceDNS 'config' 'force_dns' 1
+ config_get_bool parallelDL 'config' 'parallel_downloads' 1
+ config_get_bool debug 'config' 'debug' 0
+ config_get_bool compressedCache 'config' 'compressed_cache' 0
+ config_get_bool ipv6Enabled 'config' 'ipv6_enabled' 0
+ config_get bootDelay 'config' 'boot_delay' '120'
+ config_get dlTimeout 'config' 'download_timeout' '20'
+ config_get curlRetry 'config' 'curl_retry' '3'
+ config_get verbosity 'config' 'verbosity' '2'
+ config_get led 'config' 'led'
+ config_get targetDNS 'config' 'dns' 'dnsmasq.servers'
+ config_get dnsInstance 'config' 'dns_instance' '0'
+ config_get whitelist_domains 'config' 'whitelist_domain'
+ config_get blacklist_domains 'config' 'blacklist_domain'
+ config_get whitelist_domains_urls 'config' 'whitelist_domains_url'
+ config_get blacklist_domains_urls 'config' 'blacklist_domains_url'
+ config_get blacklist_hosts_urls 'config' 'blacklist_hosts_url'
+
+ if [ "$targetDNS" != 'dnsmasq.addnhosts' ] && [ "$targetDNS" != 'dnsmasq.conf' ] && \
+ [ "$targetDNS" != 'dnsmasq.servers' ] && [ "$targetDNS" != 'unbound.adb_list' ] && \
+ [ "$targetDNS" != 'dnsmasq.ipset' ] ; then
+ targetDNS='dnsmasq.servers'
+ fi
+
+ case "$targetDNS" in
+ dnsmasq.addnhosts)
+ outputFilter="$addnhostsOutputFilter"
+ outputFile="$addnhostsFile"
+ outputCache="$addnhostsCache"
+ outputGzip="$addnhostsGzip"
+ [ "$ipv6Enabled" -gt 0 ] && outputFilterIPv6="$addnhostsOutputFilterIPv6"
+ rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip"
+ rm -f "$ipsetFile" "$ipsetCache" "$ipsetGzip"
+ rm -f "$serversFile" "$serversCache" "$serversGzip"
+ rm -f "$unboundFile" "$unboundCache" "$unboundGzip"
+ ;;
+ dnsmasq.conf)
+ outputFilter="$dnsmasqOutputFilter"
+ outputFile="$dnsmasqFile"
+ outputCache="$dnsmasqCache"
+ outputGzip="$dnsmasqGzip"
+ rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip"
+ rm -f "$ipsetFile" "$ipsetCache" "$ipsetGzip"
+ rm -f "$serversFile" "$serversCache" "$serversGzip"
+ rm -f "$unboundFile" "$unboundCache" "$unboundGzip"
+ ;;
+ dnsmasq.ipset)
+ outputFilter="$ipsetOutputFilter"
+ outputFile="$ipsetFile"
+ outputCache="$ipsetCache"
+ outputGzip="$ipsetGzip"
+ rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip"
+ rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip"
+ rm -f "$serversFile" "$serversCache" "$serversGzip"
+ rm -f "$unboundFile" "$unboundCache" "$unboundGzip"
+ ;;
+ dnsmasq.servers)
+ outputFilter="$serversOutputFilter"
+ outputFile="$serversFile"
+ outputCache="$serversCache"
+ outputGzip="$serversGzip"
+ rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip"
+ rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip"
+ rm -f "$ipsetFile" "$ipsetCache" "$ipsetGzip"
+ rm -f "$unboundFile" "$unboundCache" "$unboundGzip"
+ ;;
+ unbound.adb_list)
+ outputFilter="$unboundOutputFilter"
+ outputFile="$unboundFile"
+ outputCache="$unboundCache"
+ outputGzip="$unboundGzip"
+ rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip"
+ rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip"
+ rm -f "$ipsetFile" "$ipsetCache" "$ipsetGzip"
+ rm -f "$serversFile" "$serversCache" "$serversGzip"
+ ;;
+ esac
+ if [ -z "${verbosity##*[!0-9]*}" ] || [ "$verbosity" -lt 0 ] || [ "$verbosity" -gt 2 ]; then
+ verbosity=1
+ fi
+ . /lib/functions/network.sh
+ . /usr/share/libubox/jshn.sh
+ # Prefer curl because it supports the file: scheme.
+ if [ -x /usr/bin/curl ] ; then
+ dl_command="curl --insecure --retry $curlRetry --connect-timeout $dlTimeout --silent"
+ dl_flag="-o"
+ else
+ dl_command="wget --no-check-certificate --timeout $dlTimeout -q"
+ dl_flag="-O"
+ fi
+ led="${led:+/sys/class/leds/$led}"
+}
+
+is_enabled() {
+ load_package_config
+
+ if [ "$debug" -ne 0 ]; then
+ exec 1>>/tmp/simple-adblock.log
+ exec 2>&1
+ set -x
+ fi
+
+ if [ "$serviceEnabled" -eq 0 ]; then
+ case "$1" in
+ on_start)
+ output "$packageName is currently disabled.\\n"
+ output "Run the following commands before starting service again:\\n"
+ output "uci set ${packageName}.config.enabled='1'; uci commit $packageName;\\n"
+ ;;
+ esac
+ return 1
+ fi
+
+ case $targetDNS in
+ dnsmasq.addnhosts | dnsmasq.conf | dnsmasq.ipset | dnsmasq.servers)
+ if dnsmasq -v 2>/dev/null | grep -q 'no-IDN' || ! dnsmasq -v 2>/dev/null | grep -q -w 'IDN'; then
+ allowIDN=0
+ else
+ allowIDN=1
+ fi
+ ;;
+ unbound.adb_list)
+ allowIDN=1;;
+ esac
+
+ case $targetDNS in
+ dnsmasq.ipset)
+ if dnsmasq -v 2>/dev/null | grep -q 'no-ipset' || ! dnsmasq -v 2>/dev/null | grep -q -w 'ipset'; then
+ output "$_ERROR_: DNSMASQ ipset support is enabled in $packageName, but DNSMASQ is either not installed or installed DNSMASQ does not support ipsets!\\n"
+ targetDNS='dnsmasq.servers'
+ fi
+ if ! ipset help hash:net >/dev/null 2>&1; then
+ output "$_ERROR_: DNSMASQ ipset support is enabled in $packageName, but ipset is either not installed or installed ipset does not support 'hash:net' type!\\n"
+ targetDNS='dnsmasq.servers'
+ fi
+ ;;
+ esac
+
+ [ ! -d "${outputFile%/*}" ] && mkdir -p "${outputFile%/*}"
+ [ ! -d "${outputCache%/*}" ] && mkdir -p "${outputFile%/*}"
+ [ ! -d "${outputGzip%/*}" ] && mkdir -p "${outputFile%/*}"
+ cacheOps 'testGzip' && return 0
+ network_flush_cache; network_find_wan wan_if; network_get_gateway wan_gw "$wan_if";
+ [ -n "$wan_gw" ] && return 0
+ output "$_ERROR_: $serviceName failed to discover WAN gateway.\\n"; return 1;
+}
+
+dnsOps() {
+ local param output_text
+ case $1 in
+ on_start)
+ if [ ! -s "$outputFile" ]; then
+ tmpfs set status "$statusFail"
+ tmpfs add error "Error: Failed to create $outputFile file."
+ output "$_ERROR_: $serviceName failed to create its data file!\\n"
+ return 1
+ fi
+ case "$targetDNS" in
+ dnsmasq.addnhosts)
+ if [ "$(uci -q get dhcp.@dnsmasq["$dnsInstance"].serversfile)" = "$serversFile" ]; then
+ uci -q del dhcp.@dnsmasq["$dnsInstance"].serversfile
+ fi
+ if ! uci -q get dhcp.@dnsmasq["$dnsInstance"].addnhosts | grep -q "$addnhostsFile"; then
+ uci add_list dhcp.@dnsmasq["$dnsInstance"].addnhosts="$addnhostsFile"
+ fi
+ param=dnsmasq_hup
+ output_text='Reloading DNSMASQ'
+ ;;
+ dnsmasq.conf)
+ uci -q del_list dhcp.@dnsmasq["$dnsInstance"].addnhosts="$addnhostsFile"
+ if [ "$(uci -q get dhcp.@dnsmasq["$dnsInstance"].serversfile)" = "$serversFile" ]; then
+ uci -q del dhcp.@dnsmasq["$dnsInstance"].serversfile
+ fi
+ param=dnsmasq_restart
+ output_text='Restarting DNSMASQ'
+ ;;
+ dnsmasq.ipset)
+ uci -q del_list dhcp.@dnsmasq["$dnsInstance"].addnhosts="$addnhostsFile"
+ if [ "$(uci -q get dhcp.@dnsmasq["$dnsInstance"].serversfile)" = "$serversFile" ]; then
+ uci -q del dhcp.@dnsmasq["$dnsInstance"].serversfile
+ fi
+ param=dnsmasq_restart
+ output_text='Restarting DNSMASQ'
+ ;;
+ dnsmasq.servers)
+ uci -q del_list dhcp.@dnsmasq["$dnsInstance"].addnhosts="$addnhostsFile"
+ if [ "$(uci -q get dhcp.@dnsmasq["$dnsInstance"].serversfile)" != "$serversFile" ]; then
+ uci set dhcp.@dnsmasq["$dnsInstance"].serversfile="$serversFile"
+ fi
+ param=dnsmasq_hup
+ output_text='Reloading DNSMASQ'
+ ;;
+ unbound.adb_list)
+ uci -q del_list dhcp.@dnsmasq["$dnsInstance"].addnhosts="$addnhostsFile"
+ if [ "$(uci -q get dhcp.@dnsmasq["$dnsInstance"].serversfile)" = "$serversFile" ]; then
+ uci -q del dhcp.@dnsmasq["$dnsInstance"].serversfile
+ fi
+ param=unbound_restart
+ output_text='Restarting Unbound'
+ ;;
+ esac
+ if [ -n "$(uci changes dhcp)" ]; then
+ uci commit dhcp
+ if [ "$param" = 'unbound_restart' ]; then
+ param='dnsmasq_restart; unbound_restart;'
+ output_text='Restarting Unbound/DNSMASQ'
+ else
+ param=dnsmasq_restart
+ output_text='Restarting DNSMASQ'
+ fi
+ fi
+ output 1 "$output_text "
+ output 2 "$output_text "
+ tmpfs set message "$output_text"
+ if eval "$param"; then
+ tmpfs set status "$statusSuccess"
+ led_on "$led"
+ output_okn
+ else
+ output_fail
+ tmpfs set status "$statusFail"
+ tmpfs add error "Error: $output_text error."
+ output "$_ERROR_: $serviceName $output_text error!\\n"
+ return 1
+ fi
+ ;;
+ on_stop)
+ case "$targetDNS" in
+ dnsmasq.addnhosts | dnsmasq.servers)
+ param=dnsmasq_hup
+ ;;
+ dnsmasq.conf | dnsmasq.ipset)
+ param=dnsmasq_restart
+ ;;
+ unbound.adb_list)
+ param=unbound_restart
+ ;;
+ esac
+ if [ -n "$(uci changes dhcp)" ]; then
+ uci -q commit dhcp
+ if [ "$param" = 'unbound_restart' ]; then
+ param='dnsmasq_restart; unbound_restart;'
+ else
+ param=dnsmasq_restart
+ fi
+ fi
+ eval "$param"
+ return $?
+ ;;
+ quiet)
+ case "$targetDNS" in
+ dnsmasq.addnhosts | dnsmasq.conf | dnsmasq.ipset | dnsmasq.servers)
+ param=dnsmasq_restart
+ ;;
+ unbound.adb_list)
+ param=unbound_restart
+ ;;
+ esac
+ eval "$param"
+ return $?
+ ;;
+ esac
+}
+
+tmpfs() {
+ local action="$1" instance="$2" value="$3"
+ local status message error stats
+ local readReload readRestart curReload curRestart ret
+ if [ -s "$jsonFile" ]; then
+ status="$(jsonfilter -i $jsonFile -l1 -e "@['data']['status']")"
+ message="$(jsonfilter -i $jsonFile -l1 -e "@['data']['message']")"
+ error="$(jsonfilter -i $jsonFile -l1 -e "@['data']['error']")"
+ stats="$(jsonfilter -i $jsonFile -l1 -e "@['data']['stats']")"
+ readReload="$(jsonfilter -i $jsonFile -l1 -e "@['data']['reload']")"
+ readRestart="$(jsonfilter -i $jsonFile -l1 -e "@['data']['restart']")"
+ fi
+ case "$action" in
+ get)
+ case "$instance" in
+ status)
+ echo "$status"; return;;
+ message)
+ echo "$message"; return;;
+ error)
+ echo "$error"; return;;
+ stats)
+ echo "$stats"; return;;
+ triggers)
+ curReload="$parallelDL $debug $dlTimeout $whitelist_domains $blacklist_domains $whitelist_domains_urls $blacklist_domains_urls $blacklist_hosts_urls $targetDNS"
+ curRestart="$compressedCache $forceDNS $led"
+ if [ "$curReload" != "$readReload" ]; then
+ ret='download'
+ elif [ "$curRestart" != "$readRestart" ]; then
+ ret='restart'
+ fi
+ echo "$ret"
+ return;;
+ esac
+ ;;
+ add)
+ case "$instance" in
+ status)
+ [ -n "$status" ] && status="$status $value" || status="$value";;
+ message)
+ [ -n "$message" ] && message="${message} ${value}" || message="$value";;
+ error)
+ [ -n "$error" ] && error="$error $value" || error="$value";;
+ stats)
+ [ -n "$stats" ] && stats="$stats $value" || stats="$value";;
+ esac
+ ;;
+ del)
+ case "$instance" in
+ all)
+ unset status;
+ unset message;
+ unset error;
+ unset stats;
+ ;;
+ status)
+ unset status;;
+ message)
+ unset message;;
+ error)
+ unset error;;
+ stats)
+ unset stats;;
+ triggers)
+ unset readReload; unset readRestart;;
+ esac
+ ;;
+ set)
+ case "$instance" in
+ status)
+ status="$value";;
+ message)
+ message="$value";;
+ error)
+ error="$value";;
+ stats)
+ stats="$value";;
+ triggers)
+ readReload="$parallelDL $debug $dlTimeout $whitelist_domains $blacklist_domains $whitelist_domains_urls $blacklist_domains_urls $blacklist_hosts_urls $targetDNS"
+ readRestart="$compressedCache $forceDNS $led"
+ ;;
+ esac
+ ;;
+ esac
+ json_init
+ json_add_object 'data'
+ json_add_string version "$PKG_VERSION"
+ json_add_string status "$status"
+ json_add_string message "$message"
+ json_add_string error "$error"
+ json_add_string stats "$stats"
+ json_add_string reload "$readReload"
+ json_add_string restart "$readRestart"
+ json_close_object
+ json_dump > "$jsonFile"
+ sync
+}
+
+cacheOps() {
+ local R_TMP
+ case "$1" in
+ create|backup)
+ [ -s "$outputFile" ] && { mv -f "$outputFile" "$outputCache"; true > "$outputFile"; } >/dev/null 2>/dev/null
+ return $?
+ ;;
+ restore|use)
+ [ -s "$outputCache" ] && mv "$outputCache" "$outputFile" >/dev/null 2>/dev/null
+ return $?
+ ;;
+ test)
+ [ -s "$outputCache" ]
+ return $?
+ ;;
+ testGzip)
+ [ -s "$outputGzip" ] && gzip -t -c "$outputGzip"
+ return $?
+ ;;
+ createGzip)
+ R_TMP="$(mktemp -u -q -t ${packageName}_tmp.XXXXXXXX)"
+ if gzip < "$outputFile" > "$R_TMP"; then
+ if mv "$R_TMP" "$outputGzip"; then
+ rm -f "$R_TMP"
+ return 0
+ else
+ rm -f "$R_TMP"
+ return 1
+ fi
+ else
+ return 1
+ fi
+ ;;
+ expand|unpack|expandGzip|unpackGzip)
+ [ -s "$outputGzip" ] && gzip -dc < "$outputGzip" > "$outputCache"
+ return $?
+ ;;
+ esac
+}
+
+fw3Ops() {
+ local action="$1" param="$2" _restart
+ case "$action" in
+ reload) /etc/init.d/firewall reload >/dev/null 2>&1;;
+ restart) /etc/init.d/firewall restart >/dev/null 2>&1;;
+ remove)
+ case "$param" in
+ dns_redirect) uci -q del firewall.simple_adblock_dns_redirect;;
+ ipset) uci -q del firewall.simple_adblock_ipset
+ uci -q del firewall.simple_adblock_ipset_rule;;
+ *)
+ uci -q del firewall.simple_adblock_dns_redirect
+ uci -q del firewall.simple_adblock_ipset
+ uci -q del firewall.simple_adblock_ipset_rule
+ ;;
+ esac
+ ;;
+ insert)
+ case "$param" in
+ dns_redirect)
+ if ! uci -q get firewall.simple_adblock_dns_redirect >/dev/null; then
+ uci -q set firewall.simple_adblock_dns_redirect=redirect
+ uci -q set firewall.simple_adblock_dns_redirect.name=simple_adblock_dns_hijack
+ uci -q set firewall.simple_adblock_dns_redirect.target=DNAT
+ uci -q set firewall.simple_adblock_dns_redirect.src=lan
+ uci -q set firewall.simple_adblock_dns_redirect.proto=tcpudp
+ uci -q set firewall.simple_adblock_dns_redirect.src_dport=53
+ uci -q set firewall.simple_adblock_dns_redirect.dest_port=53
+ fi
+ ;;
+ ipset)
+ if ! uci -q get firewall.simple_adblock_ipset >/dev/null; then
+ uci -q set firewall.simple_adblock_ipset=ipset
+ uci -q set firewall.simple_adblock_ipset.name=adb
+ uci -q set firewall.simple_adblock_ipset.match=dest_net
+ uci -q set firewall.simple_adblock_ipset.storage=hash
+ uci -q set firewall.simple_adblock_ipset.enabled=1
+ _restart=1
+ fi
+ if ! uci -q get firewall.simple_adblock_ipset_rule >/dev/null; then
+ uci -q set firewall.simple_adblock_ipset_rule=rule
+ uci -q set firewall.simple_adblock_ipset_rule.name=simple_adblock_ipset_rule
+ uci -q set firewall.simple_adblock_ipset_rule.ipset=adb
+ uci -q set firewall.simple_adblock_ipset_rule.src=lan
+ uci -q set firewall.simple_adblock_ipset_rule.dest='*'
+ uci -q set firewall.simple_adblock_ipset_rule.proto=tcpudp
+ uci -q set firewall.simple_adblock_ipset_rule.target=REJECT
+ uci -q set firewall.simple_adblock_ipset_rule.enabled=1
+ fi
+ ;;
+ *)
+ if ! uci -q get firewall.simple_adblock_dns_redirect >/dev/null; then
+ uci -q set firewall.simple_adblock_dns_redirect=redirect
+ uci -q set firewall.simple_adblock_dns_redirect.name=simple_adblock_dns_hijack
+ uci -q set firewall.simple_adblock_dns_redirect.target=DNAT
+ uci -q set firewall.simple_adblock_dns_redirect.src=lan
+ uci -q set firewall.simple_adblock_dns_redirect.proto=tcpudp
+ uci -q set firewall.simple_adblock_dns_redirect.src_dport=53
+ uci -q set firewall.simple_adblock_dns_redirect.dest_port=53
+ fi
+ if ! uci -q get firewall.simple_adblock_ipset >/dev/null; then
+ uci -q set firewall.simple_adblock_ipset=ipset
+ uci -q set firewall.simple_adblock_ipset.name=adb
+ uci -q set firewall.simple_adblock_ipset.match=dest_net
+ uci -q set firewall.simple_adblock_ipset.storage=hash
+ uci -q set firewall.simple_adblock_ipset.enabled=1
+ _restart=1
+ fi
+ if ! uci -q get firewall.simple_adblock_ipset_rule >/dev/null; then
+ uci -q set firewall.simple_adblock_ipset_rule=rule
+ uci -q set firewall.simple_adblock_ipset_rule.name=simple_adblock_ipset_rule
+ uci -q set firewall.simple_adblock_ipset_rule.ipset=adb
+ uci -q set firewall.simple_adblock_ipset_rule.src=lan
+ uci -q set firewall.simple_adblock_ipset_rule.dest='*'
+ uci -q set firewall.simple_adblock_ipset_rule.proto=tcpudp
+ uci -q set firewall.simple_adblock_ipset_rule.target=REJECT
+ uci -q set firewall.simple_adblock_ipset_rule.enabled=1
+ fi
+ ;;
+ esac
+ esac
+ if [ -n "$(uci changes firewall)" ]; then
+ uci -q commit firewall
+ if [ -z "$_restart" ]; then
+ fw3Ops 'reload'
+ else
+ fw3Ops 'restart'
+ fi
+ fi
+}
+
+process_url() {
+ local label type D_TMP R_TMP
+ if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ]; then return 1; fi
+ label="${1##*//}"; label="${label%%/*}";
+ if [ "$2" = 'hosts' ]; then
+ label="Hosts: $label"; filter="$hostsFilter";
+ else
+ label="Domains: $label"; filter="$domainsFilter";
+ fi
+ if [ "$3" = 'blocked' ]; then
+ type='Blocked'; D_TMP="$B_TMP";
+ else
+ type='Allowed'; D_TMP="$A_TMP";
+ fi
+ while [ -z "$R_TMP" ] || [ -e "$R_TMP" ]; do
+ R_TMP="$(mktemp -u -q -t ${packageName}_tmp.XXXXXXXX)"
+ done
+ if ! $dl_command "$1" $dl_flag "$R_TMP" 2>/dev/null || [ ! -s "$R_TMP" ]; then
+ output 1 "$_FAIL_"
+ output 2 "[DL] $type $label $__FAIL__\\n"
+ echo "Error: downloading '${1}'." >> "$sharedMemoryError"
+ else
+ sed -i "$filter" "$R_TMP"
+ if [ ! -s "$R_TMP" ]; then
+ output 1 "$_FAIL_"
+ output 2 "[DL] $type $label $__FAIL__\\n"
+ echo "Error: parsing '${1}'." >> "$sharedMemoryError"
+ else
+ cat "${R_TMP}" >> "$D_TMP"
+ output 1 "$_OK_"
+ output 2 "[DL] $type $label $__OK__\\n"
+ fi
+ fi
+ rm -f "$R_TMP"
+ return 0
+}
+
+download_lists() {
+ local hf w_filter j=0 R_TMP
+
+ tmpfs set message "${statusDownloading}..."
+ rm -f "$A_TMP" "$B_TMP" "$outputFile" "$outputCache" "$outputGzip"
+ if [ "$(awk '/^MemFree/ {print int($2/1000)}' "/proc/meminfo")" -lt 32 ]; then
+ output 3 'Low free memory, restarting resolver... '
+ if dnsOps 'quiet'; then
+ output_okn
+ else
+ output_fail
+ fi
+ fi
+ touch $A_TMP; touch $B_TMP;
+ output 1 'Downloading lists '
+ rm -f "$sharedMemoryError"
+ if [ -n "$blacklist_hosts_urls" ]; then
+ for hf in ${blacklist_hosts_urls}; do
+ if [ "$parallelDL" -gt 0 ]; then
+ process_url "$hf" 'hosts' 'blocked' &
+ else
+ process_url "$hf" 'hosts' 'blocked'
+ fi
+ done
+ fi
+ if [ -n "$blacklist_domains_urls" ]; then
+ for hf in ${blacklist_domains_urls}; do
+ if [ "$parallelDL" -gt 0 ]; then
+ process_url "$hf" 'domains' 'blocked' &
+ else
+ process_url "$hf" 'domains' 'blocked'
+ fi
+ done
+ fi
+ if [ -n "$whitelist_domains_urls" ]; then
+ for hf in ${whitelist_domains_urls}; do
+ if [ "$parallelDL" -gt 0 ]; then
+ process_url "$hf" 'domains' 'allowed' &
+ else
+ process_url "$hf" 'domains' 'allowed'
+ fi
+ done
+ fi
+ wait
+ [ -s "$sharedMemoryError" ] && tmpfs add error "$(cat "$sharedMemoryError")"
+ rm -f "$sharedMemoryError"
+ output 1 '\n'
+
+ [ -n "$blacklist_domains" ] && for hf in ${blacklist_domains}; do echo "$hf" | sed "$domainsFilter" >> $B_TMP; done
+ whitelist_domains="${whitelist_domains}
+$(cat $A_TMP)"
+ [ -n "$whitelist_domains" ] && for hf in ${whitelist_domains}; do hf="$(echo "$hf" | sed 's/\./\\./g')"; w_filter="$w_filter/^${hf}$/d;/\\.${hf}$/d;"; done
+
+ [ ! -s "$B_TMP" ] && return 1
+
+ output 1 'Processing downloads '
+ output 2 'Sorting combined list '
+ tmpfs set message "$statusProcessing: sorting combined list"
+ if [ "$allowIDN" -gt 0 ]; then
+ if sort -u "$B_TMP" > "$A_TMP"; then
+ output_ok
+ else
+ output_failn
+ tmpfs add error 'Error: Sorting error.'
+ fi
+ else
+ if sort -u "$B_TMP" | grep -E -v '[^a-zA-Z0-9=/.-]' > "$A_TMP"; then
+ output_ok
+ else
+ output_failn
+ tmpfs add error 'Error: Sorting error.'
+ fi
+ fi
+
+ if [ "$targetDNS" = 'dnsmasq.conf' ] || \
+ [ "$targetDNS" = 'dnsmasq.ipset' ] || \
+ [ "$targetDNS" = 'dnsmasq.servers' ] || \
+ [ "$targetDNS" = 'unbound.adb_list' ]; then
+ # TLD optimization written by Dirk Brenken (dev@brenken.org)
+ output 2 'Optimizing combined list '
+ tmpfs set message "$statusProcessing: optimizing combined list"
+# sed -E 'G;:t;s/(.*)(\.)(.*)(\n)(.*)/\1\4\5\2\3/;tt;s/(.*)\n(\.)(.*)/\3\2\1/' is actually slower than awk
+ if awk -F "." '{for(f=NF;f>1;f--)printf "%s.",$f;print $1}' "$A_TMP" > "$B_TMP"; then
+ if sort "$B_TMP" > "$A_TMP"; then
+ if awk '{if(NR=1){tld=$NF};while(getline){if($NF!~tld"\\."){print tld;tld=$NF}}print tld}' "$A_TMP" > "$B_TMP"; then
+ if awk -F "." '{for(f=NF;f>1;f--)printf "%s.",$f;print $1}' "$B_TMP" > "$A_TMP"; then
+ if sort -u "$A_TMP" > "$B_TMP"; then
+ output_ok
+ else
+ output_failn
+ tmpfs add error 'Error: Data file optimization.'
+ mv "$A_TMP" "$B_TMP"
+ fi
+ else
+ output_failn
+ tmpfs add error 'Error: Data file optimization.'
+ fi
+ else
+ output_failn
+ tmpfs add error 'Error: Data file optimization.'
+ mv "$A_TMP" "$B_TMP"
+ fi
+ else
+ output_failn
+ tmpfs add error 'Error: Data file optimization.'
+ fi
+ else
+ output_failn
+ tmpfs add error 'Error: Data file optimization.'
+ mv "$A_TMP" "$B_TMP"
+ fi
+ else
+ mv "$A_TMP" "$B_TMP"
+ fi
+
+ output 2 'Whitelisting domains '
+ tmpfs set message "$statusProcessing: whitelisting domains"
+ if sed -i "$w_filter" "$B_TMP"; then
+ output_ok
+ else
+ output_failn
+ tmpfs add error 'Error: Whitelist processing.'
+ fi
+
+ output 2 'Formatting merged file '
+ tmpfs set message "$statusProcessing: formatting merged file"
+ if [ -z "$outputFilterIPv6" ]; then
+ if sed "$outputFilter" "$B_TMP" > "$A_TMP"; then
+ output_ok
+ else
+ output_failn
+ tmpfs add error 'Error: Data file formatting.'
+ fi
+ else
+ case "$targetDNS" in
+ dnsmasq.addnhosts)
+ if sed "$outputFilter" "$B_TMP" > "$A_TMP" && \
+ sed "$outputFilterIPv6" "$B_TMP" >> "$A_TMP"; then
+ output_ok
+ else
+ output_failn
+ tmpfs add error 'Error: Data file formatting.'
+ fi
+ ;;
+ esac
+ fi
+
+ case "$targetDNS" in
+ dnsmasq.addnhosts)
+ output 2 'Creating DNSMASQ addnhosts file '
+ tmpfs set message "$statusProcessing: creating DNSMASQ addnhosts file"
+ ;;
+ dnsmasq.conf)
+ output 2 'Creating DNSMASQ config file '
+ tmpfs set message "$statusProcessing: creating DNSMASQ config file"
+ ;;
+ dnsmasq.ipset)
+ output 2 'Creating DNSMASQ ipset file '
+ tmpfs set message "$statusProcessing: creating DNSMASQ ipset file"
+ ;;
+ dnsmasq.servers)
+ output 2 'Creating DNSMASQ servers file '
+ tmpfs set message "$statusProcessing: creating DNSMASQ servers file"
+ ;;
+ unbound.adb_list)
+ output 2 'Creating Unbound adb_list file '
+ tmpfs set message "$statusProcessing: creating Unbound adb_list file"
+ ;;
+ esac
+ if mv "$A_TMP" "$outputFile"; then
+ output_ok
+ else
+ output_failn
+ tmpfs add error "Error: moving data file '${A_TMP}' to '${outputFile}'."
+ fi
+ if [ "$compressedCache" -gt 0 ]; then
+ output 2 'Creating compressed cache '
+ tmpfs set message "$statusProcessing: creating compressed cache"
+ if cacheOps 'createGzip'; then
+ output_ok
+ else
+ output_failn
+ tmpfs add error 'Error: creating compressed cache.'
+ fi
+ else
+ rm -f "$outputGzip"
+ fi
+ output 2 'Removing temporary files '
+ tmpfs set message "$statusProcessing: removing temporary files"
+ rm -f "/tmp/${packageName}_tmp.*" "$A_TMP" "$B_TMP" "$outputCache" || j=1
+ if [ $j -eq 0 ]; then
+ output_ok
+ else
+ output_failn
+ tmpfs add error 'Error: removing temporary files.'
+ fi
+ output 1 '\n'
+}
+
+boot() {
+ load_package_config
+ if create_lock; then
+ sleep "$bootDelay"
+ remove_lock
+ rc_procd start_service && rc_procd service_triggers
+ fi
+}
+
+start_service() {
+ is_enabled 'on_start' || return 1
+ local action status error message stats
+ if ! create_lock; then
+ output 3 "$serviceName: another instance is starting up "; output_fail
+ return 0
+ fi
+
+ status="$(tmpfs get status)"
+ error="$(tmpfs get error)"
+ message="$(tmpfs get message)"
+ stats="$(tmpfs get stats)"
+ action="$(tmpfs get triggers)"
+
+ if [ "$action" = 'download' ] || [ "$1" = 'download' ] || [ -n "$error" ]; then
+ action='download'
+ elif [ ! -s "$outputFile" ] && ! cacheOps 'test' && ! cacheOps 'testGzip'; then
+ action='download'
+ elif [ ! -s "$outputFile" ] && cacheOps 'testGzip' || cacheOps 'test'; then
+ action='restore'
+ elif [ "$action" = 'restart' ] || [ "$1" = 'restart' ]; then
+ action='restart'
+ elif [ -s "$outputFile" ] && [ -n "$status" ] && [ -z "$error" ]; then
+ if [ "$1" != 'hotplug' ]; then status; fi
+ exit 0
+ else
+ action='download'
+ fi
+
+ tmpfs del all
+ tmpfs set triggers
+
+ if is_chaos_calmer || ! is_ipset_procd; then
+ if [ "$forceDNS" -ne 0 ]; then
+ fw3Ops 'insert' 'dns_redirect'
+ else
+ fw3Ops 'remove' 'dns_redirect'
+ fi
+ if [ "$targetDNS" = 'dnsmasq.ipset' ]; then
+ fw3Ops 'insert' 'ipset'
+ else
+ fw3Ops 'remove' 'ipset'
+ fi
+ procd_open_instance 'main'
+ procd_set_param command /bin/true
+ procd_set_param stdout 1
+ procd_set_param stderr 1
+ procd_close_instance
+ else
+ procd_open_instance 'main'
+ procd_set_param command /bin/true
+ procd_set_param stdout 1
+ procd_set_param stderr 1
+ procd_open_data
+ json_add_array firewall
+ if [ "$forceDNS" -ne 0 ]; then
+ json_add_object ''
+ json_add_string type redirect
+ json_add_string name simple_adblock_dns_redirect
+ json_add_string target DNAT
+ json_add_string src lan
+ json_add_string proto tcpudp
+ json_add_string src_dport 53
+ json_add_string dest_port 53
+ json_add_string reflection 0
+ json_close_object
+ fi
+ if [ "$targetDNS" = 'dnsmasq.ipset' ]; then
+ json_add_object ''
+ json_add_string type ipset
+ json_add_string name adb
+ json_add_string match dest_net
+ json_add_string storage hash
+ json_add_string enabled 1
+ json_close_object
+ json_add_object ''
+ json_add_string type rule
+ json_add_string name simple_adblock_ipset_rule
+ json_add_string ipset adb
+ json_add_string src lan
+ json_add_string dest '*'
+ json_add_string proto tcpudp
+ json_add_string target REJECT
+ json_add_string enabled 1
+ json_close_object
+ fi
+ json_close_array
+ procd_close_data
+ procd_close_instance
+ fi
+
+ if [ "$action" = 'restore' ]; then
+ output 0 "Starting $serviceName... "
+ output 3 "Starting $serviceName...\\n"
+ tmpfs set status "$statusStarting"
+ if cacheOps 'testGzip' && ! cacheOps 'test' && [ ! -s "$outputFile" ]; then
+ output 3 'Found compressed cache file, unpacking it '
+ tmpfs set message 'found compressed cache file, unpacking it.'
+ if cacheOps 'unpackGzip'; then
+ output_okn
+ else
+ output_fail
+ output "$_ERROR_: $serviceName failed to unpack compressed cache!\\n"
+ action='download'
+ fi
+ fi
+ if cacheOps 'test' && [ ! -s "$outputFile" ]; then
+ output 3 'Found cache file, reusing it '
+ tmpfs set message 'found cache file, reusing it.'
+ if cacheOps 'restore'; then
+ output_okn
+ dnsOps 'on_start'
+ else
+ output_fail
+ output "$_ERROR_: $serviceName failed to move '$outputCache' to '$outputFile'!\\n"
+ action='download'
+ fi
+ fi
+ fi
+ case "$action" in
+ download)
+ if [ -s "$outputFile" ] || cacheOps 'test' || cacheOps 'testGzip'; then
+ output 0 "Force-reloading $serviceName... "
+ output 3 "Force-reloading $serviceName...\\n"
+ tmpfs set status "$statusForceReloading"
+ else
+ output 0 "Starting $serviceName... "
+ output 3 "Starting $serviceName...\\n"
+ tmpfs set status "$statusStarting"
+ fi
+ download_lists
+ dnsOps 'on_start'
+ ;;
+ restart)
+ output 0 "Restarting $serviceName... "
+ output 3 "Restarting $serviceName...\\n"
+ tmpfs set status "$statusRestarting"
+ dnsOps 'on_start'
+ ;;
+ start)
+ output 0 "Starting $serviceName... "
+ output 3 "Starting $serviceName...\\n"
+ tmpfs set status "$statusStarting"
+ dnsOps 'on_start'
+ ;;
+ esac
+ if [ -s "$outputFile" ] && [ "$(tmpfs get status)" != "$statusFail" ]; then
+ output 0 "$__OK__\\n";
+ c="$(wc -l < "$outputFile")"
+ output 3 "$serviceName is blocking $c domains (with ${targetDNS}) "; output_okn
+ tmpfs del message
+ tmpfs set status "$statusSuccess: $c domains blocked (with ${targetDNS})."
+ error="$(tmpfs get error)"
+ if [ -n "$error" ]; then
+ output "$(str_replace "$error" "Error:" "$_ERROR_:")\\n"
+ fi
+ else
+ output 0 "$__FAIL__\\n";
+ tmpfs set status "$statusFail"
+ tmpfs add error 'Error: Failed to create blocklist or restart DNS resolver.'
+ fi
+ remove_lock
+}
+
+service_started() { is_ipset_procd && procd_set_config_changed firewall; }
+service_stopped() { is_ipset_procd && procd_set_config_changed firewall; }
+restart_service() { rc_procd start_service 'restart'; }
+reload_service() { restart_service; }
+restart() { restart_service; }
+reload() { restart_service; }
+dl() { rc_procd start_service 'download'; }
+
+killcache() {
+ rm -f "$addnhostsCache" "$addnhostsGzip"
+ rm -f "$dnsmasqCache" "$dnsmasqGzip"
+ rm -f "$ipsetCache" "$ipsetGzip"
+ rm -f "$serversCache" "$serversGzip"
+ rm -f "$unboundCache" "$unboundGzip"
+ return 0
+}
+
+status_service() {
+ local status="$(tmpfs get status)" error="$(tmpfs get error)" message="$(tmpfs get message)"
+ if [ -n "$status" ] && [ -n "$message" ]; then
+ status="${status}: $message"
+ fi
+ [ -n "$status" ] && output "$serviceName $status\\n"
+ [ -n "$error" ] && output "$error\\n"
+}
+
+stop_service() {
+ load_package_config
+ fw3Ops 'remove' 'all'
+ if [ -s "$outputFile" ]; then
+ output "Stopping $serviceName... "
+ cacheOps 'create'
+ if dnsOps 'on_stop'; then
+ led_off "$led"
+ output 0 "$__OK__\\n"; output_okn
+ tmpfs set status "$statusStopped"
+ tmpfs del message
+ else
+ output 0 "$__FAIL__\\n"; output_fail
+ tmpfs set status "$statusFail"
+ tmpfs add error "Error: error stopping $serviceName."
+ output "$_ERROR_: error stopping $serviceName!\\n"
+ fi
+ fi
+}
+
+check() {
+ load_package_config
+ local string="$1"
+ local c="$(grep -c "$string" "$outputFile")"
+ if [ ! -s "$outputFile" ]; then
+ echo "No blacklist ('$outputFile') found."
+ elif [ -z "$string" ]; then
+ echo "Usage: /etc/init.d/${packageName} check string"
+ elif [ "$c" -gt 0 ]; then
+ if [ "$c" -gt 1 ]; then
+ echo "Found $c matches for '$string' in '$outputFile':"
+ else
+ echo "Found 1 match for '$string' in '$outputFile':"
+ fi
+ case "$targetDNS" in
+ dnsmasq.addnhosts)
+ grep "$string" "$outputFile" | sed 's|^127.0.0.1 ||;s|^:: ||;';;
+ dnsmasq.conf)
+ grep "$string" "$outputFile" | sed 's|local=/||;s|/$||;';;
+ dnsmasq.ipset)
+ grep "$string" "$outputFile" | sed 's|ipset=/||;s|/adb$||;';;
+ dnsmasq.servers)
+ grep "$string" "$outputFile" | sed 's|server=/||;s|/$||;';;
+ unbound.adb_list)
+ grep "$string" "$outputFile" | sed 's|^local-zone: "||;s|" static$||;';;
+ esac
+ else
+ echo "The $string is not found in current blacklist ('$outputFile')."
+ fi
+}
+
+sizes() {
+ local i
+ load_package_config
+ echo "# $(date)"
+
+ for i in $blacklist_domains_urls; do
+ [ "${i//melmac}" != "$i" ] && continue
+ if $dl_command "$i" $dl_flag /tmp/sast 2>/dev/null && [ -s /tmp/sast ]; then
+ echo "# File size: $(du -sh /tmp/sast | awk '{print $1}')"
+ if compare_versions "$(du -sk /tmp/sast)" "500"; then
+ echo "# blocklist too big for most routers"
+ elif compare_versions "$(du -sk /tmp/sast)" "100"; then
+ echo "# blocklist may be too big for some routers"
+ fi
+ rm -rf /tmp/sast
+ echo " list blacklist_domains_url '$i'"
+ echo ""
+ else
+ echo "# site was down on last check"
+ echo "# list blacklist_domains_url '$i'"
+ echo ""
+ fi
+ done
+
+ for i in $blacklist_hosts_urls; do
+ if $dl_command "$i" $dl_flag /tmp/sast 2>/dev/null && [ -s /tmp/sast ]; then
+ echo "# File size: $(du -sh /tmp/sast | awk '{print $1}')"
+ if compare_versions "$(du -sk /tmp/sast)" "500"; then
+ echo "# blocklist too big for most routers"
+ elif compare_versions "$(du -sk /tmp/sast)" "100"; then
+ echo "# blocklist may be too big for some routers"
+ fi
+ rm -rf /tmp/sast
+ echo " list blacklist_hosts_url '$i'"
+ echo ""
+ else
+ echo "# site was down on last check"
+ echo "# list blacklist_hosts_url '$i'"
+ echo ""
+ fi
+ done
+}
projects / librecmc / package-feed.git / commitdiff