@ -13,9 +13,12 @@ Mainly OpenWRT targeted but also supports traditional Linux, FreeBSD, OpenBSD, p
In the simplest case you are dealing with passive DPI. Passive DPI can read passthrough traffic,
inject its own packets, but cannot drop packets.
If the request is prohibited the passive DPI will inject its own RST packet and optionally http redirect packet.
If fake packets from DPI are only sent to client, you can use iptables commands to drop them if you can write
correct filter rules. This requires manual in-deep traffic analysis and tuning for specific ISP.
This is how we bypass the consequences of a ban trigger.
If the passive DPI sends an RST packet also to the server, there is nothing you can do about it.
@ -26,37 +29,33 @@ To do that send what DPI does not expect and what breaks its algorithm of recogn
Some DPIs cannot recognize the http request if it is divided into TCP segments.
For example, a request of the form `GET / HTTP / 1.1 \ r \ nHost: kinozal.tv ......`
we send in 2 parts: first go "GET", then `/ HTTP / 1.1 \ r \ nHost: kinozal.tv .....`.
Other DPIs stumble when the `Host:` header is written in another case: for example, "host:".
we send in 2 parts: first go `GET`, then `/ HTTP / 1.1 \ r \ nHost: kinozal.tv .....`.
Other DPIs stumble when the `Host:` header is written in another case: for example, `host:`.
Sometimes work adding extra space after the method: `GET /` => `GET /`
or adding a dot at the end of the host name: `Host: kinozal.tv.`
There is also more advanced magic for bypassing DPI at the packet level.
## When it will not work
* If DNS server returns false responses. ISP can return false IP addresses or not return anything
when blocked domains are queried. If this is the case change DNS to public ones, such as 8.8.8.8 or 1.1.1.1.
Sometimes ISP hijacks queries to any DNS server. Dnscrypt or dns-over-tls help.
when blocked domains are queried. If this is the case change DNS to public ones, such as 8.8.8.8 or 1.1.1.1.Sometimes ISP hijacks queries to any DNS server. Dnscrypt or dns-over-tls help.
* If blocking is done by IP.
* If a connection passes through a filter capable of reconstructing a TCP connection, and which
follows all standards. For example, we are routed to squid. Connection goes through the full OS tcpip stack,
fragmentation disappears immediately as a means of circumvention. Squid is correct, it will find everything
as it should, it is useless to deceive him.
BUT. Only small providers can afford using squid, since it is very resource intensive.
Large companies usually use DPI, which is designed for much greater bandwidth.
follows all standards. For example, we are routed to squid. Connection goes through the full OS tcpip stack, fragmentation disappears immediately as a means of circumvention. Squid is correct, it will find everything as it should, it is useless to deceive him. BUT. Only small providers can afford using squid, since it is very resource intensive. Large companies usually use DPI, which is designed for much greater bandwidth.
## Installation
### desktop linux system
Simple install works on most modern linux distributions with systemd or openrc, OpenWRT and MacOS.
Run install_easy.sh and answer its questions.
Run `install_easy.sh` and answer its questions.
### OpenWRT
install_easy.sh works on openwrt but there're additional challenges.
`install_easy.sh` works on openwrt but there're additional challenges.
They are mainly about possibly low flash free space.
Simple install will not work if it has no space to install itself and required packages from the repo.
@ -64,8 +63,8 @@ Another challenge would be to bring zapret to the router. You can download zip f
Do not repack zip contents in Windows, because this way you break chmod and links.
Install openssh-sftp-server and unzip to openwrt and use sftp to transfer the file.
The best way to start is to put zapret dir to /tmp and run /tmp/zapret/install_easy.sh from there.
After installation remove /tmp/zapret to free RAM.
The best way to start is to put zapret dir to `/tmp` and run `/tmp/zapret/install_easy.sh` from there.
After installation remove `/tmp/zapret` to free RAM.
The absolute minimum for openwrt is 64/8 system, 64/16 is comfortable, 128/extroot is recommended.
@ -81,33 +80,41 @@ There's no ipset support unless you run custom kernel. In common case task of br
on android is ranging from "not easy" to "almost impossible", unless you find working kernel
image for your device.
Android does not use /etc/passwd, tpws --user won't work. There's replacement.
Use numeric uids in --uid option.
Android does not use /etc/passwd, `tpws --user` won't work. There's replacement.
Use numeric uids in `--uid` option.
Its recommended to use gid 3003 (AID_INET), otherwise tpws will not have inet access.
Example : --uid 1:3003
In iptables use : "! --uid-owner 1" instead of "! --uid-owner tpws".
Example : `--uid 1:3003`
In iptables use : `! --uid-owner 1` instead of `! --uid-owner tpws`.
Write your own shell script with iptables and tpws, run it using your root manager.
Autorun scripts are here :
magisk : /data/adb/service.d
supersu : /system/su.d
magisk : `/data/adb/service.d`
supersu : `/system/su.d`
I haven't checked whether android can kill iptable rules at its own will during wifi connection/disconnection,
mobile data on/off, ...
How to run tpws on root-less android.
You can't write to /system, /data, can't run from sd card.
Selinux prevents running executables in /data/local/tmp from apps.
You can't write to `/system`, `/data`, can't run from sd card.
Selinux prevents running executables in `/data/local/tmp` from apps.
nfqws is equipped with minimalistic connection tracking system (conntrack)
It's enabled if some specific DPI circumvention methods are involved.
Currently these are `--wssize` and `--dpi-desync-cutoff` options.
Conntrack can track connection phase : SYN,ESTABLISHED,FIN , packet counts in both directions , sequence numbers.
It can be fed with unidirectional or bidirectional packets.
A SYN or SYN,ACK packet creates an entry in the conntrack table.
That's why iptables redirection must start with the first packet although can be cut later using connbytes filter.
A connection is deleted from the table as soon as it's no more required to satisfy nfqws needs or when a timeout happens.
There're 3 timeouts for each connection state. They can be changed in `--ctrack-timeouts` parameter.
`--wssize` changes tcp window size for the server to force it to send split replies.
In order for this to affect all server operating systems, it is necessary to change the window size in each outgoing packet
before sending the message, the answer to which must be split (for example, TLS ClientHello).
That's why conntrack is required to know when to stop applying low window size.
If you do not stop and set the low wssize all the time, the speed will drop catastrophically.
Linux can overcome this using connbytes filter but other OS may not include similar filter.
In http(s) case wssize stops after the first http request or TLS ClientHello.
If you deal with a non-http(s) protocol you need --wssize-cutoff. It sets the number of the outgoing packet where wssize stops.
If you deal with a non-http(s) protocol you need `--wssize-cutoff`. It sets the number of the outgoing packet where wssize stops.
(numbering starts from 1).
If a http request or TLS ClientHello packet is detected wssize stops immediately ignoring wssize-cutoff option.
If your protocol is prone to long inactivity, you should increase ESTABLISHED phase timeout using `--ctrack-timeouts`.
Default timeout is low - only 5 mins.
Don't forget that nfqws feeds with redirected packets. If you have limited redirection with connbytes
ESTABLISHED entries can remain in the table until dropped by timeout.
To diagnose conntrack state send SIGUSR1 signal to nfqws : killall -SIGUSR1 nfqws.
To diagnose conntrack state send SIGUSR1 signal to nfqws : `killall -SIGUSR1 nfqws`.
nfqws will dump current conntrack table to stdout.
Typically, in a SYN packet, client sends TCP extension "scaling factor" in addition to window size.
Typically, in a SYN packet, client sends TCP extension **scaling factor** in addition to window size.
scaling factor is the power of two by which the window size is multiplied : 0=>1, 1=>2, 2=>4, ..., 8=>256, ...
The wssize parameter specifies the scaling factor after a colon.
Scaling factor can only decrease, increase is blocked to prevent the server from exceeding client's window size.
To force a TLS server to fragment ServerHello message to avoid hostname detection on DPI use --wssize=1:6
To force a TLS server to fragment ServerHello message to avoid hostname detection on DPI use `--wssize=1:6`
The main rule is to set scale_factor as much as possible so that after recovery the final window size
becomes the possible maximum. If you set scale_factor 64:0, it will be very slow.
becomes the possible maximum. If you set `scale_factor` 64:0, it will be very slow.
On the other hand, the server response must not be large enough for the DPI to find what it is looking for.
Hostlist filter does not affect --wssize because it works since the connection initiation when it's not yet possible
Hostlist filter does not affect `--wssize` because it works since the connection initiation when it's not yet possible
to extract the host name.
`--wssize` may slow down sites and/or increase response time. It's desired to use another methods if possible.
`--dpi-desync-cutoff` allows you to set the limit on the number of the outgoing packet, at which it stops
applying dpi-desync. Useful with --dpi-desync-any-protocol=1.
applying dpi-desync. Useful with `--dpi-desync-any-protocol=1`.
If the connection falls out of the conntrack and --dpi-desync-cutoff is set, dpi desync will not be applied.
Set conntrack timeouts appropriately.
## tpws
@ -503,14 +540,19 @@ tpws is transparent proxy.
The manipulation parameters can be combined in any way.
split-http-req takes precedence over split-pos for http reqs.
split-pos works by default only on http and TLS ClientHello. use --split-any-protocol to act on any packet
`split-http-req` takes precedence over split-pos for http reqs.
split-pos works by default only on http and TLS ClientHello. use `--split-any-protocol` to act on any packet
tpws can bind to multiple interfaces and IP addresses (up to 32).
Port number is always the same.
Parameters --bind-iface* and --bind-addr create new bind.
Other parameters --bind-* are related to the last bind.
link local ipv6 (fe80::/8) mode selection :
Parameters `--bind-iface*` and `--bind-addr` create new bind.
Other parameters `--bind-*` are related to the last bind.
link local ipv6 (`fe80::/8`) mode selection :
```
--bind-iface6 --bind-linklocal=no : first selects private address fd00::/8, then global address
@ -519,70 +561,92 @@ link local ipv6 (fe80::/8) mode selection :
--bind-iface6 --bind-linklocal=force : select only LL
```
To bind to all ipv4 specify --bind-addr "0.0.0.0", all ipv6 - "::". --bind-addr="" - mean bind to all ipv4 and ipv6.
To bind to all ipv4 specify `--bind-addr "0.0.0.0"`, all ipv6 - `::`.
`--bind-addr=""` - mean bind to all ipv4 and ipv6.
If no binds are specified default bind to all ipv4 and ipv6 addresses is created.
To bind to a specific link local address do : --bind-iface6=fe80::aaaa:bbbb:cccc:dddd%iface-name
The --bind-wait* parameters can help in situations where you need to get IP from the interface, but it is not there yet, it is not raised
To bind to a specific link local address do : `--bind-iface6=fe80::aaaa:bbbb:cccc:dddd%iface-name`
The `--bind-wait*` parameters can help in situations where you need to get IP from the interface, but it is not there yet, it is not raised
or not configured.
In different systems, ifup events are caught in different ways and do not guarantee that the interface has already received an IP address of a certain type.
In the general case, there is no single mechanism to hang oneself on an event of the type "link local address appeared on the X interface."
To bind to a specific ip when its interface may not be configured yet do : --bind-addr=192.168.5.3 --bind-wait-ip=20
To bind to a specific ip when its interface may not be configured yet do : `--bind-addr=192.168.5.3 --bind-wait-ip=20`
It's possible to bind to any nonexistent address in transparent mode but in socks mode address must exist.
in socks proxy mode no additional system privileges are required
connection to local IPs of the system where tpws runs are prohibited
tpws supports remote dns resolving (curl : --socks5-hostname firefox : socks_remote_dns=true) , but does it in blocking mode.
tpws supports remote dns resolving (curl : `--socks5-hostname` firefox : `socks_remote_dns=true`) , but does it in blocking mode.
tpws uses async sockets for all activity but resolving can break this model.
if tpws serves many clients it can cause trouble. also DoS attack is possible against tpws.
if remote resolving causes trouble configure clients to use local name resolution and use
--no-resolve option on tpws side.
## Ways to get a list of blocked IP
1) Enter the blocked domains to ipset/zapret-hosts-user.txt and run ipset/get_user.sh
1. Enter the blocked domains to ipset/zapret-hosts-user.txt and run ipset/get_user.sh
At the output, you get ipset/zapret-ip-user.txt with IP addresses.
2) ipset/get_reestr_*.sh. Russian specific
2. `ipset/get_reestr_*.sh`. Russian specific
3, `ipset/get_antifilter_*.sh`. Russian specific
3) ipset/get_antifilter_*.sh. Russian specific
4, `ipset/get_config.sh`. This script calls what is written into the GETLIST variable from the config file.
4) ipset/get_config.sh. This script calls what is written into the GETLIST variable from the config file.
If the variable is not defined, then only lists for ipsets nozapret/nozapret6 are resolved.
So, if you're not russian, the only way for you is to manually add blocked domains.
Or write your own ipset/get_iran_blocklist.sh , if you know where to download this one.
Or write your own `ipset/get_iran_blocklist.sh` , if you know where to download this one.
On routers, it is not recommended to call these scripts more than once in 2 days to minimize flash memory writes.
If one of NFQWS_OPT_DESYNC_HTTP/NFQWS_OPT_DESYNC_HTTPS is not defined it takes value of NFQWS_OPT_DESYNC.
If one of NFQWS_OPT_DESYNC_HTTP6/NFQWS_OPT_DESYNC_HTTPS6 is not defined it takes value from
NFQWS_OPT_DESYNC_HTTP/NFQWS_OPT_DESYNC_HTTPS.
It means if only NFQWS_OPT_DESYNC is defined all four take its value.
If one of `NFQWS_OPT_DESYNC_HTTP`/`NFQWS_OPT_DESYNC_HTTPS` is not defined it takes value of NFQWS_OPT_DESYNC.
If one of `NFQWS_OPT_DESYNC_HTTP6`/`NFQWS_OPT_DESYNC_HTTPS6` is not defined it takes value from
`NFQWS_OPT_DESYNC_HTTP`/`NFQWS_OPT_DESYNC_HTTPS`.
It means if only `NFQWS_OPT_DESYNC` is defined all four take its value.
If a variable is not defined, the value NFQWS_OPT_DESYNC is taken.
If a variable is not defined, the value `NFQWS_OPT_DESYNC` is taken.
flow offloading control (OpenWRT only)
```
donttouch : disable system flow offloading setting if selected mode is incompatible with it, dont touch it otherwise and dont configure selective flow offloading
none : always disable system flow offloading setting and dont configure selective flow offloading
software : always disable system flow offloading setting and configure selective software flow offloading
hardware : always disable system flow offloading setting and configure selective hardware flow offloading
```
`FLOWOFFLOAD=donttouch`
The GETLIST parameter tells the install_easy.sh installer which script to call
to update the list of blocked ip or hosts.
Its called via get_config.sh from scheduled tasks (crontab or systemd timer).
Its called via `get_config.sh` from scheduled tasks (crontab or systemd timer).
Put here the name of the script that you will use to update the lists.
If not, then the parameter should be commented out.
@ -738,7 +810,7 @@ IMPORTANT: configuring routing, masquerade, etc. not a zapret task.
Only modes that intercept transit traffic are enabled.
It's possible to specify multiple interfaces like this : `IFACE_LAN="eth0 eth1 eth2"`
The INIT_APPLY_FW=1 parameter enables the init script to independently apply iptables rules.
The `INIT_APPLY_FW=1` parameter enables the init script to independently apply iptables rules.
With other values or if the parameter is commented out, the rules will not be applied.
This is useful if you have a firewall management system, in the settings of which you should tie the rules.