--dpi-desync-ttl=<int> ; set ttl for desync packet
--dpi-desync-ttl6=<int> ; set ipv6 hop limit for desync packet. by default ttl value is used.
@ -232,7 +158,7 @@ nfqws takes the following parameters:
--dpi-desync-split-pos=<1..9216> ; data payload split position
--dpi-desync-split-http-req=method|host ; split at specified logical part of plain http request
--dpi-desync-split-tls=sni|sniext ; split at specified logical part of TLS ClientHello
--dpi-desync-split-seqovl=<int> ; use sequence overlap before first sent original split segment
--dpi-desync-split-seqovl=N|-N|marker+N|marker-N ; use sequence overlap before first sent original split segment
--dpi-desync-split-seqovl-pattern=<filename>|0xHEX ; pattern for the fake part of overlap
--dpi-desync-ipfrag-pos-tcp=<8..9216> ; ip frag position starting from the transport header. multiple of 8, default 8.
--dpi-desync-ipfrag-pos-udp=<8..9216> ; ip frag position starting from the transport header. multiple of 8, default 32.
@ -267,24 +193,21 @@ nfqws takes the following parameters:
--ipset-exclude=<filename> ; ipset exclude filter (one ip/CIDR per line, ipv4 and ipv6 accepted, gzip supported, multiple ipsets allowed)
```
The manipulation parameters can be combined in any way.
### DPI desync attack
WARNING. `--wsize` parameter is now not used anymore in scripts. TCP split can be achieved using DPI desync attack.
The idea is to take original message, modify it, add additional fake information in such a way that the server OS accepts original data only
but DPI cannot recostruct original message or sees what it cannot identify as a prohibited request.
### DPI desync attack
There's a set of instruments to achieve that goal.
It can be fake packets that reach DPI but do not reach server or get rejected by server, TCP segmentation or IP fragmentation.
There're attacks based on TCP sequence numbers. Methods can be combined in many ways.
After completion of the tcp 3-way handshake, the first data packet from the client goes.
It usually has `GET / ...` or TLS ClientHello. We drop this packet, replacing with something else.
It can be a fake version with another harmless but valid http or https request (`fake`), tcp reset packet (`rst`,`rstack`),
split into 2 segments original packet with fake segment in the middle (`split`).
`fakeknown` sends fake only in response to known application protocol.
In articles these attack have names **TCB desynchronization** and **TCB teardown**.
Fake packet must reach DPI, but do not reach the destination server.
The following means are available: set a low TTL, send a packet with bad checksum,
add tcp option **MD5 signature**. All of them have their own disadvantages :
* md5sig does not work on all servers
* badsum doesn't work if your device is behind NAT which does not pass invalid packets.
### Fakes
Fakes are separate generated by nfqws packets carrying false information for DPI. They must either not reach the server or be rejected by it. Otherwise TCP connection or data stream would be broken. There're multiple ways to solve this task.
* **md5sig** does not work on all servers
* **badsum** doesn't work if your device is behind NAT which does not pass invalid packets.
The most common Linux NAT router configuration does not pass them. Most home routers are Linux based.
The default sysctl configuration `net.netfilter.nf_conntrack_checksum=1` causes contrack to verify tcp and udp checksums
and set INVALID state for packets with invalid checksum.
@ -299,25 +222,25 @@ add tcp option **MD5 signature**. All of them have their own disadvantages :
This behavior was observed on a Mediatek MT7621 based device.
Tried to modify mediatek ethernet driver with no luck, likely hardware enforced limitation.
However the device allowed to send badsum packets, problem only existed for passthrough traffic from clients.
* badseq packets will be dropped by server, but DPI also can ignore them.
* **badseq** packets will be dropped by server, but DPI also can ignore them.
default badseq increment is set to -10000 because some DPIs drop packets outside of the small tcp window.
But this also can cause troubles when `--dpi-desync-any-protocol` is enabled.
To be 100% sure fake packet cannot fit to server tcp window consider setting badseq increment to 0x80000000
* TTL looks like the best option, but it requires special tuning for each ISP. If DPI is further than local ISP websites
* **TTL** looks like the best option, but it requires special tuning for each ISP. If DPI is further than local ISP websites
you can cut access to them. Manual IP exclude list is required. Its possible to use md5sig with ttl.
This way you cant hurt anything, but good chances it will help to open local ISP websites.
If automatic solution cannot be found then use `zapret-hosts-user-exclude.txt`.
Some router stock firmwares fix outgoing TTL. Without switching this option off TTL fooling will not work.
* `hopbyhop` is ipv6 only. This fooling adds empty extension header `hop-by-hop options` or two headers in case of `hopbyhop2`.
* **hopbyhop** is ipv6 only. This fooling adds empty extension header `hop-by-hop options` or two headers in case of `hopbyhop2`.
Packets with two hop-by-hop headers violate RFC and discarded by all operating systems.
All OS accept packets with one hop-by-hop header.
Some ISPs/operators drop ipv6 packets with hop-by-hop options. Fakes will not be processed by the server either because
ISP drops them or because there are two same headers.
DPIs may still anaylize packets with one or two hop-by-hop headers.
* `datanoack` sends tcp fakes without ACK flag. Servers do not accept this but DPI may accept.
* **datanoack** sends tcp fakes without ACK flag. Servers do not accept this but DPI may accept.
This mode may break NAT and may not work with iptables if masquerade is used, even from the router itself.
Works with nftables properly. Likely requires external IP address (some ISPs pass these packets through their NAT).
* `autottl` tries to automatically guess TTL value that allows DPI to receive fakes and does not allow them to reach the server.
* **autottl** tries to automatically guess TTL value that allows DPI to receive fakes and does not allow them to reach the server.
This tech relies on well known TTL values used by OS : 64,128,255. nfqws takes first incoming packet (YES, you need to redirect it too),
guesses path length and decreases by `delta` value (default 1). If resulting value is outside the range (min,max - default 3,20)
then its normalized to min or max. If the path shorter than the value then autottl fails and falls back to the fixed value.
@ -327,42 +250,53 @@ add tcp option **MD5 signature**. All of them have their own disadvantages :
For fake,rst,rstack modes original packet is sent after the fake.
Disorder mode splits original packet and sends packets in the following order :
1. 2nd segment
2. fake 1st segment, data filled with zeroes
3. 1st segment
4. fake 1st segment, data filled with zeroes (2nd copy)
### TCP segmentation
* `multisplit`. split request at specified in `--dpi-desync-split-pos` positions
* `multidisorder`. same as `multisplit` but send in reverse order
* `fakedsplit`. split request into 2 segments adding fakes in the middle of them : fake 1st segment, 1st segment, fake 1st segment, 2nd segment
* `fakeddisorder`. same as `fakedsplit` but with another order : 2nd segment, fake 1st segment, 1st segment, fake 1st segment
Positions are defined by markers.
* **Absolute positive marker** - numeric offset inside one packet or group of packets starting from the start
* **Absolute negative marker** - numeric offset inside one packet or group of packets starting from the next byte after the end
* **Relative marker** - positive or negative offset relative to a logical position within a packet or group of packets
Relative positions :
* **method** - HTTP method start ('GET', 'POST', 'HEAD', ...). Method is usually always at position 0 but also supported `--methodeol` fooling by **tpws**. If fooled position can become 1 or 2.
* **host** - hostname start in a known protocol (http, TLS)
* **endhost** - the byte next to the last hostname's byte
* **sld** - second level domain start in the hostname
* **endsld** - the byte next to the last SLD byte
* **midsld** - middle of SLD
* **sniext** - start of the data field in the SNI TLS extension. Any extension has 2-byte type and length fields followed by data field.
Marker list example : `100,midsld,sniext+1,endhost-2,-10`.
Original packet is always dropped. `--dpi-desync-split-pos` sets split position (default 2).
If position is higher than packet length, pos=1 is used.
This sequence is designed to make reconstruction of critical message as difficult as possible.
Fake segments may not be required to bypass some DPIs, but can potentially help if more sophisticated reconstruction
algorithms are used.
Mode `disorder2` disables sending of fake segments.
When splitting all markers are resolved to absolute offsets. If a relative position is absent in the current protocol its dropped. Then all resolved offsets are normalized to the current packet offset in multi packet group (multi-packet TLS with kyber, for example). Positions outside of the current packet are dropped. Remaining positions are sorted and deduplicated.
Split mode is very similar to disorder but without segment reordering :
In `multisplit`or `multidisorder` case split is cancelled if no position remained.
1. fake 1st segment, data filled with zeroes
2. 1st segment
3. fake 1st segment, data filled with zeroes (2nd copy)
4. 2nd segment
`fakedsplit` и `fakeddisorder` use only one split position. It's searched from the `--dpi-desync-split-pos` list by a special alorightm.
First relative markers are searched. If no suitable found absolute markers are searched. If nothing found position 1 is used.
Mode `split2` disables sending of fake segments. It can be used as a faster alternative to --wsize.
For example, `--dpi-desync-split-pos=method+2,midsld,5` means `method+2` for http, `midsld` for TLS and 5 for others.
In `disorder2` and `split2` modes no fake packets are sent, so ttl and fooling options are not required.
### Sequence numbers overlap
`seqovl` adds to the first sent original segment (1st for split, 2nd for disorder) seqovl bytes to the beginning and decreases
sequence number.
In `split2` mode this creates partially in-window packet. OS receives only in-window part.
In `disorder2` mode OS receives fake and real part of the second segment but does not pass received data to the socket until first
segment is received. First segment overwrites fake part of the second segment. Then OS passes original data to the socket.
`seqovl` adds to one of the original segment `seqovl` bytes to the beginning and decreases sequence number. For `split` - to the first segment, for `disorder` - to the beginning of the penultimate segment sent (second in the original sequence).
In `split` mode this creates partially in-window packet. OS receives only in-window part.
In `disorder` mode OS receives fake and real part of the second segment but does not pass received data to the socket until first segment is received. First segment overwrites fake part of the second segment. Then OS passes original data to the socket.
All unix OS except Solaris preserve last received data. This is not the case for Windows servers and `disorder` with `seqovl` will not work.
Disorder requires `seqovl` to be less than `split_pos`. Either statically defined or automatically calculated.
Otherwise desync is not possible and will not happen.
Disorder requires `seqovl` to be less than split position. Otherwise `seqovl` is not possible and will be cancelled.
Method allows to avoid separate fakes. Fakes and real data are mixed.
### ipv6 specific modes
`hopbyhop`, `destopt` and `ipfrag1` desync modes (they're not the same as `hopbyhop` fooling !) are ipv6 only. One `hop-by-hop`,
`destination options` or `fragment` header is added to all desynced packets.
Extra header increases packet size and can't be applied to the maximum size packets.
@ -370,91 +304,36 @@ If it's not possible to send modified packet original one will be sent.
The idea here is that DPI sees 0 in the next header field of the main ipv6 header and does not
walk through the extension header chain until transport header is found.
`hopbyhop`, `destopt`, `ipfrag1` modes can be used with any second phase mode except `ipfrag1+ipfrag2`.
For example, `hopbyhop,split2` means split original tcp packet into 2 pieces and add hop-by-hop header to both.
For example, `hopbyhop,multisplit` means split original tcp packet into several pieces and add hop-by-hop header to each.
With `hopbyhop,ipfrag2` header sequence will be : `ipv6,hop-by-hop,fragment,tcp/udp`.
`ipfrag1` mode may not always work without special preparations. See "IP Fragmentation" notices.
There are DPIs that analyze responses from the server, particularly the certificate from the ServerHello
that contain domain name(s). The ClientHello delivery confirmation is an ACK packet from the server
with ACK sequence number corresponding to the length of the ClientHello+1.
### Server reply reaction
There are DPIs that analyze responses from the server, particularly the certificate from the ServerHello that contain domain name(s). The ClientHello delivery confirmation is an ACK packet from the server with ACK sequence number corresponding to the length of the ClientHello+1.
In the disorder variant, a selective acknowledgement (SACK) usually arrives first, then a full ACK.
If, instead of ACK or SACK, there is an RST packet with minimal delay, DPI cuts you off at the request stage.
If the RST is after a full ACK after a delay of about ping to the server, then probably DPI acts
on the server response. The DPI may be satisfied with good ClientHello and stop monitoring the TCP session
without checking ServerHello. Then you were lucky. 'fake' option could work.
If it does not stop monitoring and persistently checks the ServerHello, --wssize parameter may help (see CONNTRACK).
If the RST is after a full ACK after a delay of about ping to the server, then probably DPI acts on the server response. The DPI may be satisfied with good ClientHello and stop monitoring the TCP session without checking ServerHello. Then you were lucky. 'fake' option could work.
If it does not stop monitoring and persistently checks the ServerHello, --wssize parameter may help (see [CONNTRACK](#conntrack)).
Otherwise it is hardly possible to overcome this without the help of the server.
The best solution is to enable TLS 1.3 support on the server. TLS 1.3 sends the server certificate in encrypted form.
This is recommendation to all admins of blocked sites. Enable TLS 1.3. You will give more opportunities to overcome DPI.
Hosts are extracted from plain http request Host: header and SNI of ClientHello TLS message.
Subdomains are applied automatically. gzip lists are supported.
iptables for performing the attack on the first packet :
Otherwise raw sending SYN,ACK frame will cause error stopping the further processing.
If you realize you don't need the synack mode it's highly suggested to restore drop INVALID rule.
### SYNDATA mode
Normally SYNs come without data payload. If it's present it's ignored by all major OS if TCP fast open (TFO) is not involved, but may not be ignored by DPI.
Original connections with TFO are not touched because otherwise they would be definitely broken.
Without extra parameter payload is 16 zero bytes.
### Virtual Machines
### DPI desync combos
Most of nfqws packet magic does not work from VMs powered by virtualbox and vmware when network is NATed.
Hypervisor forcibly changes ttl and does not forward fake packets.
Set up bridge networking.
`--dpi-desync` takes up to 3 comma separated modes.
* 0 phase modes work during the connection establishement : `synack`, `syndata``--wsize`, `--wssize`. [hostlist](((#multiple-strategies))) filters are not applicable.
* In the 1st phase fakes are sent before original data : `fake`, `rst`, `rstack`.
* In the 2nd phase original data is sent in a modified way (for example `fakedsplit` or `ipfrag2`).
Modes must be specified in phase ascending order.
### CONNTRACK
@ -540,11 +419,7 @@ If nfqws receives a partial ClientHello it begins reassemble session. Packets ar
Then they go through desync using fully reassembled message.
On any error reassemble is cancelled and all delayed packets are sent immediately without desync.
There is special support for all tcp split options for multi segment TLS. Split position is treated
as message-oriented, not packet oriented. For example, if your client sends TLS ClientHello with size 2000
and SNI is at 1700, desync mode is fake,split2, then fake is sent first, then original first segment
and the last splitted segment. 3 segments total.
There is special support for all tcp split options for multi segment TLS. Split position is treated as message-oriented, not packet oriented. For example, if your client sends TLS ClientHello with size 2000 and SNI is at 1700, desync mode is `fake,multisplit`, then fake is sent first, then original first segment and the last splitted segment. 3 segments total.
### UDP support
@ -624,9 +499,9 @@ nfqws sees packets with internal network source address. If fragmented NAT does
This results in attempt to send packets to internet with internal IP address.
You need to use nftables instead with hook priority 101 or higher.
### multiple strategies
### Multiple strategies
`nfqws` can apply different strategies to different requests. It's done with multiple desync profiles.
**nfqws** can apply different strategies to different requests. It's done with multiple desync profiles.
Profiles are delimited by the `--new` parameter. First profile is created automatically and does not require `--new`.
Each profile has a filter. By default it's empty and profile matches any packet.
Filter can have hard parameters : ip version, ipset and tcp/udp port range.
@ -642,7 +517,7 @@ Otherwise verification goes to the next profile.
It's possible that before knowing L7 and hostname connection is served by one profile and after
this information is revealed it's switched to another profile.
If you use 0-phase desync methods think carefully what can happen during strategy switch.
Use `--debug` logging to understand better what `nfqws` does.
Use `--debug` logging to understand better what **nfqws** does.
Profiles are numbered from 1 to N. There's last empty profile in the chain numbered 0.
It's used when no filter matched.
@ -654,6 +529,99 @@ This way you may never unblock all resources and only confuse yourself.
IMPORTANT : user-mode ipset implementation was not designed as a kernel version replacement. Kernel version is much more effective.
It's for the systems that lack ipset support : Windows and Linux without nftables and ipset kernel modules (Android, for example).
### Virtual machines
Most of nfqws packet magic does not work from VMs powered by virtualbox and vmware when network is NATed.
Hypervisor forcibly changes TTL and does not forward fake packets.
Set up bridge networking.
### IPTABLES for nfqws
This is the common way to redirect some traffic to nfqws :
This variant works if DPI is stateful and does not track all packets separately in search for "bad requests". If it's stateless you have to redirect all outgoing plain http packets.
nft add rule inet ztest pre iifname $IFACE_WAN tcp sport "{80,443}" ct reply packets 1-3 queue num 200 bypass
```
To engage `datanoack` or `ipfrag` for passthrough traffic special POSTNAT configuration is required. Generated packets must be marked as **notrack** in the early stage to avoid being invalidated by linux conntrack.
If your device supports flow offloading (hardware acceleration) iptables and nftables may not work. With offloading enabled packets bypass standard netfilter flow. It must be either disabled or selectively controlled.
Newer linux kernels have software flow offloading (SFO). The story is the same with SFO.
In `iptables` flow offloading is controlled by openwrt proprietary extension `FLOWOFFLOAD`. Newer `nftables` implement built-in offloading support.
Flow offloading does not interfere with **tpws** and `OUTPUT` traffic. It only breaks nfqws that fools `FORWARD` traffic.
## tpws
tpws is transparent proxy.
@ -663,7 +631,7 @@ tpws is transparent proxy.
--debug=0|1|2|syslog|@<filename> ; 1 and 2 means log to console and set debug level. for other targets use --debug-level.
--debug-level=0|1|2 ; specify debug level for syslog and @<filename>
--bind-addr=<v4_addr>|<v6_addr>; for v6 link locals append %interface_name : fe80::1%br-lan
--bind-addr=<v4_addr>|<v6_addr>; for v6 link locals append %interface_name : fe80::1%br-lan
--bind-iface4=<interface_name> ; bind to the first ipv4 addr of interface
--bind-iface6=<interface_name> ; bind to the first ipv6 addr of interface
--bind-linklocal=no|unwanted|prefer|force
@ -686,6 +654,7 @@ tpws is transparent proxy.
--skip-nodelay ; do not set TCP_NODELAY for outgoing connections. incompatible with split.
--local-tcp-user-timeout=<seconds> ; set tcp user timeout for local leg (default : 10, 0 = system default)
--remote-tcp-user-timeout=<seconds> ; set tcp user timeout for remote leg (default : 20, 0 = system default)
--fix-seg=<int> ; recover failed TCP segmentation at the cost of slowdown. wait up to N msec.
--no-resolve ; disable socks5 remote dns
--resolver-threads=<int> ; number of resolver worker threads
--maxconn=<max_connections> ; max number of local legs
@ -707,10 +676,9 @@ tpws is transparent proxy.
--hostlist-auto-fail-time=<int> ; all failed attemps must be within these seconds (default : 60)
--hostlist-auto-debug=<logfile> ; debug auto hostlist positives
--split-http-req=method|host ; split http request at specified logical position.
--split-tls=sni|sniext ; split at specified logical part of TLS ClientHello
--split-pos=<numeric_offset> ; split at specified pos. split-http-req takes precedence over split-pos for http reqs.
--split-any-protocol ; split not only http and https
--split-pos=N|-N|marker+N|marker-N ; comma separated list of split positions
--tlsrec=sni|sniext ; make 2 TLS records. split at specified logical part. don't split if SNI is not present.
--tlsrec=N|-N|marker+N|marker-N ; make 2 TLS records. split at specified logical part. don't split if SNI is not present.
--tlsrec-pos=<pos> ; make 2 TLS records. split at specified pos
--mss=<int> ; set client MSS. forces server to split messages but significantly decreases speed !
--mss-pf=[~]port1[-port2] ; MSS port filter. ~ means negation
--tamper-start=[n]<pos> ; start tampering only from specified outbound stream position. byte pos or block number ('n'). default is 0.
--tamper-cutoff=[n]<pos> ; do not tamper anymore after specified outbound stream position. byte pos or block number ('n'). default is unlimited.
--daemon ; daemonize
@ -736,13 +703,49 @@ tpws is transparent proxy.
--uid=uid[:gid] ; drop root privs
```
The manipulation parameters can be combined in any way.
### TCP segmentation in tpws
**tpws** like **nfqws** supports multiple splits. Split [markers](#tcp-segmentation) are specified in `--split-pos` parameter.
On the socket level there's no guaranteed way to force OS to send pieces of data in separate packets. OS has a send buffer for each socket. If `TCP_NODELAY` socket option is enabled and send buffer is empty OS will likely send data immediately. If send buffer is not empty OS will coalesce it with new data and send in one packet if possible.
In practice outside of massive transmissions it's usually enough to enable `TCP_NODELAY` and use separate `send()` calls to force custom TCP segmentation. But if there're too many split segments Linux can combined some pieces and break desired behaviour. BSD and Windows are more predictable in this case. That's why it's not recommended to use too many splits. Tests revealed that 8+ can become problematic.
Since linux kernel 4.6 **tpws** can recognize TCP segmentation failures and warn about them. `--fix-seg` can fix segmentation failures at the cost of some slowdown. It waits for several msec until all previous data is sent. This breaks async processing model and slows down every other connection going through **tpws**. Thus it's not recommended on highly loaded systems. But can be compromise for home systems.
If you're attempting to split massive transmission with `--split-any-protocol` option it will definitely cause massive segmentation failures. Do not do that without `--tamper-start` and `--tamper-cutoff` limiters.
**tpws** works on socket level and receives in one shot long requests (TLS with kyber) that should normally require several TCP packets. It tampers entire received block without knowing how much packets it will take. OS will do additional segmenation to meet MTU.
`--disorder` sends every odd packet with TTL=1. Server receives even packets fastly. Then client OS retransmits odd packets with normal TTL and server receives them. In case of 6 segments server and DPI will see them in this order : `2 4 6 1 3 5`. This way of disorder causes some delays. Default retransmission timeout in Linux is 200 ms.
`--oob` sends one out-of-band byte in the end of the first split segment.
`--oob` and `--disorder` can be combined only in Linux. Others OS do not handle this correctly.
### TLSREC
`split-http-req` takes precedence over split-pos for http reqs.
`--tlsrec` allow to split TLS ClientHello into 2 TLS records in one TCP segment. It accepts single pos marker.
split-pos works by default only on http and TLS ClientHello. use `--split-any-protocol` to act on any packet
`--tlsrec` breaks significant number of sites. Crypto libraries on servers usually accept fine modified ClientHello but middleboxes such as CDNs and ddos guards - not always. Use of `--tlsrec` without filters is discouraged.
tpws can bind to multiple interfaces and IP addresses (up to 32).
### MSS
`--mss` sets TCP_MAXSEG socket option. Client sets this value in MSS TCP option in the SYN packet.
Server replies with it's own MSS in SYN,ACK packet. Usually servers lower their packet sizes but they still don't fit to supplied MSS. The greater MSS client sets the bigger server's packets will be.
If it's enough to split TLS 1.2 ServerHello, it may fool DPI that checks certificate domain name.
This scheme may significantly lower speed. Hostlist filter is possible only in socks mode if client uses remote resolving (firefox `network.proxy.socks_remote_dns`).
`--mss` is not required for TLS1.3. If TLS1.3 is negotiable then MSS make things only worse. Use only if nothing better is available. Works only in Linux, not BSD or MacOS.
### Other tamper options
`--hostpad=<bytes>` adds padding headers before `Host:` with specified number of bytes. If `<bytes>` is too large headers are split by 2K. Padding more that 64K is not supported and not accepted by http servers.
It's useful against stateful DPI's that reassemble only limited amount of data. Increase padding `<bytes>` until website works. If minimum working `<bytes>` is close to MTU then it's likely DPI is not reassembling packets. Then it's better to use regular split instead of `--hostpad`.
### Supplementary options
**tpws** can bind to multiple interfaces and IP addresses (up to 32).
Port number is always the same.
@ -778,44 +781,83 @@ To bind to a specific ip when its interface may not be configured yet do : `--bi
It's possible to bind to any nonexistent address in transparent mode but in socks mode address must exist.
In socks proxy mode no additional system privileges are required. Connections to local IPs of the system where tpws runs are prohibited.
In socks proxy mode no additional system privileges are required. Connections to local IPs of the system where **tpws** runs are prohibited.
tpws supports remote dns resolving (curl : `--socks5-hostname` firefox : `socks_remote_dns=true`) , but does it in blocking mode.
tpws uses async sockets for all activities. Domain names are resolved in multi threaded pool.
**tpws** uses async sockets for all activities. Domain names are resolved in multi threaded pool.
Resolving does not freeze other connections. But if there're too many requests resolving delays may increase.
Number of resolver threads is choosen automatically proportinally to `--maxconn` and can be overriden using `--resolver-threads`.
To disable hostname resolve use `--no-resolve` option.
`--disorder` is an additional flag to any split option.
It tries to simulate `--disorder2` option of `nfqws` using standard socket API without the need of additional privileges.
This works fine in Linux and MacOS but unexpectedly in FreeBSD and OpenBSD
(system sends second fragment then the whole packet instead of the first fragment).
### Multiple strategies
`--tlsrec` and `--tlsrec-pos` allow to split TLS ClientHello into 2 TLS records in one TCP segment.
`--tlsrec=sni` splits between 1st and 2nd chars of the hostname. No split occurs if SNI is not present.
`--tlsrec-pos` splits at specified position. If TLS data block size is too small pos=1 is applied.
`--tlsrec` can be combined with `--split-pos` and `--disorder`.
`--tlsrec` breaks significant number of sites. Crypto libraries on end servers usually accept fine modified ClientHello
but middleboxes such as CDNs and ddos guards - not always.
Use of `--tlsrec` without filters is discouraged.
`--mss` sets TCP_MAXSEG socket option. Client sets this value in MSS TCP option in the SYN packet.
Server replies with it's own MSS in SYN,ACK packet. Usually servers lower their packet sizes but they still don't
fit to supplied MSS. The greater MSS client sets the bigger server's packets will be.
If it's enough to split TLS 1.2 ServerHello, it may fool DPI that checks certificate domain name.
This scheme may significantly lower speed. Hostlist filter is possible only in socks mode if client uses remote resolving (firefox `network.proxy.socks_remote_dns`).
`--mss` is not required for TLS1.3. If TLS1.3 is negotiable then MSS make things only worse.
Use only if nothing better is available. Works only in Linux, not BSD or MacOS.
### multiple strategies
`tpws` supports multiple strategies as well. They work mostly like with `nfqws` with minimal differences.
`filter-udp` is absent because `tpws` does not support udp. 0-phase desync methods (`--mss`) can work with hostlist in socks modes with remote hostname resolve.
**tpws** like **nfqws** supports multiple strategies. They work mostly like with **nfqws** with minimal differences.
`filter-udp` is absent because **tpws** does not support udp. 0-phase desync methods (`--mss`) can work with hostlist in socks modes with remote hostname resolve.
This is the point where you have to plan profiles carefully. If you use `--mss` and hostlist filters, behaviour can be different depending on remote resolve feature enabled or not.
Use `--mss` both in hostlist profile and profile without hostlist.
Use `curl --socks5` and `curl --socks5-hostname` to issue two kinds of proxy queries.
See `--debug` output to test your setup.
### IPTABLES for tpws
Use the following rules to redirect TCP connections to 'tpws' :
It's also possible to use `-j REDIRECT --to-port 988` instead of DNAT but in the latter case transparent proxy must listen on all IP addresses or on a LAN interface address. It's not too good to listen on all IP and it's not trivial to get specific IP in a shell script. `route_localnet` has it's own security impact if not protected by additional rules. You open `127.0.0.0/8` subnet to the net.
This is how to open only single `127.0.0.127` address :
```
iptables -A INPUT ! -i lo -d 127.0.0.127 -j ACCEPT
iptables -A INPUT ! -i lo -d 127.0.0.0/8 -j DROP
```
Owner filter is required to avoid redirection loops. **tpws** must be run with `--user tpws` parameter.
ip6tables work almost the same with minor differences. ipv6 addresses should be enclosed in square brackets :
There's no `route_localnet` for ipv6. DNAT to localhost (`::1`) is possible only in **OUTPUT** chain. In **PREROUTING** chain DNAT is possible to any global address or link local address of the interface where packet came from.
@ -1281,33 +1323,29 @@ For low storage openwrt see `init.d/openwrt-minimal`.
### Android
Its not possible to use nfqws and tpws in transparent proxy mode without root privileges.
Without root tpws can run in --socks mode.
Its not possible to use **nfqws** and **tpws** in transparent proxy mode without root privileges. Without root **tpws** can run in `--socks` mode.
Android has NFQUEUE and nfqws should work.
Android has NFQUEUE and **nfqws** should work.
There's no ipset support unless you run custom kernel. In common case task of bringing up ipset
on android is ranging from "not easy" to "almost impossible", unless you find working kernel
image for your device.
There's no `ipset` support unless you run custom kernel. In common case task of bringing up `ipset` on android is ranging from "not easy" to "almost impossible", unless you find working kernel image for your device.
Android does not use /etc/passwd, `tpws --user` won't work. There's replacement.
Use numeric uids in `--uid` option.
Its recommended to use gid 3003 (AID_INET), otherwise tpws will not have inet access.
Although linux binaries work it's recommended to use Android specific ones. They have no problems with user names, local time, DNS, ...
Its recommended to use gid 3003 (AID_INET), otherwise **tpws** will not have inet access.
Example : `--uid 1:3003`
In iptables use : `! --uid-owner 1` instead of `! --uid-owner tpws`.
Nfqws should be executed with `--uid 1`. Otherwise on some devices or firmwares kernel may partially hang. Looks like processes with certain uids can be suspended. With buggy chineese cellular interface driver this can lead to device hang.
**nfqws** should be executed with `--uid 1`. Otherwise on some devices or firmwares kernel may partially hang. Looks like processes with certain uids can be suspended. With buggy chineese cellular interface driver this can lead to device hang.
Write your own shell script with iptables and tpws, run it using your root manager.
Write your own shell script with iptables and **tpws**, run it using your root manager.
Autorun scripts are here :
magisk : `/data/adb/service.d`
supersu : `/system/su.d`
How to run tpws on root-less android.
How to run **tpws** on root-less android.
You can't write to `/system`, `/data`, can't run from sd card.
Selinux prevents running executables in `/data/local/tmp` from apps.
Use adb and adb shell.
@ -1341,7 +1379,7 @@ You will need :
* root shell access. true sh shell, not microtik-like console
* startup hook
* r/w partition to store binaries and startup script with executable permission (+x)
* tpws can be run almost anywhere but nfqws require kernel support for NFQUEUE. Its missing in most firmwares.
* **tpws** can be run almost anywhere but **nfqws** require kernel support for NFQUEUE. Its missing in most firmwares.
* too old 2.6 kernels are unsupported and can cause errors. newer 2.6 kernels are OK.
If binaries crash with segfault (rare but happens on some kernels) try to unpack upx like this : upx -d tpws.
`ipset/zapret-hosts-users-exclude.txt.gz` или `ipset/zapret-hosts-users-exclude.txt`
При режимах фильтрации `MODE_FILTER=hostlist` или `MODE_FILTER=autohostlist` система запуска передает `nfqws` или `tpws` все листы, файлы которых присутствуют.
При режимах фильтрации `MODE_FILTER=hostlist` или `MODE_FILTER=autohostlist` система запуска передает **nfqws** или **tpws** все листы, файлы которых присутствуют.
Передача происходит через замену маркеров `<HOSTLIST>` и `<HOSTLIST_NOAUTO>` на реальные параметры `--hostlist`, `--hostlist-exclude`, `--hostlist-auto`.
Если вдруг листы include присутствуют, но все они пустые, то работа аналогична отсутствию include листа.
Файл есть, но не смотря на это дурится все, кроме exclude.
@ -1320,21 +1319,21 @@ tpws и nfqws решают нужно ли применять дурение в
Этот режим позволяет проанализировать как запросы со стороны клиента, так и ответы от сервера.
Если хост еще не находится ни в каких листах и обнаруживается ситуация, похожая на блокировку,
происходит автоматическое добавление хоста в список `autohostlist` как в памяти, так и в файле.
`nfqws` или `tpws` сами ведут этот файл.
**nfqws** или **tpws** сами ведут этот файл.
Чтобы какой-то хост не смог попась в `autohostlist` используйте `hostlist-exclude`.
Если он все-же туда попал - удалите запись из файла вручную. Процессы автоматически перечитают файл.
`tpws`/`nfqws` сами назначают владельцем файла юзера, под которым они работают после сброса привилегий,
**tpws**/**nfqws** сами назначают владельцем файла юзера, под которым они работают после сброса привилегий,
чтобы иметь возможность обновлять лист.
В случае `nfqws` данный режим требует перенаправления в том числе и входящего трафика.
Крайне рекомендовано использовать ограничитель `connbytes`, чтобы `nfqws` не обрабатывал гигабайты.
В случае **nfqws** данный режим требует перенаправления в том числе и входящего трафика.
Крайне рекомендовано использовать ограничитель `connbytes`, чтобы **nfqws** не обрабатывал гигабайты.
По этой же причине не рекомендуется использование режима на BSD системах. Там нет фильтра `connbytes`.
На linux системах при использовании nfqws и фильтра connbytes может понадобится :
Было замечено, что некоторые DPI в России возвращают RST с неверным ACK. Это принимается tcp/ip стеком
linux, но через раз приобретает статус INVALID в conntrack. Поэтому правила с `connbytes` срабатывают
через раз, не пересылая RST пакет `nfqws`.
через раз, не пересылая RST пакет **nfqws**.
Как вообще могут вести себя DPI, получив "плохой запрос" и приняв решение о блокировке:
@ -1344,7 +1343,7 @@ linux, но через раз приобретает статус INVALID в con
4) Подмена сертификата: (только для https) полный перехват TLS сеанса с попыткой всунуть что-то
свое клиенту. Применяется нечасто, поскольку броузеры на такое ругаются.
`nfqws` и `tpws` могут сечь варианты 1-3, 4 они не распознают.
**nfqws** и **tpws** могут сечь варианты 1-3, 4 они не распознают.
Всилу специфики работы с отдельными пакетами или с TCP каналом tpws и nfqws распознают эти ситуации
по-разному.
Что считается ситуацией, похожей на блокировку :
@ -1375,7 +1374,7 @@ linux, но через раз приобретает статус INVALID в con
незаблокированный сайт. Эту ситуацию, увы, придется вам контролировать вручную.
Заносите такие домены в `ipset/zapret-hosts-user-exclude.txt`, чтобы избежать повторения.
Чтобы впоследствии разобраться почему домен был занесен в лист, можно включить `autohostlist debug log`.
Он полезен тем, что работает без постоянного просмотра вывода `nfqws` в режиме debug.
Он полезен тем, что работает без постоянного просмотра вывода **nfqws** в режиме debug.
В лог заносятся только основные события, ведущие к занесению хоста в лист.
По логу можно понять как избежать ложных срабатываний и подходит ли вообще вам этот режим.
@ -1567,11 +1566,11 @@ nfqws начнет получать адреса пакетов из локал
`POSTNAT=0`
Существует 3 стандартных опции запуска, настраиваемых раздельно и независимо: `tpws-socks`, `tpws`, `nfqws`.
Существует 3 стандартных опции запуска, настраиваемых раздельно и независимо: `tpws-socks`, **tpws**, **nfqws**.
Их можно использовать как по отдельности, так и вместе. Например, вам надо сделать комбинацию
из методов, доступных только в `tpws` и только в `nfqws`. Их можно задействовать вместе.
`tpws` будет прозрачно локализовывать трафик на системе и применять свое дурение, `nfqws` будет дурить трафик,
исходящий с самой системы после обработки на `tpws`.
из методов, доступных только в **tpws** и только в **nfqws**. Их можно задействовать вместе.
**tpws** будет прозрачно локализовывать трафик на системе и применять свое дурение, **nfqws** будет дурить трафик,
исходящий с самой системы после обработки на **tpws**.
А можно на эту же систему повесить без параметров socks proxy, чтобы получать доступ к обходу блокировок через прокси.
Таким образом, все 3 режима вполне могут задействоваться вместе.
Так же безусловно и независимо, в добавок к стандартным опциям, применяются все custom скрипты в `init.d/{sysv,openwrt,macos}/custom.d`.
@ -1587,7 +1586,7 @@ nfqws начнет получать адреса пакетов из локал
Одновременное использование tpws и nfqws без пересечения по L3/L4 (то есть nfqws - udp, tpws - tcp или nfqws - port 443, tpws - port 80 или nfqws - ipv4, tpws - ipv6) проблем не представляет.
`tpws-socks` требует настройки параметров `tpws`, но не требует перехвата трафика.
`tpws-socks` требует настройки параметров **tpws**, но не требует перехвата трафика.
Остальные опции требуют раздельно настройки перехвата трафика и опции самих демонов.
Каждая опция предполагает запуск одного инстанса соответствующего демона. Все различия методов дурения
для `http`, `https`, `quic` и т.д. должны быть отражены через схему мультистратегий.
@ -1885,7 +1884,7 @@ zapret_custom_firewall_v4
zapret_custom_firewall_v6
```
zapret_custom_daemons поднимает демоны `nfqws`/`tpws` в нужном вам количестве и с нужными вам параметрами.
zapret_custom_daemons поднимает демоны **nfqws**/**tpws** в нужном вам количестве и с нужными вам параметрами.
Для систем традиционного linux (sysv) и MacOS в первом параметре передается код операции: 1 = запуск, 0 = останов.
Для openwrt логика останова отсутствует за ненадобностью.
Схема запуска демонов в openwrt отличается - используется procd.
@ -1914,7 +1913,7 @@ zapret_custom_firewall_nft поднимает правила nftables.
В macos firewall-функции ничего сами никуда не заносят. Их задача - лишь выдать текст в stdout,
содержащий правила для pf-якоря. Остальное сделает обертка.
Особо обратите внимание на номер демона в функциях `run_daemon` и `do_daemon`, номера портов `tpws`
Особо обратите внимание на номер демона в функциях `run_daemon` и `do_daemon`, номера портов **tpws**
и очередей `nfqueue`.
Они должны быть уникальными во всех скриптах. При накладке будет ошибка.
Поэтому используйте функции динамического получения этих значений из пула.
@ -1950,11 +1949,11 @@ zapret_custom_firewall_nft поднимает правила nftables.
Имена интерфейсов WAN и LAN известны из настроек системы.
Под другими системами роутер вы настраиваете самостоятельно. Инсталлятор в это не вмешивается.
инсталлятор в зависимости от выбранного режима может спросить LAN и WAN интерфейсы.
Нужно понимать, что заворот проходящего трафика на `tpws` в прозрачном режиме происходит до выполнения маршрутизации,
Нужно понимать, что заворот проходящего трафика на **tpws** в прозрачном режиме происходит до выполнения маршрутизации,
следовательно возможна фильтрация по LAN и невозможна по WAN.
Решение о завороте на `tpws` локального исходящего трафика принимается после выполнения маршрутизации,
Решение о завороте на **tpws** локального исходящего трафика принимается после выполнения маршрутизации,
следовательно ситуация обратная: LAN не имеет смысла, фильтрация по WAN возможна.
Заворот на `nfqws` происходит всегда после маршрутизации, поэтому к нему применима только фильтрация по WAN.
Заворот на **nfqws** происходит всегда после маршрутизации, поэтому к нему применима только фильтрация по WAN.
Возможность прохождения трафика в том или ином направлении настраивается вами в процессе конфигурации роутера.
Деинсталляция выполняется через `uninstall_easy.sh`. После выполнения деинсталляции можно удалить каталог `/opt/zapret`.
@ -1982,7 +1981,7 @@ zapret_custom_firewall_nft поднимает правила nftables.
## Установка на openwrt в режиме острой нехватки места на диске
Требуется около 120-200 кб на диске. Придется отказаться от всего, кроме `tpws`.
Требуется около 120-200 кб на диске. Придется отказаться от всего, кроме **tpws**.
**Инструкция для openwrt 22 и выше с nftables**
@ -1991,7 +1990,7 @@ zapret_custom_firewall_nft поднимает правила nftables.
***Установка:***
1) Скопируйте все из `init.d/openwrt-minimal/tpws/*` в корень openwrt.
2) Скопируйте бинарник `tpws` подходящей архитектуры в `/usr/bin/tpws`.
2) Скопируйте бинарник **tpws** подходящей архитектуры в `/usr/bin/tpws`.
3) Установите права на файлы: `chmod 755 /etc/init.d/tpws /usr/bin/tpws`
4) Отредактируйте `/etc/config/tpws`
* Если не нужен ipv6, отредактируйте `/etc/nftables.d/90-tpws.nft` и закомментируйте строки с редиректом ipv6.
@ -2019,7 +2018,7 @@ zapret_custom_firewall_nft поднимает правила nftables.
***Установка:***
1) Скопируйте все из `init.d/openwrt-minimal/tpws/*` в корень openwrt.
2) Скопируйте бинарник `tpws` подходящей архитектуры в `/usr/bin/tpws`.
2) Скопируйте бинарник **tpws** подходящей архитектуры в `/usr/bin/tpws`.
3) Установите права на файлы: `chmod 755 /etc/init.d/tpws /usr/bin/tpws`
4) Отредактируйте `/etc/config/tpws`
* Если не нужен ipv6, отредактируйте /etc/firewall.user и установите там DISABLE_IPV6=1.
@ -2049,7 +2048,7 @@ tpws будет работать в любом случае, он не треб
Хотя linux варианты под Android работают, рекомендуется использовать специально собранные под bionic бинарники.
У них не будет проблем с DNS, с локальным временем и именами юзеров и групп.\
Рекомендую использовать gid 3003 (AID_INET). Иначе можете получить permission denied на создание сокета.
Рекомендуется использовать gid 3003 (AID_INET). Иначе можете получить permission denied на создание сокета.
Например: `--uid 1:3003`\
В iptables укажите: `! --uid-owner 1` вместо `! --uid-owner tpws`.\
Напишите шелл скрипт с iptables и tpws, запускайте его средствами вашего рут менеджера.
@ -2057,7 +2056,7 @@ tpws будет работать в любом случае, он не треб
magisk : /data/adb/service.d\
supersu: /system/su.d
`nfqws` может иметь такой глюк. При запуске с uid по умолчанию (0x7FFFFFFF) при условии работы на сотовом интерфейсе
**nfqws** может иметь такой глюк. При запуске с uid по умолчанию (0x7FFFFFFF) при условии работы на сотовом интерфейсе
и отключенном кабеле внешнего питания система может частично виснуть. Перестает работать тач и кнопки,
но анимация на экране может продолжаться. Если экран был погашен, то включить его кнопкой power невозможно.
Изменение UID на низкий (--uid 1 подойдет) позволяет решить эту проблему.
bol-van
5 months ago