其實網路上關於 NAT Loopback 的文章到處都有,從原理,到如何在實作等網路上各種資源都有,那這篇文章跟網路上的教學又有什麼不同?
我唯一能夠貢獻的是 NAT Loopback 關於 的問題,在某些情況下, 會因為 的關係導致無法運作。當初我遇到這個問題時也是百思不得其解,於 google 世界到處尋找,都沒有看到任何線索。
最後只好自己深入 kernel 內找尋原因,從 Linux Kernel Network Stack 開始翻找。
經過一些時間的研究與証實後,也終於確認了某個原因,然後將這個問題的關鍵字轉換後,也有找到一個沒有上到 upstream 的 kernel patch 針對此問題處理。
The only thing I can contribute to is NET Loopback about & in some cases, because & can't work. When I first met this problem, I couldn't think much about it, looking around the world at google, without seeing any clues.
ended up only getting deep & nbsp; kernel & nbsp; & & nbsp; & & nbsp; & & & & bn; & & & kn; & & bn; & & & bn; & & & & ; & & bn; & & & ; & ; & & bn; & & & ; & & ; & & bspn;
在真正踏入核心問題以前,還是要來說明一下什麼是 NAT Loopback,對於這個概念有基本的認知與瞭解後,再來實際看看 kernel 上面出現了什麼問題,以及那個 patch 是如何解決這個問題
Before we really get to the heart of the problem, let's talk about what's NT Loopback, and when we have basic knowledge and understanding of the concept, let's actually look at & nbsp; kernel what's going on up there, and the & nbsp; patch & nbsp; how to solve this problem.
首先,假設有一個以下的網路環境,我們在 Router 後面設置了兩台 機器,一台是 Web Server,另外一台則是一般的 PC。
由於該 PC 跟該 Web Server 都屬於同一個網域且都在 Router 底下,因此兩台機器之間若要透過 IP addresss 來傳輸基本上沒有太多問題。
First, assuming that there is a network environment below Router we have two machines behind us, one Web Server, and the other is a common PC.
For that PC and the other & nbsp; Web Server & nbsp; all in the same domain and all in Router & nbsp; below, there are basically no problems in transmitting between the two machines.
但是外網的機器想要存取該 Web Server 的話,由於 Web Server 本身的 IP address 屬於 Private Network,譬如192.168.0.0/16這個範圍內。
因此外網的機器本身並沒有辦法直接存取到該 Web Server,但是若我們能夠將封包送到前面的 Router,再透過某種方式告訴 Router 說這個封包不是給你的,請幫我往下轉發給底下的 Web Server,則封包就可以很順利的到達 Web Server 去,一切的連線就順利完成。
But the outside machine wants to access the & nbsp; Web Server by Web Server its own IP address belonging to
1
|
1.2.3.4:8001 ---> 192.168.1.5:80
|
對於 Router 來說,當看到封包的 ip:port 是 1.2.3.4:8001,則會將封包標頭改成 192.168.1.5:80,然後依照本機端內的 route rules 將其轉發到底下的 Web Server 去。
For Louter  :port & nbsp; 1.2.3.4:8001, the seal will be changed to 192.168.1.5:80, followed by roote riles Web Server & nbsp; go.
所以假設今天外網的機器(9.8.7.6)發送了一個封包,其流向是
當 Router 收到此封包後,就會將其轉換成
So let's say today's outer web machine (9.8.7.6), sends a package that flows to
when & nbsp; router & nbsp; when you get this package, you convert it to
當 Web Server 收到此封包後,會有一個回應的封包,此封包的流向是
當此封包到達 Router 後, Router 會先查詢看看這個封包是不是經過上述規則轉換的,若是的話就將封包內容重新轉成(進來的封包轉換其 Destination, 回去的封包轉換其 Source)
When & nbsp; Web Server receives this package, there will be a response package, which will flow to
when the package reaches & nbsp; Louter and Router first check to see if the package has been converted through the above-mentioned rules and, if so, re-route the contents of the package (for incoming packages to replace their Destination, return packages to convert their Source
).
這樣外網的機器 (9.8.7.6) 就可以很順利跟內網內的 Web Server 溝通了。
So the outer-net machine (9.8.7.6) can easily communicate with the inner-net & nbsp; Web Server
上述的這個行為有些會稱 Port Forwarding,有些會稱 Virtual Server,不論怎麼稱呼,其背後的意義都相同。
Some of the above-mentioned actions are called Port Forwarding; others & nbsp; Virtual Server, however they may be called, have the same meaning behind them.
然而在真實的環境中,我們通常不會去死記這些 IP address,我們會使用 DNS 的服務來幫這些 IP address 設定一組好記的名稱,舉例來說可以設定 webserver.com 指向 1.2.3.4。
在這種情況下,外面機器想要存取該 webserver 的流程就會是
In a real environment, however, we usually don't forget these IP address, we use DNS services to help these IP address set a good list of names, for example Webserver.com point to 1.2.3.4.
In such a situation, the outside machine wants to access & nbsp; Webserver the process will be:
- 外網機器(9.8.7.6)想要存取 webserver.com,因此向 DNS server 詢問其對應的 IP address
- DNS server 回應 webserver.com 就是 1.2.3.4,因此外網機器接下來會發送封包到 1.2.3.4
- 封包到達 1.2.3.4 後,根據 DNAT 的規則轉送到底下真正的 web server。
- 底下的 web server 回送封包,透過 1.2.3.4 送回到外網機器(9.8.7.6)
其流程可以用下列兩張圖來說明
The process can be described as
假設我們都已經瞭解上述的概念後,接下來我們將該外網電腦()的角色給放到同樣區網內(192.168.1.6)來看,基本上 代表的涵意就是讓內網的機器能夠遵循原本的流程去存取內網的機器。
Assuming that we all know the above concepts, then we put the role of the Extranet computer in the same area network
在這種情況下,若內網的機器想要依循上述的流程運行
In this case, if the machine on the inside of the Internet wants to follow the process described above,
- 首先內網機器 (192.168.1.6) 透過 DNS 的服務,得到 webserver.com 指向 1.2.3.4
- 接下來將封包送往到 1.2.3.4,遇到 DNAT 後將封包轉換
所以假設今天內部機器(192.168.1.6)發送了一個封包,其流向是
當 Router 收到此封包後,就會將其轉換成 - 當 web server 收到封包後就會回應一個封包,該封包透過 Router 就會依循上述的模式回到內網的機器(192.168.1.6)。
上述的流程看起來是順利也沒有問題的,但是有時候實體網路環境中,可能這些機器(PC,Server)是接在同一台 switch 底下,譬如下列這種情況,
或是 Router 內含 Hardware L2 switch。
The processes described above seem smooth and unproblematic, but there are times when the machines (PC, Server) are attached to the same switch, for example,
(只做DNAT,不做snat)
& nbsp; & nbsp; & nbsp;
4.當封包到達switch時,就會發現這是個同網段的封包,所以就直接幫他回傳給內網機器 192.168.1.6了
5.當內網機器收到這個封包時,就會感受到一臉困惑。
一開始送出去的封包是
所以期望收回到的封包應該是
所以當他看到不符合期望的封包標頭時,就會將其丟棄
整個流程如下圖所示
The whole process is
這邊最大的問題就是 web server 送回去的封包必須要先給 Router 將其根據 DNAT 的規則給重新反轉一次。
但是在此環境下,因為中間有一台 switch 存在,所以封包就沒有送回到 router 那邊去處理而是直接送回去給內網機器了。
The biggest problem here is webserver the package to be returned must first be given to Router the rule to reverse it according to DNA T
, but in this environment, because there is a switch there is one in the middle, the package is not returned to rooter it is processed over there and returned directly to the inner machine.
若要能夠處理上述的情況,我們就必須要想辦法將封包也送回到 router 端去處理,為了達到這個目的我們可以在 router 也採用 SNAT (Source NAT)
規則大概如下,只要是從某個 interface 近來的,就將此封包標頭內的 Source IP Address 變成 192.168.1.1。
To be able to deal with the above, we must find a way to return the package to rooter & nbsp; end up in root & nbsp; also use the SNAT (Source Nat)
rule, presumably as follows, provided it is from one & nbsp; interface & nbsp; nearest & nbsp; Source IP Address & nbsp; become & nbsp; 192.168.1.1.
1
|
in_interface=xxxx, source ip=192.168.1.1:xxxx
|
至於實際上要採用 Masquerade 或是 SNAT 來決定怎麼轉換 Source IP 都可以。
As for the actual use of Masquerade or SNT to decide how to convert Source IP
因此,目前的設定中,Router同時會進行 SNAT 以及 DNAT,因此假設內網機器(192.168.1.6)要對 1.2.3.4:80進行存取。
接下來以下圖來解釋每個步驟中封包的變化。
Therefore, in the current setup, Router will also do SNAT and DNAT, so assume that the in-network machine (192.168.1.6) is to access 1.2.3.4:80.
next to explain the change of envelopes in each step.
( DNAT, snat, 从192)
藍色區域
1,2:
接下來封包會進入 router,執行 SNAT/DNAT
3,4:
& nbsp; & nbsp; & nbsp; 當封包到達 web server後,接下來 web server 會回傳一個封包回去 When the package reaches & nbsp; webserver; & nbsp; returns a package to 當內網機器(192.168.1.6)收到此封包後因為與預期的相同,所以就可以正確地建立起連線並且開始傳輸。 When the inner network machine (192.168.1.6) receives the package, it is able to establish the connection correctly and start transmission because it is the same as expected. 到這邊我們已經完成了最基本的 NAT Loopback,基本上大部分的情況都可以依照這種思路來完成。 We've already done the most basic & nbsp; NT Loopback, and most of the situation can be done along these lines. Of course, if there are some Hardware in your network that will help you to do something secretly, then your bag may be affected by the whole transfer problem, and be very careful here. 前面講了這麼多話之後,我們來看看實際操作上可能會遇到的問題。 After all we've said, let's look at the practical problems that might be encountered. 為了簡化問題,我們假設 router 含有八個實體連接埠,其中第一個連接埠跟底下的switch有連結。 To simplify the problem, we assume & nbsp; rooter & nbsp; contains eight physical ports, the first of which is connected to the bottom of the switch. 假設這一台 Router 我們系統中有透過 Linux bridge 創建了一個 bridge br0,然後我們幫八個連接埠都接到該 br0底下,其中第一個連接埠對應到系統上的 interface 是 eth0 Let's say this Louter & nbsp; there's   in our system; Linux Bridge & nbsp; there's a Bridge br0, and then we get the & nbsp; under br0, the first port of the connection to the system is & nbsp; eth0 从eth0收到包,又要从eth0发出 & nbsp; & nbsp; & nbsp; & nbsp; & nbsp; & nbsp; & nbsp; & nbsp; & nbsp; & nbsp; & nbsp; & nsp; & nbsp; & & & nbsp; & & nbsp; & & & ; & nbsp; & nbsp; & & & ; & & ng; & bbsp; & & ng; & & n; & bbsp; & bbsp; & & & ng; & & n; & bbsp; & & & ngbsp; & & nbsp; & & nbsp; & & nbsp; & & & nbsp; & & & & ngbsp; & & & & ngbsp; & & & ngbsp; & & & & ngbsp; & ngbbsp; & & & & & ngbbsp; & ngbsp; & & ngbsp; & & ngbbsp; & & & & ngbbbbbsp; & & & & & & & & & ngbbbbbbbsp; & & & & ngbbbbsp; & & & & & & ngbbbsp; & & ngbsp; ngbsp; & & & & & & & ngbbbbbsp; & & & & & & & & 在這種情況下,剛剛上述 NAT Loopback 的封包會遇到一問題。 In this case, the package that was just & nbsp; NAT Loopback will have a problem. 上面程式碼有一個最重要的地方 There's one of the most important places on the code above: 可是在當前的網路拓墣中你就是要這個封包去轉發,所以可以觀察到上述程式碼還有一個關鍵點 < < & & & @rbin_MODE> > (p-> & & BR_HARPIN_MODE), ; > ; rel = noopener 後來發現網路上也有其他人遇到一樣的問題,該使用者因為沒有辦法針對 user-space 去進行修改,所以只能從 kernel 內進行一些小部分的修改,希望可以處理這個問題 There were similar problems encountered by other people on the Internet, and because the user had no means of amending user-space it had to do so from kernel there had been minor modifications in 其實上述的問題一些家用 router 不會遇到的一個原因是 kernel 太舊了,就如同該 patch 所說, 於 2.6.35 後的系統就會有這樣的問題存在,有些家用 router 的 kernel 還在 2.6.x 然後沒有追上新的,因此剛好逃過此問題。 In fact, some of the above-mentioned problems are root & nbsp; one of the reasons that will not be encountered is too old, as as stated, in 2.6.35& nbsp; there will be problems in the subsequent systems, some of which use & nbsp; rooter& nbsp; kernel is still 2.6.x and has not been followed, so it is easy to escape the problem. In the below network topology a web server behind a router is on private IP address space, and the router performs NAT to forward traffic to its public IP address to the web server behind it. The NAT configuration would look like below: When a client out on the Internet with IP address 2.2.2.2 establishes a connection to the web server, the router performs NAT as configured. The client receives the reply packet it expects, and the connection is established. When a client on the same internal network as the web server requests a connection to the web server's public IP address, the connection breaks. The client receives the reply packet, but it discards it because it expects a packet back from 1.1.1.1, and not from 192.168.1.2. As far as the client is concerned the packet is invalid and not related to any connection the client previously attempted to establish. To fix the issue, an additional NAT rule needs to be introduced on the router to enforce that all reply traffic flows through the router, despite the client and server being on the same subnet. The rule below is very specific to only apply to the traffic that the issue could occur with - if there are many servers the issue occurs with, the rule could be made broader to save having one such exception per forwarded service. With that additional rule, the flow now changes: The client receives the reply packet it expects, and the connection is established. However, the web server only ever sees a source IP address of 192.168.1.1 for all requests from internal clients regardless of the internal client's real IP address. There is no way to avoid this without either using a router that can do application level DNS inspection and can rewrite A records accordingly, or a split DNS server that serves the internal clients the internal server IP address and external clients the external server IP address. This is called - among other terms - hair pin NAT because the traffic flow has clients enter the router through the same interface it leaves through, which when drawn looks like a hair pin. http://chenchun.github.io/network/2017/10/09/hairpin
1,2:
當封包到達 switch 時,查了一下目的地是 ,因此就會幾該封包送回到 router 去處理。
當封包到達 router 時,會根據之前的記錄瞭解該封包有使用過 SNAT 以及 DNAT,因此會將封包標頭給重新修改
3,4:
1,2:
when the package reaches and therefore several packages will be returned to root process.
when the package reaches root
當然若是你網路中間有遇到一些 Hardware 會幫你偷偷做事情的,那你的封包可能就會被影響導致整個傳輸都出問題,這邊要特別小心。
以下列這張圖為範例
uses the example of
所以這時候大概可以看到如下面的架構
so you can probably see the structure of the following.
1
2
3
4
5
br0:
eth0
eth1
...
eth8
當內網機器的封包送到 router時,會先透過 eth0進入到系統後到達 br0,接下來進行 SNAT 以及 DNAT 的處理。
然後最後封包又要從 br0 往 eth0 出去,一切的料想都是如此美好。
然而實際上就會發現封包不見了!!
根據 Linux kernel 3.6 source code,當系統底下的 bridge 再轉發封包的時候,會呼叫到 br_forward 去處理。
The package of the inner net machine will be delivered to
The package of DNA T Span style=" Color: rgba (255, 102, 0)" before br0 & nbsp; & nbsp; & nbsp; & & nbsp; DNA T ;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
void br_forward(const struct net_bridge_port *to, struct sk_buff *skb, struct sk_buff *skb0)
{
if (should_deliver(to, skb)) {
if (skb0)
deliver_clone(to, skb, __br_forward);
else
__br_forward(to, skb);
return;
}
if (!skb0)
kfree_skb(skb);
}
1
2
3
4
5
6
7
static inline int should_deliver(const struct net_bridge_port *p,
const struct sk_buff *skb)
{
return (((p->flags & BR_HAIRPIN_MODE) || skb->dev !=p->dev) &&
p->state==BR_STATE_FORWARDING);
}
skb->dev !=p->dev,如果當前封包進入的 bridge port 跟出去的 bridge port 是一樣的話,那就不會轉發,導致這個封包被丟棄了…
skb-> dev! =p-> dev, if the package entered in front & nbsp;
(p->flags & BR_HAIRPIN_MODE),
根據這篇 patch, 只要針對 interface 去啟用 hairpin_mode 就可以讓封包順利從同個點進出來回了。
但是事情依然沒有這樣簡單,這樣完畢後封包的 IP 的確都有正確的修改了,但是在 MAC Address 的部分有點問題,Source MAC沒有如預期的被修改,所以這邊又要依賴另外一個工具 ebtables 來進行 MAC 的修改,再者種情況下,封包就可以順利通過了。
因此我們的 Router 就有三種設定
1.打開 hairpin mode
2.執行 iptables 的 SNAT/DNAT(改 IP)
3.透過 ebtables 的 SNAT (改 MAC)
這邊可以參考這個 patch
在這個 patch 中,該程式碼會先針對有進行 DNAT 的封包進行標記,然後在 bridge forward 的過程中,將該封包的 Source MAC 進行修改,最後再讓該封包通過往下轉發。
, and it was hoped that the code would first be available for
in this patch; the code would first be used for DNA T the seal would be marked and the package would then be turned over to & nbsp; the bridge between & & & & nbsp; & & & & & & & & & mbsp; in the process, the package would be changed./ip firewall nat
add chain=dstnat dst-address=1.1.1.1 protocol=tcp dst-port=80 \
action=dst-nat to-address=192.168.1.2
add chain=srcnat out-interface=WAN action=masquerade
/ip firewall nat
add chain=srcnat src-address=192.168.1.0/24 \
dst-address=192.168.1.2 protocol=tcp dst-port=80 \
out-interface=LAN action=masquerade
发表评论