I. INTRODUCTION
DUE to the flexibility, easy deployment, relatively low cost, wide coverage, and massive connectivity of unmanned aerial vehicles (UAVs) [1], [2], UAV ommunications have attracted increasing research attention and applications. UAV systems have been used to take many tasks, such as Internet of things (IoT) data transmission [1], border surveillance [3], disaster monitoring [4], and emergency communications [5]. At present, tasks are getting more complex, and a single UAV faces different challenges such as limited power, short-communication ranges, rapid topology, low bandwidth, and handover [6]. Therefore, the applications of UAVs become networking and clustering, and flying ad-hoc network (FANET) has been widely investigated [7].
Benefiting from the centerless, self-organizing, and multihop communication, FANET meets the requirements of multinode collaboration for complex tasks [8]. However, FANET also faces many challenges in practical applications:
· Resource management: FANET is a network that is limited by energy and bandwidth, where the dynamicity of UAVs may lead the network to be reconfigured frequently [8]. This may lead to network instability for the conventional centralized resource management schemes, which are also low efficient and lack scalability.
· Routing protocol: In FANET, the performance of routing protocol determines the stability, accuracy and timeliness of data transmission. Applications to FANET have different requirements for UAV speeds, and the link is with a high bit error rate (BER) due to the interference [9]. Additionally, the requirements of reliability from the FANET are also diverse. To meet the requirements of FANET, such as reliable services, multi-service transmission, low latency, and efficient routing, the routing protocol needs to satisfy the unique features especially with ultra-dense, dynamic topology, and diverse services. Designing a reliable routing protocol for the FANET becomes another challenge.
This paper focuses on the swarm FANET, mainly discusses the application of the cross-layer design method and mean field game (MFG) theory to optimize the routing protocol in the large-scale FANET. In the traditional hierarchical model, the limitation of communication is broken by the cross-layer design method. All layers can share local information through cross-layer interaction, which can effectively improve the utilization of the resource of FANET. It is clear that for analyzing, modeling, and solving power control of large-scale networks. The MFG theory provides a feasible method. A cross-layer resource management method is proposed, considering the power resources of the physical layer, the frequency resources of the medium access control (MAC) layer, and the routing resources of the network layer. According to the characteristics of FANET, the dynamic source routing (DSR) protocol is focused on. Furthermore, we add the link quality measurement to improve the DSR protocol.
We list our main contributions of our paper below:
· Cross-layer resource management algorithm using MFG: In this paper, for the conventional centralized resource management schemes that are low efficient in swarm UAV networks with limited resources, the resource management algorithm is proposed, which combines the physical layer, MAC layer, and network layer. The cross-layer design can ffectively improve the resource utilization and MFG can take into account both local and entire network resource utilization of FANET. Through the management and allocation of power resources and frequency resources in the physical layer and MAC layer using MFG, we design the cross-layer optimization cost function to maximize the energy utilization efficiency.
· Cross-layer resource management dynamic source routing (CLRM-DSR) protocol in FANET: Aiming at the rapid change of the network topology caused by high dynamic characteristics of FANET, we have added the link quality into DSR to provide the optimal solution for network routing. In the CLRM-DSR protocol, the link quality combines with the cross-layer cost function, which can be obtained by the cross-layer resource management algorithm. By comparing the link quality, the data transmission path in the network layer is planned, and the network performance of information transmission is improved.
· FANET simulation platform: A FANET simulation platform is built, which is used to construct a discrete-time simulation environment for the CLRM-DSR and the proposed cross-layer resource management algorithm. It integrates the physical layer, MAC layer and network routing implementation. Meanwhile, this simulation platform can also implement the algorithm verification of the conventional mobile ad-hoc network (MANET), the vehicle ad-hoc network (VANET).
The rest of this paper is structured as follows. Section II summarizes the research status about the cross-layer design, MFG and routing protocols. Section III models the swarm FANET system and cross-layer resource management problem. In Section IV, how to obtain the cross-layer resource management algorithm by decomposing that problem into subproblems and solving them is described. Meanwhile, how to apply these algorithms in the DSR protocol is explained. Section V introduces the FANET simulation platform and analyzes the performance improvement of the cross-layer resource management algorithm in the ultra-dense scenario. Finally, the conclusion and future work are drawn in Section VI. Table I
ACRONYMS AND FULL SPELLING.
lists the acronyms that appear in this paper.
II. RELATED WORKS
Typically, network resource management has been widely studied, which is always described as a constrained optimization problem with a specific utility function or cost function [10]. The FANET resource management problem involves more complex aspects: rate allocation, bandwidth allocation, power control, frequency scheduling, access control, topology management, routing, or end-to-end quality of service [11]. It aims to allocate network resources dynamically, improve network performance, maximize network utility and provide end-to-end quality of service under the conditions of low latency, time-varying channel, finite bandwidth, and dynamic topology [12].
One of the tools for designing resource management algorithms is the cross-layer design [13]. It breaks the communication restriction between layers, which is different from the modular layered design method [14]. In this way, the network divided into multiple layers can be designed, analyzed, optimized, and controlled as a whole. As described in [15], the cross-layer framework created for routing, relay selection, and power allocation is capable of improving user utility and reducing power loss. In [16], the cross-layer design method was adopted for real-time video transmission. When the data need to be transmitted, the payload will be passed from the transport layer to the network layer and the MAC layer, while the capacity information will be fed back from the MAC layer to the other layers. By combining multiple protocol layers, [17] implemented cross-layer modeling and optimization of multi-dimensional resources. However, the cross-layer design method used in FANET lacks a unified cross-layer optimization goal, and the dynamic relationship between the layers remains unclear [18]. Therefore, we need an approach that can take into account the multi-dimensional cross-layer resource model, interaction mechanism, and spatiotemporal dynamic relationship of resources.
Currently, many game models for multi-hop networks are presented [19] and researchers have applied game theory to the resource management of the FANET [20]-[22]. Game theory has some advantages over other conventional methods. Firstly, it can solve different problems with multiple players. During the game, multiple different goals on a player can be jointly optimized, and the rational behavior characteristics of multiple participants can be described. Secondly, it can model the interaction between players, analyze equilibrium and provide insights to design distributed resource management algorithm [1].
Authors in [23] presented a survey of using game theoretic techniques to deal with the problems in UAV wireless communication networks and those game theories are compared. Conventional game theory has complex information interaction and high communication overhead when applied to swarm FANETs. Meanwhile, it takes a longer time to reach equalization with the increasing number of UAVs [1]. Fortunately, MFG is an effective method to solve UAV resource management in such ultra-dense networks due to the following reasons:
· Less information interaction complexity: Game theory can maximize the utilization of resources in the local and entire network, and improve the utilization of network resources while ensuring user fairness and service quality. They are used to optimize the spectrum efficiency of the local and whole network. However, unlike conventional game theory, MFG can model and analyze the interaction between individual behavior and collective behavior of a swarm FANET. Instead of modeling the interaction process between players, MFG models individual behavior through group actions, which minimizes the complexity of information interaction.
· Low communication overhead: By definition, MFG has state space, action space and policy control. The player's policy control can map a state in the state space to action in the action space for a pre-defined period time in MFG. As a result, the players' information that needs to exchange among different UAVs in the FANET is low, and the communication overhead generated by information exchange is less than the applications of other game theories. MFG theory processes the influence of all nodes by using mean field theory (MFT), and replaces the sum of a single effect with an average effect, which simplifies the analysis process and reduces the calculation amount of the game solution.
· Adaptability of ultra-dense scenario: As the number of nodes increases, the value of the mean field in MFG is more able to characterize the real environmental value. While the number of nodes tends to infinity, the MFG equilibrium approaches the Nash equilibrium or the Markov equilibrium. MFG is a special type of stochastic differential game, which is mainly used for the modeling of large-scale systems dynamic analysis.
For the advantages of MFG, the MFG has been proposed for power control in different types of cellular networks. The author in [24] designed an interference-aware and power con-
APPLICABILITY OF PROTOCOLS TO UAV NETWORK.
trol scheme based on MFG to improve energy and spectrum efficiency in a 5G ultra-dense cellular network. In [25], MFG is adopted to divide complex large-scale optimization problems into a series of local optimization problems, significantly reducing the communication overhead and convergence time of interference management in the heterogeneous network. Thus, each small cell base station can implement its policy by using only its local information and some macroscopic information. Authors in [26] presented a distributed control and energy-aware power control algorithm for ultra-dense device-to-device communication (D2D) networks based on MFG theory. The control problem considers both the energy remaining in the device batteries and the interference caused by the communication devices to others. Authors in [27] proposed a downlink power control algorithm in ultra-dense cellular networks where base stations (BSs) are uniformly distributed over a two-dimensional infinite Euclidean plane. In this model, users are distributed independently of BSs, and the BSs with no user within the coverage stay dormant to serve energy.
Despite numerous research conducted in this field, most schemes have focused on terrestrial networks. Different game-theoretic techniques and their applications have different wireless communication and networking problems in specific communication networks [28]. In swarm FANET, due to the limitation of node capacity, the resources management using MFG needs to solve multiple problems, such as the power control, the frequency scheduling and the interference problem.
FANET appears as an extension of VANET or MANET at the beginning, which can work in areas that VANET or MANET cannot reach [29]. In the related researches and experiments of the FANET, the routing protocol required for networking uses the existing MANET routing protocol [30], [31]. The author in [32] focused on the enhancement of the opti-
A swarm scenario of FANET.
ized link-state routing (OLSR) protocol in FANET specifically on node-link expiration and residual energy. Authors in [33] proposed the solution that counts the link quality information directly using the physical layer to optimize the routing. [34] presented a novel routing protocol for FANET using modified AntHocNet, which is better than generic ant colony optimization (ACO) and other traditional routing protocols utilized in FANET. [35] enhanced the FANET network packet delivery rate by reducing unnecessary rebroadcast messages in ad-hoc on-demand distance vector routing (AODV) protocol. [36] designed the adaptive HELLO message interval in routing protocol to improve the energy utilization efficiency of the nodes in FANET, which makes the routing protocol meets the network performance requirements with the lowest energy cost. Stanford research institute (SRI) International conducted the first flight experiment of the FANET [37]. Topology broadcast based on reverse-path forwarding (TBRPF) is used as the network layer routing protocol, and TBRPF is a proactive routing protocol, which effectively reduces the routing overhead [38]. Authors in [39] designed another FANET test platform using the dynamic source routing (DSR) protocol. Khare believes that the DSR protocol only initiates the route lookup process when the source nodes have data transmission, which is more effective for the high mobility of the node and the dynamic change of the topology of the FANET than the proactive routing protocol [40]. In another recent study in [41], the authors concluded that DSR has the highest throughput and is the most suitable routing protocol among others in a high movement and dynamic network. Table II summarizes the routing protocols in MANET and their applicability to the UAV environment.
III. SYSTEM MODEL AND PROBLEM MODEL
A. System Model
A swarm FANET scenario is shown in Fig. 1. In this paper, this FANET scenario can be described by using a directed graph. There is a pair of the source node (node i)-destination node (node j) on this scenario, and some mathematical symbols are listed as follows:
· [TeX:] $$\mathcal{G}=(\mathcal{V}, E)$$, considering asymmetric links in the FANET, the topology is defined as a directed graph, where [TeX:] $$\mathcal{V}=\{1,2, \cdots, M\}$$ is the set of UAVs and E is the wireless links for communication between nodes.
· [TeX:] $$\mathcal{A}^{\mathcal{G}}=\left\{a_{i j}\right\}_{i, i \in V}, \text { when }(i, j) \in E, a_{i j}=1$$, it means that node i can directly communicate with node [TeX:] $$j$$.
· [TeX:] $$\mathcal{V}=S \cup R \cup D$$, assuming that the FANET system is composed of a series of transmitting nodes, receiving nodes, and relay nodes, where [TeX:] $$\mathcal{S}$$ is the transmitting node set, [TeX:] $$\mathcal{R}$$ is the receiving node set, and [TeX:] $$\mathcal{D}$$ is the relay node set.
· [TeX:] $$\mathcal{F}=\left\{f_{\min }, f_{\min +1}, \cdots, f_{\max -1}, f_{\max }\right\}$$, which is the available radio spectrum resources in the FANET. Mean-while, the bandwidth between adjacent spectrum can be defined as w.
· [TeX:] $$E^{\max }$$, the available maximum energy for UAV.
· [TeX:] $$\mathcal{Q}(t)=\left\{q_{1}, \cdots, q_{M}\right\}$$, the set of packet queue length on UAVs at time t.
[TeX:] $$\mathcal{X}=\left\{x_{1}, \cdots, x_{M}\right\}$$, the set of UAV geographical location. Each element is composed of longitude, latitude, and height.
The corresponding UAV energy model, information queue model, and mobility model are defined as the sequel.
The energy model of UAV is defined as
where [TeX:] $$p_{j, t}$$ is the power of node j at time t. [TeX:] $$\delta$$ is the step in the time dimension, and its value depends on the designed algorithm.
The information queue model of UAV is
where [TeX:] $$D_{j} 1_{\text {success }}(t, s)$$ is the process of transmitting a packet from the queue on the UAV, [TeX:] $$A_{j, t+1}$$is the process of adding packets to the queue on the UAV.
The mobility model is defined as
where [TeX:] $$w_{j, t+1}$$ is a Wiener process with a mean of zero and a limited variance. [TeX:] $$\delta_{x}$$ is a value greater than zero. [TeX:] $$v\left(t, x_{j, t}\right)$$ is a function related to the speed of UAVs.
Establishing an appropriate interference model can improve the overall performance and resource utilization of a wireless network [47]. In the FANET, the interference interaction model is considered as follows. Assuming that nodes i and j are communicating at frequency f, the transmission of node j's neighbor node k at frequency f will interfere the node j. Since the signal is transmitted over the wireless link, the path loss model and channel fading model are considered as (4) and (5).
The path loss model is
where [TeX:] $$\alpha(f)>2$$ is the path loss index, [TeX:] $$z_{j}$$ is the location of node j, and [TeX:] $$\left\|z_{i}-z_{j}\right\|$$ is the Euclidean distance between nodes i and j.
The channel fading dynamic model is [48]
where [TeX:] $$\mu^{f}$$ is a non-negative constant, [TeX:] $$\eta^{f}$$ is greater than zero but less than infinity, and [TeX:] $$W_{i, j}^{f}(t)$$ is a mutually independent Wiener process.
Based on path loss model and channel fading model, the channel gain of link (i, j) is defined as
Therefore, the interference of the link (k, j) at frequency f and time t to link (i, j) is
B. Problem Model
In order to reduce inter-link interference and maximize energy utilization, the problem of cross-layer resource management can be constructed as a stochastic game model, which is defined as [TeX:] $$M=(\mathcal{N}, \mathcal{X}, \mathcal{A}, \mathcal{T})$$:
· [TeX:] $$\mathcal{N}$$: The set of players, [TeX:] $$\mathcal{N}=\{1,2, \cdots, N\}$$, where N represents the number of players in the game. It is defined as [TeX:] $$\mathcal{N}=S$$, which refers to the set of transmitting nodes.
· [TeX:] $$\mathcal{A}$$: Action set of players, including UAV power selection, spectrum allocation, and next-hop node selection.
· [TeX:] $$\mathcal{T}$$: The cost function combining with physical layer power resources, MAC layer frequency resources, and network layer routing resources.
In order to maximize the utilization of local and entire network resources without significantly increasing routing overhead, the cost function on link (i, j) is defined as
where [TeX:] $$c_{i, j}^{f}(t)$$ is the energy efficiency on link (i, j), the higher the efficiency of the energy on the link, the smaller the loss on the link, which is more suitable for data transmission than other links. [TeX:] $$N_{i}(f)$$ is background noise. [TeX:] $$Q_{i}(t)$$ and [TeX:] $$Q_{j}(t)$$ are the length of packet queue at nodes i and j, respectively. The greater the difference between the packet queues on node i and node j, the more stable the link, otherwise it will not be selected for data transmission. [TeX:] $$v_{i, j}^{t}$$is the relative speed of nodes i and j at time t. The smaller the relative speed between nodes, the more stable the link for data transmission. In general, the smaller the value of the cost function, the more stable the link and the more suitable the link for data transmission.
Supposing that [TeX:] $$K_{i}$$ is the number of relays passed by player [TeX:] $$S_{i}$$ in the transmission path, and the cost of player [TeX:] $$S_{i}$$ can be defined as
Decomposition of the original optimization problem.
where [TeX:] $$j=0 \text { and } j=K_{i}+1$$ represent player j itself and the receiving node corresponding to player j, respectively. When the distance from node j to the receiving node j + 1 is closer than other receiving nodes, node j + 1 is a feasible next hop node for node j. And the set of feasible next hop nodes of node e j is defined as [TeX:] $$\operatorname{next}(j)$$.
Then the cross-layer resource management problem can be described as
where objective function (12a) is to minimize the cost of node i. The first condition (12b) constrains the frequency of UAV communication. The second constraint (12c) takes into account the limited transmitting power of UAVs. Whereas the third constraint (12d) means that node j + 1 is in the set of j's next hop, and node j can communicate with it in one hop. The last constraint (12e) indicates that the packet can be received correctly only if the signal to interference plus noise ratio (SINR) at the receiving node j is greater than the threshold value [TeX:] $$\gamma^{t h}$$.
IV. SOLUTION TO THE CROSS-LAYER RESOURCE MANAGEMENT PROBLEM
The cross-layer cost function in (8) and cross-layer resource management problem in (12) are shown in the previous section. However, it is difficult to solve them directly because of the complexity of the original optimization problem [49]. Meanwhile, the UAV resource management and the choice of the path are relatively independent in origin problem (12). Thus, it can be decomposed into the link energy utility maximization subproblem and relay selection subproblem.
Power and frequency are two variables that need to be optimized in the energy utility maximization subproblem. Frequency scheduling is a fast process that executes at the beginning of a time slot ranging from time 0 to time t. In contrast, power control is a slow process. Since frequency scheduling and power control are two separate processes, the energy utility maximization subproblem can be further decomposed into a power control subproblem and a frequency scheduling subproblem. Fig. 2 shows the decomposition of the original problem. The brief solutions to these subproblems are as follows:
1) Energy utility maximization subproblem: This problem can be solved by power control and frequency scheduling algorithms under the condition of a given link (i, j).
2) Power control subproblem: Given the frequency [TeX:] $$f \in F$$, the power control algorithm is designed by MFG theory.
3) Frequency scheduling subproblem: The optimal frequency scheduling scheme, [TeX:] $$f^{*}=\arg \max c_{i, j}^{f}$$ , canbe obtained by the power control algorithm under the condition of any frequency [TeX:] $$f \in F$$.
4) Relay selection subproblem: It can be transformed into a problem that maximizes [TeX:] $$\left[Q_{i}(t)-Q_{j}(t)\right]^{+} e^{-v_{i, j}^{t} \beta}$$. Any node [TeX:] $$j \in \operatorname{next}(i)$$ can get the optimal relay node [TeX:] $$j^{*}$$ by executing the energy utility maximization algorithm and calculating arg min [TeX:] $$T_{i, j}$$.
A. Power Control Solution
In the MFG model of power control, the players are a set of sending nodes. The action space includes all feasible power resources. The state space is defined as the node energy state space and the interference state space of the node. The utility function is as (11).
Firstly, the value function is defined as
where [TeX:] $$c_{i, j}^{f}(T)$$ is the utility function at the final time T. According to the Bellman optimality principle, an optimal control strategy should be satisfied: the remaining decision must constitute the optimal strategy for the state of the previous decision-making regardless of the state and decision of the past.
For any link (i, j), the optimal power scheme [TeX:] $$p_{i}^{*}(t \rightarrow T)$$ can make equation in (14) true at any time [TeX:] $$t \in(0, T)$$
Secondly, the mean field function is defined as
where N is the number of players, which is huge and even goes to infinity. I is a Boolean indicator function. [TeX:] $$x=(e, \mu)$$ is the system state of MFG, where e and [TeX:] $$\mu$$ are energy state and interference state, respectively.
According to MFG theory, the MFG system consists of Hamilton-Jacobi-Bellman (HJB) equation and Fokker-Planck-Kolmogorov (FPK) equation. The value function satisfies the HJB equation and the mean field function satisfies the FPK equation. The MFG equilibrium solution can be obtained by solving related equations.
The HJB equation and the FPK equation of the MFG system are further derived. Assuming that dt is an infinitesimal amount of time, (16) at time [TeX:] $$t \rightarrow t+d t$$ can be obtained according to Bellman's optimality principle.
where [TeX:] $$\sigma=\left(0, \sigma_{\mu}\right)$$ and [TeX:] $$B_{t}=\left(0, B_{t}^{\mu}\right)$$.
(17) can be obtained by expanding in (16) through the Taylor formula and Ito's rule.
where [TeX:] $$d e=-p\left(t_{0}, x_{0}\right) d t$$ and [TeX:] $$d \mu=\nabla_{t} \mu\left(t_{0}, x_{0}\right) d t$$. When the higher order infinity is ignored, (18) can be obtained by taking the expectation of both sides of (17),
Since the expectation of Brownian motion [TeX:] $$d B_{t}^{\mu}$$ is zero at time dt, (18) can be simplified into (19),
where [TeX:] $$\Delta_{\mu} u\left(t_{0}, x_{0}\right)=\frac{\partial^{2} u\left(t_{0}, x_{0}\right)}{\partial \mu^{2}}$$. Thus, (20) can be arranged as
The following (21) can be obtained by sorting (20).
The general form of the HJB equation can be expressed as
where [TeX:] $$H\left(c, p, \nabla_{x} u(t, x)\right)$$ is the Hamiltonian term.
The FPK equation can be derived by using the test function method. The detailed steps can be referenced in [50]. The derived FPK is
The HJB equation in (22) and the FPK equation in (23) have been deduced. Next, the mean field equilibrium (MFE) needs to be achieved. If the objective function of the optimal control problem is convex, the convergence point is MFE. When solving MFE, the finite difference method can be used. This method essentially achieves MFE through iterative steps.
The iteration step size in terms of time, energy and interference is defined as
By discretizing the time axis [TeX:] $$[0, T]$$, energy state space [TeX:] $$\left[0, E^{\max }\right]$$, interference state space [TeX:] $$\left[0, I^{\max }\right]$$ in space [TeX:] $$X \times Y \times Z$$, and iterating the MFG system equation according to the iteration step size, the MFE can be solved.
In this paper, the Lax-Friedrichs method is used to solve the FPK equation [50]. In this method, the following conditions are assumed: [TeX:] $$f(t, x)=f_{i}^{j}$$, where [TeX:] $$t=j \delta_{t}$$, and [TeX:] $$x=i \delta_{x}$$. The operator of Lax-Friedrichs can be achieved as
The mean field iteration in (28) can be obtained by substituting the above operator into the FPK equation as
SYMBOLS AND NOTATIONS USED IN DERIVATION.
Due to the existence of the Hamilton term, the finite difference algorithm cannot be used to solve the HJB equation directly. Therefore, the HJB equation is reconstructed to its corresponding optimal control problem, the newly defined problem is
The Lagrangian function corresponding to the new problem is obtained by introducing a Lagrangian multiplier.
After discretizing (30), the Lagrangian multiplier iteration in (31) is obtained by finding the solution [TeX:] $$\frac{\partial L_{d}}{\partial m_{i, j}^{t}}=0$$,
The above equation is backward. If [TeX:] $$\lambda_{i, j}^{I}$$is given, the Lagrange multiplier can be updated iteratively through the above equation.
At any time t, for any pair of source node i and destination node j, the following equation can be obtained by seeking the solution of [TeX:] $$\frac{\partial L_{d}}{\partial p_{i, j}^{t}}=0,$$
Our model is a convex game model, combined with convex programming theory to ensure that the problem model is a convex problem. So, the equilibrium solution must exist and is the optimal solution.
According to the above formula, the mean field [TeX:] $$m_{i, j}^{t+1}$$, Lagrangian parameter[TeX:] $$\lambda_{i, j}^{t-1}$$ and power level [TeX:] $$p_{i, j}^{t}$$ can be iteratively updated to obtain the optimal power value. The power control algorithm is designed as shown in Algorithm 1 and the symbols and notations used in the above derivation are summarized in Table III.
B. Frequency Scheduling and Relay Selection
In Algorithm 1, given the link information (i, j), frequency [TeX:] $$f$$, and time information [0, T], it is necessary to initialize the mean field distribution [TeX:] $$m_{:,:}^{0}$$ Lagrange multiplier [TeX:] $$\lambda_{:,:}^{I}$$, and power level [TeX:] $$p_{:,:}^{0}$$, and iteratively update the mean field, Lagrange multiplier, and power in the time axis X, energy state space Y , and interference state space Z. Mean field is obtained by solving FPK equation, and value function is obtained by solving HJB equation. When the algorithm reaches the convergence condition, the optimal power control strategy [TeX:] $$p_{i, j}^{*}(f)$$ can be obtained.
The frequency scheduling algorithm needs to maximize the utility function [TeX:] $$c_{i, j}^{f}$$. Since the optimized value of the utility function in power has been solved by Algorithm 1. The maximized value of which can be obtained by traversing the available frequency f over a given link. Therefore, the frequency scheduling algorithm can be designed as shown in Algorithm 2. First of all, we give the link information (i, j) and initialize [TeX:] $$f^{*} \text { and } c_{i, j}^{*} \text { to } 0$$. Then, it calculates the utility function by traversing the frequencies at the available frequencies. By
Frequency scheduling algorithm
comparing the utility function, the optimal frequency [TeX:] $$f^{*}$$ and the maximized value [TeX:] $$c_{i, j}^{*}$$
The relay selection needs to consider the cost function of the link [TeX:] $$T_{i, j}^{f}$$. The smaller cost in a link, the better performance and stability of information transmission. When minimizing the cost function, the link utility function [TeX:] $$c_{i, j}^{f}$$ can be obtained by the frequency scheduling algorithm, and the packet queue length and relative speed can be calculated locally. Therefore, the relay selection algorithm can be designed as shown in Algorithm 3. It calculates the cost function of all next hop nodes in next(i) which node i can be reached, by comparing the value of the cost function, node [TeX:] $$j^{*}$$ corresponding to the optimal link quality is obtained. Then, the data in node i will be transmitted to it.
In summary, the cross-layer resource management algorithm TONG LI et al.:A MEAN FIELD GAME-THEORETIC CROSS-LAYER OPTIMIZATION ...
Relay selection algorithm
consists of three parts as shown in Fig. 3 with a power control algorithm, a frequency scheduling algorithm, and a relay selection algorithm. The relay selection algorithm completes the selection of the relay node by calling the power control algorithm and the frequency scheduling algorithm. Correspondingly, the power control algorithm and the frequency scheduling algorithm provide services for the relay selection algorithm. These three algorithms are used together to solve the cross-layer resource management problem in swarm FANET.
C. CLRM-DSR Protocol
The DSR protocol is a typical MANET routing protocol [7]. This protocol does not need to broadcast routing information periodically to maintain routing tables, the routing overhead in the DSR protocol is less than that in the OLSR and other routing protocols [45]. As two mechanisms of DSR, route lookup and route maintenance ensure that every node can construct an available route in the network. However, since there is a route lookup process before the packet is sent, the end-to-end delay in the DSR increases. More importantly, the nodes in the DSR protocol don't exchange link quality information. When the network topology and links change, the performance of the FANET network using the DSR protocol is unstable, especially in terms of data packet transmission rate [51].
In the FANET, the nodes move at a high speed so that the routing information found by using the DSR protocol is likely to expire [52]. The link quality between nodes can ensure the stability of the links and routing information when nodes are looking for routes. Therefore, as shown in Fig. 4, link quality is employed in the route response message, which refers to the optimal cost function obtained by the cross-layer resource management algorithm. To reduce the routing overhead, the format of the routing request message in the CLRM-DSR protocol leaves it as it is in the DSR protocol.
The CLRM-DSR routing protocol will send a request when there is a need for information transmission without periodically broadcasting routing information. When a node needs to send data, the source node finds whether the routing infor-
Cross-layer resource management algorithm.
Routing packet format in CLRM-DSR.
mation of the destination node information exists locally, if it exists, the node can send the information directly, otherwise it needs to send a routing request packet 3(a) to perform the route discovery process to find the path to the destination node.
When the destination node receives a routing request packet, it sends a route response message in the unicast mode, the intermediate node calls the cross-layer resource management algorithm to obtain the link quality metric [TeX:] $$T_{i, *}$$. The link quality field in the packet will be replaced by the new value after adding [TeX:] $$T_{i, *}$$. When the source node receives the route response packet, it can obtain the link quality metric to other nodes [TeX:] $$T_{i, *}$$. Since on the transmission path, [TeX:] $$T_{i, *}$$ is updated according to the state of the link, the link corresponding to the smallest [TeX:] $$T_{i, *}$$ obtained at the source node is the most stable link, which is the optimal link for data transmission.
V. EXPERIMENTS AND RESULTS
In this paper, the CLRM-DSR protocol based on the proposed cross-layer resource management algorithm is simulated and compared with the conventional DSR protocol. A specific FANET scenario is considered which is formed by UAVs in a space of [TeX:] $$3 \mathrm{~km} * 3 \mathrm{~km} * 3 \mathrm{~km}$$, and the number of UAVs ranges from 10 to 60, of which 3-18 nodes send constant bit rate (CBR) to the corresponding destination nodes [42]. To reach the equilibrium state quickly, the initial power is set to half of the maximum power. Ultra-dense deployment and
Simulation node spatial distribution scenario.
three-dimensional distribution are the characteristics of ultra-dense FANET driven by UAV clusters. In the simulation, the FANET simulation scenario with node spatial distribution is adopted as shown in Fig. 5. Detailed simulation parameters are listed in Table IV.
To demonstrate the performance of the algorithm, the packet delivery rate, end-to-end delay, and routing overhead are mainly considered. The performance of the CLRM-DSR and conventional DSR protocol are compared by different speeds and numbers of UAVs based on the statistics, which are shown separately in Section V-B.
A. Simulation Platform
In this paper, a FANET simulation platform is built. In this platform, the object-oriented method is used to model nodes in MATLAB. As shown in Fig. 6, the node model contains node-related attribute parameters, node movement model module, a complete protocol stack, data packet receiving and sending module, and information statistics module. Nodes interact through event response or command to simulate the operation of the FANET. This node model can complete simulation tests, including the interface of network layer routing protocol, MAC layer access protocol interface, MAC layer frequency scheduling, and physical layer power control.
Node model in simulation platform.
Data acquisition from IoTs to MEC server using a UAV.
1) Network layer routing protocol interface: The routing protocol is one of the critical issues of the FANET, which has a significant impact on network stability and performance [53]. Thus, the simulation platform provides a unified routing protocol initialization interface in the network layer, which can support the development and performance verification of various routing protocols.
2) MAC layer access protocol interface: The MAC layer access protocol is another essential issue in the FANET. Like the routing protocol interface, the simulation platform provides a unified initialization interface of MAC layer access protocol, which can support the development and performance verification of various MAC layer access protocols.
3) MAC layer frequency scheduling: The frequency resources of UAVs are limited. It is essential to schedule the frequency resources properly and improve resources utilization to achieve better network performance. The simulation platform builds a complete FANET environment. Based on this platform, the MAC scheduling scheme can be implemented, and the performance can be verified.
4) Physical layer power control: As UAVs become minia-turized, intelligent, and clustered, which makes the problem of energy limitation more prominent. It is necessary to design an efficient power control algorithm to improve energy utilization efficiency. In this simulation platform, the performance of the proposed power control algorithm can be verified.
Fig. 7 demonstrates the running process of the simulation platform. The platform will complete the parameters configuration of the FANET, after it is started. Subsequently, the discrete event simulation process completes the simulation of the FANET operation by simulating the processing of services in the network, including the packet receiving process, packet sending process, etc. Finally, the statistics process completesThe statistics process completes the statistics of network performance indicators, which is convenient for evaluating the proposed algorithm.
In this paper, the proposed cross-layer resource management algorithm and the CLRM-DSR protocol in FANET are verified in this platform. Meanwhile, this simulation platform can also implement the algorithm verification of the conventional mobile ad-hoc network and the vehicle network.
B. Simulation Results
In the actual simulation, all nodes are traversed in a simulation interval to achieve the purpose of processing the data services of all network nodes. Furthermore, more stable indicators statistics of performance are achieved through traversal many times.
1) Performance of CLRM-DSR and DSR protocol about different speeds of UAVs: In this section, the impact of UAV speeds on the performance of two routing protocols is analyzed. In this scenario, the number of UAVs is 30, the number of communication node pairs is 8. The packet delivery rate, end-to-end delay, and routing overhead under different UAV speeds (60 m/s, 70 m/s, 80 m/s, 90 m/s, and 100 m/s [3]) are shown in Fig. 8.
Fig. 8(a) shows the packet delivery rate of the DSR and the CLRM-DSR protocol with different speeds of UAVs. As illustrated in Fig. 8(a), the packet delivery rate of both protocols shows a downward trend with the increasing speeds of UAVs. When the speed of UAVs is 60 m/s, the difference between the packet delivery rate of the CLRM-DSR and the DSR protocol is 3%. However, when the speed of UAVs is 100 m/s, the difference is 6%. As the speed of UAVs increases, the change of network topology is intensified, and the communication links between the nodes are broken, which will lead to a decrease in packet delivery rate. UAVs have a wide range of speed variations in practical applications. The CLRM-DSR protocol performs better packet delivery rate performance than the DSR protocol as shown in Fig. 8(a). The main reason is that the CLRM-DSR protocol introduces the results of cross-layer resource management policy and uses link quality to select stable links, which reduces repeated routing request packets caused by frequent breaking links. In general, CLRM-DSR weakens the impact of UAVs speeds on the packet delivery rate and improves the packet delivery rate.
Fig. 8(b) indicates the end-to-end delay of the DSR and the CLRM-DSR protocol with different UAVs speeds. As illustrated in Fig. 8(b), the end-to-end delay of both protocols shows an upward trend with the increasing speeds of UAVs. When the speed of UAVs is 60 m/s, the difference between the end-to-end delay of the CLRM-DSR and the DSR protocol is 210 ms. However, when the speed of UAVs is 100 m/s,
Network performance about different speeds of UAVs.
the difference is 280 ms. The UAV with high mobility causes drastic changes in the network topology, and results in unstable links, which makes the DSR protocol cache or even the newly created routing information outdated. The possibility of data retransmission will increase. To correctly transmit data, the route search process needs to be reinitialized, and the time it takes by finding routes frequently increases the end-to-end delay. However, the CLRM-DSR protocol selects stable paths based on link quality, which reduces the probability of route errors and the extra delay caused by the route lookup process. In general, the end-to-end delay in the CLRM-DSR protocol is lower than that in the DSR protocol, which weakens the impact of UAV speeds on the end-to-end delay and reduces the end-to-end delay.
Fig. 8(c) depicts the routing overhead of the DSR and the CLRM-DSR protocol with different UAVs speeds. As shown in Fig. 8(c), the routing overhead of both protocols shows an upward trend with the increasing speeds of UAVs. When the speed of UAVs is 60 m/s, the difference between the routing overhead of the CLRM-DSR and the DSR protocol is 600 kb. However, when the speed of UAVs is 100 m/s, the difference is 1600 kb. With the increase of UAVs speed, the frequently broken links lead to a large amount of wrong routing information. As a typical on-demand routing protocol, when the data need to be transmitted, the route lookup process in the DSR protocol should be reinitialized. This process brings several control packets, including routing request packets and routing reply packets, which results in a sharp increase in routing overhead. Limited by the network capacity, the routing overhead will tend to be flat at last. Since the CLRM-DSR protocol reduces the impact of node mobility on the communication path by link quality, the number of control packets transmitted in the network is reduced. Therefore, the CLRM-DSR protocol is more effective than DSR protocol in routing overhead, which weakens the impact of UAVs speeds on the routing overhead and reduces the routing overhead.
2) Performance of CLRM-DSR and DSR with different numbers of UAVs: This section mainly analyzes the impact of UAV numbers on the performance of the two routing protocols. In this scenario, the UAVs move at a speed of 100 m/s. The packet delivery rate, end-to-end delay, and routing overhead under the different numbers of UAVs (10, 20, 30, 40, and 50) are demonstrated in Fig. 9.
Fig. 9(a) shows the packet delivery rate of the DSR protocol and the CLRM-DSR protocol with different numbers of UAVs. The UAVs are sparsely distributed, when the number of UAVs is relatively small. In this scenario, the path between nodes is few or even doesn't exist. At this time, the performance is no difference between the CLRM-DSR protocol and the DSR protocol. Therefore, when the number of UAVs is 10, the delivery rates of the two protocols are the same. The connectivity probability between UAVs increases with the increasing number of UAVs. The UAV with CLRM-DSR protocol selects the path according to the link quality, which ensured that the selected link is more stable than that selected by DSR. Thus, it can achieve better performance than the conventional DSR protocol in the packet delivery rate. For the DSR protocol, when the number of UAVs increases, the interference and collisions in the network become more serious. For instance, when the number of UAVs exceeds 20, the packet delivery rate decreases significantly. For the CLRM-DSR protocol, when the number of UAVs is not particularly large, the performance gain brought by the presented protocol is greater than the impact of the increase in network scale.
Fig. 9(b) depicts the end-to-end delay of the DSR protocol and the CLRM-DSR protocol with different numbers of UAVs. When the number of UAVs is less, some nodes in the network
Network performance with regard to different numbers of UAVs.
may be unconnected or have only one available path. In order to send data, a lot of routing overhead is generated in the network. Thus, communication under two routing protocols has a high end-to-end delay. Overall, the CLRM-DSR protocol has an advantage over the DSR in end-to-end delay. It is because that the number of paths between UAVs increases as the number of UAVs increases. For the DSR protocol, it needs to continually re-route because of the frequently disconnected communication links. The CLRM-DSR protocol can alleviate the frequent disconnection of connections and avoid the extra delay caused by re-routing. When the number of UAVs exceeds 40, the interference and conflicts in the system are severe, and the delay of both protocols will increase.
Fig. 9(c) illustrates the routing overhead of the DSR protocol and the CLRM-DSR protocol with different numbers of UAVs. When the number of UAVs is too less, the links between the UAVs are unstable. The source UAVs need to continuously perform the routing request process to send data, which leads to a large amount of routing overhead. With the increasing number of UAVs, the number of routing request packets is reduced. However, the UAVs still need to continually find the right routes due to the frequently disconnected communication links. Result from the mechanism of routing based on link quality, the selected path can be more robust in the CLRM-DSR protocol. The network with the CLRM-DSR protocol will lead to less routing overhead than the DSR protocol.
Through the simulation comparison of CLRM-DSR and DSR in the designed simulation platform, a conclusion can be obtained that the CLRM-DSR protocol has improved compared with the conventional DSR protocol in packet delivery rate, end-to-end delay, and routing overhead. Besides, the cross-layer resource management algorithm using the MFG has a gain effect in swarm UAV communications.
VI. CONCLUSION
In the paper, we study the cross-layer design method and MFG theory in large-scale FANET. The cross-layer design method breaks the communication limitations between layers. Each layer can share local information through inter-layer interaction, which can effectively improve the efficiency of resource utilization of FANET. The MFG provides a feasible analysis, modeling and solution method for large-scale network power control. We jointly consider physical layer power resources, MAC layer frequency resources, and network layer routing resources, and proposes a cross-layer resource management method. Meanwhile, we present a routing protocol named CLRM-DSR based on cross-layer optimization with MFG for multi-hop swarm UAV networks, which adds link quality measurement. The simulation results prove the advantage of CLRM-DSR used MFG in swarm UAV networks. Last but not least, the paper studies the homogeneous network communication scenario composed of UAVs. Under the requirements of the future space-air-ground integrated networking, various networks will coordinate with each other to complete tasks, and the network performance of the proposed algorithm applied in heterogeneous networks is unknown. The study of cross-layer design and the application of MFG in different routing protocols of a heterogeneous network is one of our future works. At the same time, MAC layer protocol design, path planning, node rate control, congestion control, and other aspects also have a great impact on FANET. For further research, we intend to study the above issues in depth.