TMCnet News

A FSA Model for Performance Evaluation over Mobile Ad Hoc Networks with Low Traffic Volume [Sensors & Transducers (Canada)]
[August 15, 2014]

A FSA Model for Performance Evaluation over Mobile Ad Hoc Networks with Low Traffic Volume [Sensors & Transducers (Canada)]


(Sensors & Transducers (Canada) Via Acquire Media NewsEdge) Abstract: A Mobile Ad hoc network (MANET), which is composed of nodes that are able to move arbitrarily, differs from a direct connection network in the way that it is multi-hopping and self-organizing and thus able to operate without the help of prefixed infrastructures. However, problems such as unfavorable wireless links and dynamic topology are challenging, resulting in the proposal of a collection of routing protocols for MANETs. Nevertheless the performance of protocols may deteriorate dramatically as deployment scenario changes due to the application dependent philosophy behind algorithms. In this paper, the performance evaluation problem for MANETs and is explored and a novel performance ranking model, termed FSA, is proposed. For simplicity but without loss of generality, the performance of two routing protocols DSDV and DSR are studies. The FSA is able to rank the performance of DSDV and DSR depending on the average value and standard deviation results. Extensive simulations show that an overall 20.74 %, at most, gain may be achieved based on the FSA model. Copyright © 2014 IFSA Publishing, S. L.



Keywords: Mobile ad hoc network, Performance evaluation, FSA, Routing, Fuzzy logic.

(ProQuest: ... denotes formulae omitted.) 1. Introduction In the past decades, mobile traffic generated by devices such as smartphones, iphones, laptops and mobile gateways has been growing rapidly. While traditional direct connection techniques evolve to provide better access to the Internet, a new type of wireless network, mobile ad hoc network (MANET), has emerged. A MANET [1] differs from a direct connection network in the way that it is multihopping and self-organizing and thus able to operate without the help of prefixed infrastructures. However, challenges such dynamic topology, unreliable wireless links and resource constraints impede the wide applications of MANETs.


Routing, because of its importance, has always been the research focus since the introduction of MANETs, is complex because it has to react efficiently to unfavorable conditions and support traditional IP services. Tremendous efforts have thus been devoted to the design of routing in MANETs, leading to the emergence of protocols such as DSDV [2], DSR [3], AODV [4] and OLSR [5] to improve performance in terms of delay, throughput et al.

However, the application independent nature of routing protocols, as shown in Table 1, results in the absence of a one-for-all solution for MANETs. As seen in Table 1, the performance of the same protocol (e.g., AODV and DSR) varies because of the change of node pause time, mobility speed and traffic volume. Consequently, performance evaluation become vital since it provides the benchmark via which the network operator is capable of selecting the optimal protocol adaptively.

Generally speaking, there are three methods to evaluate the performance of a given routing protocol, namely practical implementation [6], mathematical derivation [7] and simulation [8]. Results achieved by practical implementation are credible but they are scenario related and can not be repeated. Mathematical derivation is comprehensive, but it is complicated and assumptions in the mathematical model degrade the credibility. Simulation offers the ability to evaluate multiple systems in a number of scenarios in a repeatable manner. However, just as with mathematical modeling, modeling assumptions may decrease the credibility of the results.

A combined model, termed FSA (Fuzzified Simple Additive Weight [9] - Analytic Hierarchy Process [10]), which depends on simulation and mathetical methods, is employed in this paper to rank alternative routing protocols by considering the relative importance of multiple performance metrics is proposed in this paper. The standard deviation is considered especially by FSA.

This paper is organized as follows. Section 2 outlines the simulation background and states the problem. The third part introduces and validates the FSA model via extensive simulations and the final section concludes this paper.

2. Simulation and Problem Statement 2.1. Simulator To date, a number of simulation tools (e.g., NS-2 [14], GloMoSim [15], OPNET [16], QualNet [17] and MATLAB [18]) have been developed for mobile ad hoc network simulations among which NS2 2 a sound one. In addition to the flexibility and convenience, the open source property also contributes to the success of NS-2. The role for NS-2 is so important in the research community of mobile ad hoc networks that it has become the de-facto reference simulator [19]. Since only a small network (32 nodes) is simulated in this thesis, the problem of scalability for NS-2 can be ignored. Therefore NS-2 is adopted in this paper.

2.2. Simulation Configurations For simplicity but without loss of generality, DSDV, a typical proactive routing protocol, and DSR, a typical reactive routing protocol, are selected as two alternative protocols for comparisons. In this way, the efficiency of the proposed adaptive algorithm can be observed clearly. However, the results can be applied to other cases directly. The performance of a typical mobile ad hoc network in the university campus where several laptops/mobile phones share a common access point to access the Internet, as shown in Fig. 1, is investigated as an example for the FSA model. The simulation configurations and results are itemized in Table 2 and Table 3 respectively. It lasts for 3000 s for each round to avoid the initialization bias. 50 independent simulation runs are averaged to obtain the final results.

2.3. Simulation Results and Analysis As seen in Table 3, DSR behaves better in terms of packet delivery ratio and energy cost in all three cases due to its on-demand nature, which avoids the use of stale routes as well as periodic routing information broadcast. On the contrary, DSDV outperforms DSDV in delay, jitter and throughput in all cases. The key reason is its proactive philosophy. DSDV is able to establish route much more quickly by searching routing table in the cache which is updated periodically. Instead, DSR initiates a route discovery process when necessary which consumes more time.

2.4. Problem Statement For a network operator who has time sensitive applications, DSR is better a solution compared to DSDV on average. On the contrary, for reliable packet delivery service, DSDV is preferred. However, when the standard deviation is considered, the network operator may be reluctant to decide which one is better.

A sound solution, it is suggested, is to develop a performance evaluation method through which the operator may choose the optimal protocol dynamically for specific application scenarios. Meanwhile, the performance evaluation model should consider the standard deviation simultaneously.

3. FSA Model The proposed FSA model involves three steps and the first one is to decompose the evaluation problem into a hierarchy structure, composed of an objective layer, a criteria layer and an alternative layer so that a hard problem can be more easily understandable. One tiling to note is that results in Table 3 are used in this section to compute the weights for alternatives.

3.1. Hierarchy Structure The objective in this paper is to evaluate the performance of DSR and DSDV in MANETs with several performance metrics considered and rank them accordingly given the operator's preference of performance metrics which are regarded as criteria of a network operator. Fig. 2 shows the hierarchy structure with three layers, the objective layer, criteria layer and alternative layer.

3.2. Weight for Metrics and Alternative Protocols The following step is to compute weights for both metrics and alternative protocols.

3.2.1. Weight for Metrics A decision maker is assumed to be able to compare any two elements, say Ei and £}, at the same level of the hierarchy structure and provide a numerical value eij according to his/her preference, eij > 0 for any i=l,2,...,n and j=l,2,...,n. The reciprocal property e/,=l/e,y holds. The rules for pair-wise comparison are listed in Table 4.

Several assumptions are made in this paper for the relative importance of criteria in this paper. They are as follows: (I) Packet delivery ratio is moderately more important than delay; (II) Packet delivery ratio is moderately more important than jitter; (III) Packet delivery ratio and throughput are equally important; (IV) Packet delivery ratio is moderately more important than energy cost; (V) Delay and jitter are equally important; (VI) Delay and energy cost are equally important; (VII) Jitter and energy cost are equally important; (VIII) Throughput is moderately more important than delay;(IX) Throughput is moderately more important than jitter; (X) Throughput is moderately more important than energy cost.

One thing to note is that these parameters are application dependent and the choices here are for a specific application scenario.

According to Table 4, the above 10 assumptions lead to the comparison matrix for criteria as follows ...(1) where PDR, Thrput and EC denote packet delivery ratio, throughput and energy cost respectively.

There are several methods to derive weights from a comparison matrix of which geometric mean method (GMM) is a straight forward and reliable altemative[20]. In GMM, the normalized weight is computed firstly via ...(2) where ay (i,j=\,2,...,n) denotes the value of ifth elements in comparison Matrix (1) and n is number of elements in the row.

Combining (2) with Matrix (1), the normalized weights for criteria are obtained in Table 5.

As observed, the weights for packet delivery ratio and throughput are equal, indicating the same importance of those two metrics. Delay, jitter and energy cost have the same weight which accounts for one third of that for packet delivery ratio, revealing that they are less important compared to packet delivery ratio. Qualitatively, a protocol that has a better performance in terms of packet delivery ratio and throughput is more likely to be selected.

A decision maker may give inconsistent judgments for the comparison matrix and therefore FSA is designed with capability of measuring the consistency based on the idea of cardinal transitivity. A matrix M is consistent if and only if aikxaiy= ay, where ay is the ij'th element of the Matrix (1). However, this condition can rarely be satisfied in practice, especially in scenarios with a large number of criteria or alternatives. The violation level of consistency changes with person or context. In FSA, a metric Consistency Ratio (C.R.), developed by Satty [10], is employed to indicate the extent to which the consistency is violated as follows ...

where C and coi denote the pair-wise comparison matrix and weight for the Vth element respectively, n represents the number of elements and R.I. is the random index of a pair-wise comparison matrix that depends on the number of elements in the matrix as itemized in Table 6. As long as C.R.<0.1, the matrix is believed to be consistent [10]. The C. R. of Matrix (1) equals 0 indicating that Matrix (1) is consistent.

3.2.2. Weight for Alternative Protocols 1) Principles of fuzzy numbers.

Prior to deriving synthetic weights for alternatives, some principles regarding fuzzy numbers are introduced for future usage. A fuzzy number M on R is defined to be a triangular fuzzy number if its membership function pm (x) has the following characteristics: ...

where /, m, and u denote the lower, middle and upper bounds of a triangular fuzzy number respectively, as shown in Fig. 3.

The triangular fuzzy number can be expressed also by (1, m, u), and when 1 = m = u, it is a crisp number by convention. Consider two triangular fuzzy numbers Ml and M2 where Ml= (11, ml, ul) and M2= (12, m2, u2), the operation laws are as follows.

...

2) Reciprocal relationship.

For the element eij, the reciprocal relationship holds ...

where eij=(lj, nuj, Uij), representing the fuzzy comparison results of the i'th alternative over the fth alternatives.

3) Fuzzy pair-wise comparison matrices for alternatives.

The fuzzy comparison matrices are constructed based on the attributes of metrics. For "the larger the better" metrics such as packet delivery ratio and throughput, the pair-wise comparison value, %*, for the j'th alternative over the k'th alternative under the i'th metric is given by ...(4) where (I) ßj denotes the average performance of the j'th alternative under the inh metric; (II) a.ij=ßj -A, A denotes corresponding standard deviation; (III) yij=ßj +A; (IV) a,(max)=max{a,7, ai2,..., aim} and m denotes the number of alternatives; (V) /?,(max)=max{/?,;, ßi2,...,ßim}, y,(max)=max{yii, yi2,...,yim}.

Similarly, for parameters that are "the smaller the better", the pair-wise comparison value aijk becomes ...(5) where a,(min)=min{a,7, ai2,..., aim}, /?,(nhn)=min{/?,7, ßi2,...,ßim} and y/(min)=min{yü, yi2,..., yim}.

Packet delivery ratio is classified to be a "the larger the better" metric and thereby the fuzzy comparison matrix for alternatives is given by ...

Unlike packet delivery ratio, delay belongs to "the smaller the better" class and hence the fuzzy comparison matrix for DSDV and DSR, under delay, is ...

Similar to delay, the fuzzy comparison matrix for jitter becomes ...

Unlike delay and jitter, throughput is a "the larger the better" parameter and the fuzzy comparison matrix for DSDV and DSR, under throughput, is ...

Energy cost is a "the smaller the better" metric and hence formula (5) is used to obtain the fuzzy comparison matrix for alternatives.

...

4) Fuzzy geometric mean method (FGMM).

In the geometric mean method, elements in each row are multiplied and normalized. Similarly, the normalized weights in FGMM [20] are computed via ...(6) Table 7 itemizes fiizzy weights for packet delivery ratio, delay, jitter, throughput and energy cost, using FGMM. As seen, the values in the middle of the intervals are identical to those generated by geometric mean method. However, weights for DSDV and DSR overlap with each other.

Fuzzy weights in Table 7 are aggregated by ...(7) Fig. 4 presents synthetic weights for DSDV and DSR for cases of 2 streams. As seen, the synthetic weight for DSR overlaps with that of DSDV's. The next step is to determine which weight is larger. Optimist considers DSR to be a better solution since "DSR-2" could be larger than "DSDV-2" while pessimist regards DSR worse than DSDV due to the reason that "DSR-2" could be smaller than "DSDV-2".

5) FSA model.

According to Mikhailov [21], the weights for metrics and alternatives can be obtained by solving a linear program ...(8) where di and d2 denote tolerance parameters, 2 symbolizes the consistency index and m/a) and lj(a) are lower and upper bounds of a-cut intervals. It is suggested by Mikhailov that di = ¿fe =1. If 2>1, the comparisons are considered consistent.

...(9) ...(10) where aij=( lj, mih uii).

As shown in (9) and (10), the weights obtained from (8) depend on the value of a and thus they are considered to be a function of a. Mikhailov aggregates weights via ...(11) where L denotes number of a values, al represents the lnh value for a and coi(a) is the weight for a specific value of a.

The a dependent weights for DSR and DSDV under the metric packet delivery ratio, energy, delay, jitter and throughput are thus can be are obtained using (11) as shown in Fig. 5 to Fig. 9 where DSDVagg and DSR-agg. Denote aggregation weights for DSDV and DSR respectively.

As shown in Fig. 5 and Fig. 6, the a dependent weight for DSR increases with the increase of a in terms of packet delivery ratio and energy cost and the aggregated weight for DSR exceeds that of DSDV.

On the contrary, the weights for DSR under the metrics delay, jitter as well as throughput decrease with the increase of a in Fig. 7 to Fig. 9. The aggregated weights for DSR under the above three metrics are smaller than that of DSDV, indicating that DSDV outperforms DSR in delay, jitter and throughput.

These weights for DSDV and DSR are synthesized, and shown in Table 8. As shown, DSR has a larger synthetic weight compared to DSDV and thus it is preferred.

4. Results Validation 4.1. Validation Model Four sets of simulations simili, sim#2, sim#3 and sim#4 are carried out as shown in Fig. 10 to validate the FSA model. As seen, both simili and sim#3 continue to employ the same protocol whereas the other two switch to a different protocol. Simül and simH2 are combined to determine the effect of switch from DSDV to DSR whereas sim#3 and sim#4 are combined to reveal the effectiveness of the switch to DSDV. The results are itemized in Table 9 and Table 10.

4.2. Fuzzy Synthetic Improvement Ratio Index (FSIRI) A metric, tenned fuzzy performance improvement ratio, denoted by FPIR, is developed to speci~ the level of difference between two alternatives under certain metrics. FPJR is defined as the quotient of the difference between the reference and target protocols for a value of the reference protocol. For metrics that are "the larger the better", FPlRref.tar is computed via ...(12) where Ptarget and Pref denote the performance of the target and reference protocols respectively.

For metrics that are "the larger the better" ...(13) ...(14) where atarget and aref are the average performance of target and reference protocols respectively, Aref and Atarget denote corresponding standard deviations.

For "the smaller the better" metrics, ...(15) ...(16) FSIRI is obtained by aggregating FPJR values with weights for metrics: ...(17) A positive FSIRI is desired since it indicates system improvement when a target protocol is selected. On the contrary, a negative FSIRJ reveals performance deterioration if the target protocol is selected. The FSIRJ values of the simulations in Fig. 10 are computed and listed in Fig. 11.

As shown in Fig. 11, an improvement ratio of 20.74 % is obtained in Sim#2 via switching from the original DSDV protocol to DSR. On the contrary, when DSDV replaces the original DSR as that in sim#4, the overall performance deteriorates. It is therefore concluded that DSR is more suitable for the case of 2 traffic streams, which is identical with results in Table 8.

5. Conclusion and Future Work In spite of various attributes and units for different performance metrics, the proposed FSA model is able to evaluate two routing protocols DSDV and DSR with five competing metrics and thus rank them reliably. The standard deviation is especially explored to increase accuracy of the final results. Extensive simulations show that appropriate protocol switch, basing on the performance evaluation results, may lead to a 20.74 % improvement at most. Despite only one case being studied in this paper using the FSA method, it is generic to other cases with different requirements.

The FSA is appropriate for scenarios where the decision maker is certain about his/her preference on the performance metrics and only the average value is considered. In the future, the FSA model will be fuzzified to incorporate the uncertainty of the decision maker.

Acknowledgment The project was supported by the State Key Laboratory of Advanced Optical Communication Systems Networks, Suzhou Science and Technology Fund (SZS201304) and Key Lab of Nanodevices and Nanoapplications, Suzhou Institute of Nano-Tech and Nano-Bionics, CAS (14ZS05).

References [1] . Cacciapuoti A. S., CalefFi M. and Paura L., Reactive routing for mobile cognitive radio ad hoc networks, Ad Hoc Networks, Vol. 10,2012, pp. 803-815.

[2] . C. E. Perkins and P. Bhagwat, Highly dynamic Destination-Sequenced Distance-Vector routing (DSDV) for mobile computers, in Proceedings of the SIGCOM'94, 1994, pp. 234-244.

[3] . D. Johnson et al., The Dynamic Source Routing Protocol (DSR) for Mobile Ad Hoc Networks for IPv4, RFC, 4728.

[4] . Perkins C., Belding-Royer E., Das S., Ad hoc on demand distance vector (AODV) routing (RFC 3561), IETF MANET Working Group, 2003.

[5] . Clausen T., Jaqcquet P., Optimized link state routing (OLSR) RFC 3626, IETF Networking Group, 2003.

[6] . Lundgren, H., Lundberg, D., Nielsen, J., Nordstrom, E., and Tschudin, C., A large-scale testbed for reproducible ad hoc protocol evaluations, in Proceedings of IEEE Wireless Communications and Networking Conference, Vol. 1,2002, pp. 412-418.

[7] . Kaynia M., Jindal N., Oien G. E., Improving the performance of wireless ad hoc networks through MAC layer design, IEEE Transactions on Wireless Communications, Vol. 10,2011, pp. 240-252.

[8] . Martinez, F. J., Toh, C. K., Cano, J. C., Calafate, C. T., & Manzoni, P., A survey and comparative study of simulators for vehicular ad hoc networks (VANETs), Wireless Communications and Mobile Computing, Vol. 11, 2011,pp. 813-828.

[9] . Y. Wang, A simple additive weighting method for time-series multi-indices decision making and its applications, Journal of Systems Engineering and Electronics, Vol. 10, 1999, pp. 21-26.

[10] . T. L. Saaty, How to make decision: The Analytic Hierarchy Process, European Journal of Operational Research, Vol. 48, 1990, pp. 9-26.

[11] . K P. Lego et al., Comparative study of ad hoc routing protocol AODV, DSR and DSDV in mobile ad hoc network, Indian Journal of Computer Science and Engineering, Vol. 1,2010, pp. 364-371.

[12] . T. Clausen, P. Jacquet, L. Viennot, Comparative study of routing protocols for mobile ad hoc networks, in Proceedings of the First Annual Mediterranean Ad Hoc Network Workshop, 2002.

[13] . Kaynia M, Jindal N, Oien G E., Improving the performance of wireless ad hoc networks through MAC layer design, IEEE Transactions on Wireless Communications, Vol. 10,2011, pp. 240-252.

[14] . The Network Simulator: ns-2, http://www.isi.edu /nsnam/ns/ [15] . Global Mobile Information System Simulation Library (GloMoSim), http://pcl.cs.ucla.edu/projects /glomosim/.

[16] . OPNET Modeler, http://www.opnet.com/solutions /network_rd/modeler.html [17] . Qualnet, http://www.scalable-networks.com /products/qualnet/ [18] . MATLAB: the language of technical computing, http ://www.math works .com/products/matlab.

[19] . Di Caro G A., Analysis of simulation environments for mobile ad hoc networks, Dalle Molle Institute for Artificial Intelligence, Tech. Rep, 2003.

[20] . Xu Z., On consistency of the weighted geometric mean complex judgement matrix in AHP, European Journal of Operational Research, Vol. 126, 2000, pp. 683-687.

[21] . L. Mikhailov, Deriving priorities from fuzzy pairwise comparison judgements, Fuzzy sets and systems, Vol. 134, 2003, pp. 365-385 1 Heng LUO,2 Jianping CHEN, 3 Ai-Huang GUO, "Ning WANG, 5 Lu GAO 2,4,5 Suzhou University of Science and Technology, Suzhou, China l'2 Suzhou key laboratory of mobile networking and applied technologies, Suzhou 3 Tongji University, Shanghai, China 1 Tel.: 8613862404231 1 E-mail: [email protected] Received: 18 April 2014 /Accepted: 30 June 2014 /Published: 31 July 2014 (c) 2014 IFSA Publishing, S.L.

[ Back To TMCnet.com's Homepage ]