WiMAX was formally introduced in 2004, with the publishing of the IEEE 802.16-2004 standard (IEEE 802.16-2004, 2004), which specifies the air interface, including the medium access control (MAC) layer and multiple physical (PHY) layer specifications for Fixed Broadband Wireless Access systems WiMAX claims to provide not only higher data rates over larger geographical areas but also claims to provide sophisticated QoS support capability matching its wired counterparts. However, due to lack of deployment there is a need to bridge the gap between the performance perception and the actual performance limitations of WiMAX. In this chapter we intend to present experimental results that would give an insight into the stability of the WiMAX link in comparison to its performance claims. Some of the performance results presented in this chapter is based on our previous efforts that has been recorded and published in (Yousaf et al., 2007).
1.1 Motivation and Objectives
Despite the high performance claim of WiMAX, it is yet desired to see a high deployment density. Due to this lack of deployment not enough off-the-field empirical data is available that would provide quantitative and qualitative attestation of the performance and operational capabilities and efficiencies of the WiMAX link. Such a provision of performance data will enable network service providers, planners and designers to make informed choices when deploying and integrating WiMAX into their network infrastructure. Availability of such a data could also be potentially used as a benchmark by telecommunication engineers and researchers to develop accurate mathematical and simulation models that could compare more realistically to the actual performance behaviour of the WiMAX link. This would thus enable them to develop and recommend better and efficient algorithms and solutions in terms of Next Generation Wireless Access Networks.
This lack of availability of real empirical data was the main motivation behind conducting field experiments over an actual 3.5 GHz WiMAX test-bed established at Communication Networks Institute (CNI), Technische Universität Dortmund. The tests were designed so as to give a better understanding and insight into the performance of the WiMAX link in terms of data throughput, link stability and QoS support for various services and traffic types at specified distances from the Base Station (BS). This experimental link was stressed in both uplink and downlink direction to determine the performance boundaries in an urban environment.
1.2 Related Work
Although some performance data is available but these are mostly based on analytical and/or simulation models (Hoymann, 2005). Also some experimental results are provided in (Schwengler & Pendharkar, 2005), but the experiments were conducted by emulating the WiMAX link using different channel models. The experiments performed in (Schwengler & Pendharkar, 2005) were conducted with multi-rate support enabled in which the Subscriber Station (SS) is allowed to choose a specific modulation scheme according to its SNR and the experiments were conducted for a limited range of modulation schemes. There is no mention of the transmit power of the base station or the type of the antennas used, a factor crucial to the correct understanding of the system's scope, and the maximum distance over which the tests were conducted was around 3000 meters (around 2 miles).
In our experiments, the multi-rate support (or the Adaptive Modulation & Coding (AMC) feature) was disabled to correctly ascertain the throughput and link quality for each of the eight available modulation schemes (please see table 1) with minimum transmit power of 13dBm over distances ranging from 220 meters to 9400 meters (9.4 km) from the BS. The AMC feature enables the system to dynamically adapt the modulation scheme and forward error correction (FEC) coding to actual link conditions for each uplink and downlink direction.
The link was tested for both the TCP and UDP traffic generated in both uplink and downlink direction, thereby emulating data relating to a case of real customer deployment.
In terms of testing the QoS support in WiMAX, experimental measurements have been made in (Scalabrino et al., 2006) using VoIP traffic and targeting the residential broadband access scenario, in which multiple SS, each having the same QoS configuration, communicate with a single base station in a point to multipoint (PMP) configuration. In our tests we have targeted a SOHO/SME broadband access scenario in which a single SS will be providing services to hundreds of users and/or multiple departments, each with a possibly unique QoS requirements, in a Virtual LAN (VLAN) configuration environment, where subscribers sharing the same QoS class are mapped onto the same service pipe, a virtual connection between a subscriber's application and the network resources, over the WiMAX link.
2. Test Bed Arrangement and Setup
For the field experiments an IEEE 802.16d (IEEE 802.16-2004, 2004) compliant equipment operating in a 3.5 GHz licensed band was selected.
As the WiMAX link is tested in terms of its capacity and its ability to provide QoS to multiple simultaneous applications services, two different test-beds were composed and configured. Both the test-beds had the same generic setup as shown in figure 1 but differed in terms of the equipment configuration.
On the subscriber side, the Client1 laptop is used for generating user traffic in the uplink direction and captures and analyses data in the downlink direction whereas Client2 laptop is running scripts for remote access and configuration of the WiMAX base station (AU-IDU) and the Customer Premises Equipment (CPE) (SU-IDU). The two client laptops and the Subscriber Unit Indoor Unit (SU-IDU) are connected via an Ethernet Interface (Ieth) to the VLAN switch, where the VLAN feature is disabled except for the QoS testing, described later. The SU-IDU in turn is connected to the Subscriber Unit Outdoor Unit (SU-ODU), which is a vertically polarized directional antenna with an 18dBi gain.
On the BS side, a Server is configured for generating traffic in the downlink, over the LUT (Link Under Test), and capturing and analyzing data in the uplink. The Access Unit Indoor Unit (AU-IDU) is connected via an IF cable (Iradio) to the Access Unit Outdoor Unit (AU-ODU), which consists of a 10.5 dBi gain omni-directional antenna and a high-power, full-duplex multi-carrier radio unit. The AU-IDU consists of two RJ45 ports namely; management port and data port, for micro base station remote configuration and the latter for data communication. These two ports are connected to the Server via Ethernet (i.e., Idata& Imgmt).
3. Measurement Methodology
The tests were divided into two categories:
Link Capacity/Throughput Testing, in which the ability, reliability and the robustness of the WiMAX link was tested in both the uplink and downlink direction for all eight modulation schemes by transmitting TCP and UDP traffic and real time video streams at different distances from the BS, and
QoS Testing, in which the inherent QoS support feature of the WiMAX was tested.
To attain reliable results with an appreciable level of confidence, each test run was conducted ten times and the duration of each test run was 60 seconds, accounting to a total approximate test duration of 260 minutes excluding the time for initializing the test scripts and making necessary configurations on the equipment. The throughput of the uplink and the downlink were measured separately in order to exclude interference effects.
The amount of traffic generated was always equal to the net physical data rate for each respective modulation scheme (see table 1) and corresponding packet losses were obviously measured.
The link capacity was tested for both Line Of Sight (LOS) and Near-LOS scenarios, but Non-LOS (NLOS) communication was not possible. The CPE were stationed away from the BS at the following distances and locations:
Table 2 summarizes the relevant test configuration of the AU.
To stress-test the link the transmit power of the AU was set to the minimum of 13dBm and the Automatic Transmit Power Control (ATPC) feature was enabled for the SU and AU. Although the maximum transmit power of the BS is 28dBm (6.918 Watts EIRP incl. 10.5dBi antenna gain, but the limit of 20dBm (1.096 Watts EIRP incl. 10.5dBi antenna gain) was imposed by the terms and conditions of the 3.5GHz research license. As a QoS-Profile a non real-time setting with a Committed Information Rate (CIR) of 12Mbps is chosen. The Multirate support in the AU was disabled, as mentioned earlier, in order to set the modulation scheme manually for each measurement to validate the link behaviour for each specific modulation scheme.
The outdoor tests were carried out by mounting the antenna on the roof of a car and powering the whole SU by a portable DC to AC power supply, which also powers the switch and the client PC.
3.1 Uplink and Downlink Test Configurations
For the uplink and downlink tests, TCP and UDP traffic was generated using IPerf (Iperf), which reported bandwidth, delay jitter and datagram loss, and captured on the receiver end by Wireshark (Wireshark) to capture and analyze the received data. In case of uplink capacity tests, the Client1 runs the IPerf server instance whereas the Server runs the IPerf client instance and Wireshark application; and this arrangement is reversed for the downlink capacity tests. The Client2 machine is used primarily to manage and configure the AU remotely by establishing an SSH session and is also used to monitor the configuration and connection statistics of the SU (for example, RSSI, SNR, modulation scheme, transmit and receive power level of the SU). This automatic management, configuration and monitoring of the AU and the SU is controlled by custom test processing scripts developed in Perl.
The captured data is then fed to the processing scripts where it is analysed for various parameters and mean values are hence calculated. The results are then graphically depicted using the GNU-Plot/Excel application.
3.2 QoS Test Configurations
In order to test the inherent ability of the WiMAX protocol to support QoS over the broadband wireless links the general set up is the same as in figure 1 with the exception that the VLAN feature in the switch is now enabled and the two client machines (Client1 and Client2) are placed in two separate VLANs. Client1 is generating UDP traffic with transmission parameters emulating a real time VoIP service, whereas Client2 is generating TCP traffic streams emulating a non-real time service application such as HTTP etc. The two VLANs are mapped onto two different service pipes, where the service pipe designated to carry the real time application data is configured with a Real Time Polling Services (rtPS) service class and the second service pipe carrying the non-real time application data is configured with a Best Effort (BE) service class. The two clients generate simultaneous traffic using the QAM64 ¾ modulation and coding scheme over the maximum bandwidth of the link so that the effect of these two interfering traffic streams over the QoS enabled WiMAX link can be duly analyzed.
The QoS tests were then performed for all the eight available modulation schemes.
4. Analysis and Evaluation
This section will provide the measurement results for the test configurations discussed
above and also discuss the effect of transmit power on the overall link quality.
4.1 Throughput Tests
Table 3 shows the TCP and UDP average throughput measured, which can be compared to the net physical bit rates for the corresponding modulation scheme given in table 1, for the eight modulation schemes at reference distances of 220 meters, 5400 meters and 9400 meters and the transmit power was set at a constant minimum of 13dBm. In the downlink direction (see figure 4) TCP throughput remains almost stable but shows greater inconsistency especially at higher modulation schemes (QAM64). It has been experimentally verified and observed that the TCP throughput stability is dependent on distance and transmit power. In terms of distance, the higher modulation schemes shows greater consistency at 220m (figure 4 (a)) than at 5400m (figure 4 (b)) or 9400m (figure 4 (c)). The effect of transmit power on the TCP downlink throughput is subsequently discussed.
Table 3. Average Throughputs for Uplink and Downlink TCP & UDP Traffic
The TCP throughput in the uplink direction is more stable at all the three reference distances and for all the modulation schemes and is similar, in performance, to figure 4 (a). This may be due to the quality, high sensitivity and the signal processing capabilities of the BS receiver.
In contrast to TCP downlink traffic, UDP has shown more throughput stability for all the modulation schemes in both the uplink and downlink direction at all three reference distances and the stability in throughput is similar to figure 4 (a)). The average UDP throughputs are given in table 3.
4.2 Effect of the Transmit Power and Link Layer ARQ on the TCP Throughput Performance
In the previously discussed results, the link was stress tested by keeping the transmit power of the base station to the minimum value of 13dBm at all distances and for all tests. In order to evaluate the effect of transmit power on the overall link quality, a second set of tests at 20dBm transmit power was carried out at d=9400m by transmitting TCP traffic in the downlink direction using QAM64 ¾, as that was more critical in terms of throughput stability (as evident from figure 4 (c)). As seen in figure 5, there is marked improvement in terms of link and throughput stability at 20dBm, especially for TCP traffic, which earlier had shown considerable instability at 13dBm.
However, even at 20dBm there are still fluctuations in the throughput which may not suit throughput sensitive applications. It is suggested that for TCP sessions using higher modulation schemes, the link layer (L2) ARQ algorithm in the BS must be enabled so that the errors can be detected and retransmissions requested at the link level. This will have a significant impact on the throughput stability, especially for the more susceptible higher modulation schemes.
The effect of ARQ on the TCP throughput for the QAM64 ¾ modulation scheme for power levels of 13dBm and 20dBm is shown in figure 6 (a) and 6 (b) respectively. As depicted in figure 6, when the ARQ feature is disabled, there are fluctuations in the TCP throughput, even at higher power levels (20dBm in our case). With the ARQ feature enabled, the TCP throughput shows an equally stable throughput performance for both the reference transmit-power levels of 13dBm and 20dBm.
4.3 TCP Window Size for Optimum Performance
During TCP connection, setting the right TCP Receive Window (RWIN) size is crucial from the perspective of throughput performance. The size of the RWIN is negotiated at the start of the connection during the TCP handshake process. The transmitter will stop sending data after it has transmitted data equal to the negotiated window size and will wait for an acknowledgement before resuming transmission.
Smaller values of RWIN will have adverse affect on the session quality as packets will be frequently dropped by the overflowing buffer. This dropping of packets will result in data retransmissions and will not only result in reduced average throughput but also affect the QoS of delay and throughput sensitive applications. This is depicted in figure 7, which shows the variation in the throughput of the TCP traffic in the downlink direction using the QAM64 ¾ modulation scheme for the RWIN size set to 30KB. For this test the Automatic Transmit Power Control (ATPC) feature is enabled which will dynamically adapt the transmit power of the SS so that transmissions can be received by the BS at optimal levels. The test is conducted at a distance of 220m. As can be seen from the figure 7 a 30KB RWIN size results in an unstable low throughput due to packet losses and repeated retransmissions.
To find the optimal boundaries for RWIN, it is observed that the RWIN should be between 85KB (default setting in Linux) and 256KB. At RWIN 256KB, the TCP session throughput not only increases to 10Mbps but shows a stable and constant throughput performance as depicted in figure 8.
4.4 QoS Tests
To ascertain the degree of QoS support, the tests were conducted by sending TCP background traffic, using QAM64 ¾ modulation, through a non real-time connection at full capacity of the bandwidth throughout the 60 seconds test duration. A 12 Mbps UDP traffic is sent in periodic durations of 10 seconds through a real-time connection with a committed bit rate of 5 Mbps and 10 Mbps the result of which is shown in figure 9 (a) and 9 (b) respectively.
From figure 9 it is seen that the throughput of the TCP data stream is exactly maximal (i.e, it is occupying the full available bandwidth spectrum) in periods where there is no UDP traffic. As soon as the UDP traffic is generated, the system will guarantee the required bandwidth as UDP is sent on a link which is assigned a higher QoS priority. This availability of bandwidth to the high priority UDP stream will be made at the cost of hogging the required bandwidth from the TCP session.
Thus, it is evident from the figure 9 (a) and 9 (b) that the TCP background traffic will occupy only that portion of the bandwidth which is not utilized by the UDP traffic and upon greater demand for bandwidth by the real-time traffic, the system ensures the guaranteed provisioning of the demanded bandwidth by claiming it from non-real time traffic, which in turn will experience bandwidth reduction, and this behaviour is consistent with the overall QoS philosophy.
Similar QoS tests were conducted for all the 8 modulation schemes and it was observed that the QoS paradigm applies consistently across all modulation schemes. As an example, figure 10 below shows the QoS tests for BPSK ½, QAM16 ½, and QAM64 ¾ for a real time UDP stream of 5Mbps.
5. Conclusions and Future Work
With the results gained from this pilot project, and within the given limits, resources and distance limitations, the field tests conducted at the CNI's WiMAX test-bed can be regarded as meeting the functional specification and requirements of a wireless broadband access network as per the IEEE 802.16-d standard. The measurements and results gathered through stress testing the WiMAX link demonstrates satisfactory throughput and level of services at different distances for both LOS and Near-LOS, even at the lowest transmission power (13dBm) at distances as far as 9.4km. An important observation is that Non Line of Sight (NLOS) operation was not possible in our tests, unless relays are used as demonstrated in (Marques et al., 2007), and the correct operation depends on the accurate adjustment of the CPE in order to get the best possible receive signal. It has been observed that even one centimetre change of the vertical or horizontal orientation could have a strong influence on the RSSI and the SNR and with this the quality of the link, adversely affecting the throughput.
Although the transmit power does influence the throughput stability, but it was observed that enabling ARQ at the link layer will not only stabilise the throughput but it will also increase the throughput and will compensate the effect of low transmit power on average throughput. Also important is to ensure the correct setting of the RWIN size for TCP traffic as smaller window sizes will adversely impact not only the average throughput but the link stability as well.
The WiMAX also has an effective QoS mechanism that applies equally well across all 8 modulation schemes.
Considerable degradation of service was observed during adverse weather conditions, such as during rain fall, resulting in delay, jitter and inconsistent service due to packet losses. Such weather conditions will prohibit the use of bandwidth intensive streaming multimedia services over the WiMAX link.
As part of the future work, more field tests are being planned at distances beyond 9.4 km and by introducing a second SS. Further investigation into the effect of transmit power on the link throughput and extensive testing of the QoS feature is also planned, particularly in terms of interference of different traffic types through traffic flow using all available service classes simultaneously. Similar tests are planned to be executed using the IEEE 802.16e; the mobility variant of WiMAX, also called the Mobile WiMAX.