WARP Project Forums - Wireless Open-Access Research Platform

You are not logged in.

#1 2016-May-29 04:40:17

Edlmann
Member
Registered: 2015-Sep-09
Posts: 30

High PER of PHY under Interference

We are currentlyin the Process of Evaluating a TDMA based MAC under interference from commercial 802.11 a traffic and are seeing behaviour we cannot quite explain ourselves.

In the current setup, we have 3 WarpV3 nodes setup on the same table (with distances < 1.5m between each other), with an interference system consisting of an 802.11a AP + Wlan stick in the next room over (currently running on 20 MHz bandwidth with 11 MBit/s and a raw bitrate of 65-130 MBit/s, depending on link quality).
Our frame_receive + frame_transmit are still configured as here (the problem discussed in that thread has been successfully solved with your proposed config, thanks again).

The interference packets are received by the Boards with about -50 to -70 dBm, but we already see strange behaviour here: Only rarely do we see the same Interference packet beeing received by all 3 nodes, often times its only decoded by a single node. A large number of the interference packets seems to be not decoded at all.

As for the transmissions of our MAC, a large number of those is not received by the 2 other nodes aswell (~15% are not seen by the other two nodes at all, another 10% only by a single other node), even though the receptions that manage to be decoded are at -20 to -45 dBm. Even packets with a large distance to received interference packets (over 1.9ms since the last decoded interference packet) get lost completely sometimes.

We have checked everything we could think of software wise, now the question would be if there could be side-effects to our approach of transmission without regard to the RX-State of the system. In the current implementation, both RX_BUSY + CCA_BUSY are ignored during TX. Another question would be if it is possible to transmit directly after discarding a reception using hw_clear_rx_started(), in our testing this seemed to work fine, but are there additional effects inside the PHY here?

Greetings

Offline

 

#2 2016-May-29 09:02:52

chunter
Administrator
From: Mango Communications
Registered: 2006-Aug-24
Posts: 1212

Re: High PER of PHY under Interference

Edlmann wrote:

In the current setup, we have 3 WarpV3 nodes setup on the same table (with distances < 1.5m between each other), with an interference system consisting of an 802.11a AP + Wlan stick in the next room over (currently running on 20 MHz bandwidth with 11 MBit/s and a raw bitrate of 65-130 MBit/s, depending on link quality).

The 802.11a AP is a commercial Wi-Fi device, and not a WARPv3 kit configured as an 802.11a AP, right? Can you clarify what you mean by "with 11 MBit/s and a raw bitrate of 65-130 MBit/s"? Is every rate/MCS chosen by the interfering Wi-Fi link something you would expect to be decoded by your primary WARPv3 experiment in the other room (rate/MCS 0-7, up to 54Mbps for NONHT and 65Mbps for HTMF)?

Edlmann wrote:

The interference packets are received by the Boards with about -50 to -70 dBm, but we already see strange behaviour here: Only rarely do we see the same Interference packet beeing received by all 3 nodes, often times its only decoded by a single node. A large number of the interference packets seems to be not decoded at all.

Is this observation in the presence of the traffic being sent through your primary TDMA link? If you disable all traffic in your TDMA link, do the WARPv3 kits reliably decode packets from the interfering Wi-Fi link that are decoded by at least one other WARPv3 kit (as to eliminate the question of whether or not the packet is possible to be decoded if were using a rate not supported by the 802.11 Reference Design)?

Edlmann wrote:

As for the transmissions of our MAC, a large number of those is not received by the 2 other nodes aswell (~15% are not seen by the other two nodes at all, another 10% only by a single other node), even though the receptions that manage to be decoded are at -20 to -45 dBm. Even packets with a large distance to received interference packets (over 1.9ms since the last decoded interference packet) get lost completely sometimes.

Similar to my previous question, are the packet drops only presence in the interfering Wi-Fi stream in the other room? Without it, packets are decoded reliably?

Assuming I'm understanding the experiment properly (which isn't guaranteed -- please correct me if it looks like I'm not understanding the setup), then it doesn't surprise me that you are seeing packet drops from your colliding streams even with the 15-50dB difference in relative receive powers. If the PHY starts decoding the weaker interfering Wi-Fi packet when the stronger packet from the network you care about comes in, that stronger packet can be lost if the PHY hasn't had time to reset back to start over on a new reception.

Offline

 

#3 2016-May-29 11:15:32

Edlmann
Member
Registered: 2015-Sep-09
Posts: 30

Re: High PER of PHY under Interference

chunter wrote:

The 802.11a AP is a commercial Wi-Fi device, and not a WARPv3 kit configured as an 802.11a AP, right? Can you clarify what you mean by "with 11 MBit/s and a raw bitrate of 65-130 MBit/s"? Is every rate/MCS chosen by the interfering Wi-Fi link something you would expect to be decoded by your primary WARPv3 experiment in the other room (rate/MCS 0-7, up to 54Mbps for NONHT and 65Mbps for HTMF)?

Yes, the AP is a commercial device. With the specific hardware we are using, the control over the utilized MCS is heavily limited, only the supported standard (b/g/a) can be chosen. What we are observing is the raw link bitrate switching between 65 Mbit up to 130 Mbit/s, with MCS 7, 14 and 15 respectively. We only expect packets of MCS 7 to be decoded. However according to wavemon, the link is utilizing 65 MBit/s for periods of >10 s about once a minute, and even in these periods we see only few packets beeing decoded, and not by all stations.

As for the interfering datastream, we are running iperf over udp with 11 MBit/s between the two devices.

chunter wrote:

Is this observation in the presence of the traffic being sent through your primary TDMA link? If you disable all traffic in your TDMA link, do the WARPv3 kits reliably decode packets from the interfering Wi-Fi link that are decoded by at least one other WARPv3 kit (as to eliminate the question of whether or not the packet is possible to be decoded if were using a rate not supported by the 802.11 Reference Design)?

When disabling all transmissions of our MAC protocol, we see the following traffic pattern:

Interference

Each entry in the activity rows indicates a received packet which triggered the frame_receive callback, but was discarded due to its MCS (we don't use any of the interfering MCS's in our TDMA, and can discard these receptions using an early hw_clear_rx based on this). The timer-rows, coloring aswell as the indicated packet length have no significance without the system running. The scale at the top is in ms.

What seems puzzling to us is that a large number of packets (~20 %) is not decoded by all 3 nodes, even though they are in close physical proximity and without any attenuation elements.

chunter wrote:

Similar to my previous question, are the packet drops only presence in the interfering Wi-Fi stream in the other room? Without it, packets are decoded reliably?

Even without interference, we see a higher rate of lost packets than expected. Here, we still see up to 1% of packets not beeing received by at least one other station, with both medium and high transmission power.
[Edit] I have to correct myself here. The 1% figure was from an initial test on a single channel, but running additional tests on different channels leads me to believe that there was some interference i did not know about during the measurement. On other channels only single transmissions out of a million get lost.[/Edit]

chunter wrote:

Assuming I'm understanding the experiment properly (which isn't guaranteed -- please correct me if it looks like I'm not understanding the setup), then it doesn't surprise me that you are seeing packet drops from your colliding streams even with the 15-50dB difference in relative receive powers. If the PHY starts decoding the weaker interfering Wi-Fi packet when the stronger packet from the network you care about comes in, that stronger packet can be lost if the PHY hasn't had time to reset back to start over on a new reception.

Our goal is to discard the received interference packets as early as possible (pased on the PLCP, since we use a different set of MCS's compared to the interferer), in the current setup we can discard 100% of interferer packets based on PLCP alone, which means that as soon as we enter the frame_receive, the packet will be discarded + hw_clear_rx called (by the framework). Only for the duration from the packetDetector detecting a packet until we reach the callback should the PHY be blocked and new receptions be possible immediately afterwards - based on the way we understood the 1.5 reference design. If this is the sole cause of the issues we're seeing, could you please elaborate on the time the PHY will be blocked in this scenario? We were under the impression that this time is low enough to achieve a reasonable PER.

In case a packet is not discarded, it will be processed by the MAC directly on CPU_LOW. This can lead to processing delays of >30-40us and may generate a transmission response.
If hw_rx_finish() is called for every packet, including the interference packets, the MAC performance deteriorates even more.

Last edited by Edlmann (2016-May-30 07:13:35)

Offline

 

#4 2016-May-30 07:45:45

Edlmann
Member
Registered: 2015-Sep-09
Posts: 30

Re: High PER of PHY under Interference

To maybe take a step back from the actual MAC protocol implementation we are utilizing, could you maybe give your opinion on two goals we are trying to achieve:

1.) We can differentiate between packets from our MAC protocol and the interference system based on the PLCP alone. What we were hoping to achieve with the Reference Design 1.5.1 is to be able to cancel a reception asap - which currently as we see it would be at the beginning of frame_receive if we don't want to modify the hardware design. After this early cancel, the Warp nodes should be free for other receptions / execute a transmission that would be decodable by other nodes (potentially at a worse BER due to interference). How long does it take in your opinion until the PHY is free again here?

2.) The frame_transmit function should (optimally) be a fire & forget function. Independent of current PHY/MAC/Medium state, if frame_transmit is called, it should generate an actual transmission. From our understanding of the design, our current implementation achieves this. However, we have not seen a single transmission started while the medium is busy with an interferer packet suceed. Does cancelling the reception using hw_clear_rx free up the PHY for TX aswell, or could there be side-effects at play here preventing a transmission from suceeding?

We will try to create an implementation based on NOMAC where we can recreate the behaviour, will post here once we are able to create such an implementation.

Last edited by Edlmann (2016-May-30 07:48:23)

Offline

 

#5 2016-May-30 14:35:06

Edlmann
Member
Registered: 2015-Sep-09
Posts: 30

Re: High PER of PHY under Interference

An implementation where the problems we are having can be clarified + specified can be found here. It is not directly based on NOMAC, but rather a heavily stripped down version of our CPU_LOW implementation, with all changes to the framework we made intact, incase it is some side-effect of the (minor) adaptations we had to make due to compatibility/legacy reasons.

In a free channel using this implementation, all stations manage to communicate with each other and are rarely loosing packets (For each link we ran ~10k packets, with none beeing lost). The main issue is only present on the channel where the interference system is operating.

To outline the behaviour of the system in the interference channel (as it is currently implemented in the zip archive):

Once a reception is started, decide wether it is from us (MCS = 2) or the interfering system (MCS >= 7).
If MCS = 2, finish the reception & log the reception into a buffer.
If MCS != 2 we tested two variants:
      1. first call hw_rx_finish, then call hw_clear_rx
      2. call clear_rx immediately (& log the time at which this happens)
          wait for a specified amount of time (50 + 100 us respectively in the screenshots below)
       both:    If we are station 1, generate a transmission to the other nodes

Legend to the following screenshots:
Timescale at the top: in us
Activity for each station: RX/TX activity
Green event: Node is transmitting something
Blue event: Node is receiving a packet with valid MCS
Orange: Interference event / clear_rx event, depending on label

As for our evaluation results:
Variant 1 (hw_rx_finish before hw_rx_clear)
http://i.imgur.com/wg7oOg7.png

If stations wait until the full interference packet has decoded + only then start transmitting, the other stations receive the packet fine. This is as expected.

Variant 2 with a delay of 50 us
http://i.imgur.com/WEsepn2.png
There are 3 cases displayed here:
At the left, only station 1 started decoding the packet. The other 2 did not generate an rx_start event. In this case, the following transmission by station 1 (while the medium is still busy) succeeds fine.
In the middle, station 1 + 2 started decoding. Station 2 immediately cleared the reception with rx_clear(), the time of this happening is indicated by the 0xDD event's starting point. Only station 6 receives the succeeding transmission. Station 2 - even though it is idle - does not.
On the right: All stations start decoding the reception. The successive transmission by station 1 is not heard by any node.

Variant 2 with a delay of 100 us
http://i.imgur.com/viBxWQS.png
Even when increasing the delay until the transmission is started to 100 us, the image does not change. Only if the interfering packet is small enough (seen on the right) that the 100 us guard is enough time, the transmission has a chance of suceeding. (This was taken before i cleaned up the rendering to be easier to follow, but the basic idea is the same. aa => Inteference packet, dd => hw_clear_rx called)

What we expected:
With the ability to clear an ongoing reception in 1.5.1, we expected the nodes to be able to receive another packet after hw_clear_rx has been called. The medium is still busy, but the nodes itself should be free + the difference in transmission power enough for the transmission to succeed.

Last edited by Edlmann (2016-May-31 05:03:11)

Offline

 

#6 2016-May-31 10:29:22

murphpo
Administrator
From: Mango Communications
Registered: 2006-Jul-03
Posts: 5159

Re: High PER of PHY under Interference

What we were hoping to achieve with the Reference Design 1.5.1 is to be able to cancel a reception asap - which currently as we see it would be at the beginning of frame_receive if we don't want to modify the hardware design. After this early cancel, the Warp nodes should be free for other receptions / execute a transmission that would be decodable by other nodes (potentially at a worse BER due to interference). How long does it take in your opinion until the PHY is free again here?

It depends on whether a valid SIGNAL field was decoded.

Part of the Rx PHY's responsibility is to generate an RX_END for every RX_START and, whenever possible, align the RX_END to the actual end of the incoming waveform. The 11a and 11n PHY headers allow nodes to calculate the duration of a waveform even if they are not able to decode the waveform (unsupported MCS / bandwidth / short GI / 2+ spatial streams / LDPC / etc). Our PHY implements this behavior in the "Sync & Antenna Sel/PHY CCA/Rate-Length Busy" subsystem. This block interprets the RATE and LENGTH fields in any valid SIGNAL field and runs a series of counters to assert RX_END at the end of the incoming waveform.

This logic runs independent of the demod/decoding pipeline. The Rate/Length Busy state can be cleared with a software reset of the Rx PHY (i.e. toggle WLAN_RX_REG_CTRL_RESET). Asserting the software reset after RX_START and before RX_END will properly assert RX_END and RX_END_ERROR, informing the MAC core the Rx event terminated unexpectedly. After this reset the Rx PHY will return to idle, ready to receive a new packet.

Edit: by the way, what tool creates the timeline images you posted above?

Offline

 

#7 2016-May-31 11:54:36

Edlmann
Member
Registered: 2015-Sep-09
Posts: 30

Re: High PER of PHY under Interference

Thanks for your answer.

Toggling WLAN_RX_REG_CTRL_RESET is what we did before 1.5, which worked fine but there was some delay before the PHY was able to receive again. This was probably because we had to reset all 3 blocks: MAC, TX + RX, otherwise there were unintended sideeffects.

From the changelog it seemed that this could now be done cleaner + without fully resetting the PHY/MAC

Removed RX_END latches that blocked Rx PHY. Now there's a single RX_STARTED latch that blocks future receptions. MAC software can clear the RX_STARTED latch anytime (even before the current Rx finishes) to release the Rx PHY for the next reception.

So the WLAN_MAC_CTRL_MASK_RESET_RX_STARTED_LATCH bit (called by wlan_mac_hw_clear_rx_started()) only clears software events, while the actual PHY hardware is not cleared, did I understand your answer correctly there?

The tool is custom built with visjs and basically just a mapper between binary events generated by the boards + visjs.

Last edited by Edlmann (2016-May-31 12:00:15)

Offline

 

#8 2016-May-31 12:01:45

murphpo
Administrator
From: Mango Communications
Registered: 2006-Jul-03
Posts: 5159

Re: High PER of PHY under Interference

Edlmann wrote:

Thanks for your answer.

Toggling WLAN_RX_REG_CTRL_RESET is what we did before 1.5, which worked fine but there was some delay before the PHY was able to receive again.

From the changelog it seemed that this could now be done cleaner + without fully resetting the PHY

Removed RX_END latches that blocked Rx PHY. Now there's a single RX_STARTED latch that blocks future receptions. MAC software can clear the RX_STARTED latch anytime (even before the current Rx finishes) to release the Rx PHY for the next reception.

So the WLAN_MAC_CTRL_MASK_RESET_RX_STARTED_LATCH bit (called by wlan_mac_hw_clear_rx_started()) only clears software events, while the actual PHY hardware is not cleared, did I understand your answer correctly there?

Right. In both versions the MAC core implements a latch that blocks the Rx PHY's pkt det logic, preventing the start of a new Rx event.

In v1.4 this latch asserted on RX_END. This required MAC software to wait for every RX_END event so it could clear the latch and allow new Rx events. This architecture was also susceptible to bad states (as you found) when the Rx PHY was reset and did not generate an RX_END.

In v1.5 we moved this latch to assert on RX_START. Once this latch asserts the Rx PHY will continue the Rx process but will not start a new one. The MAC software can clear the RX_STARTED latch anytime during or after the Rx.

In both versions the Rx PHY still tries to assert RX_END at the end of the incoming waveform via the Rate-Length Busy logic. This behavior is required to keep the DCF MAC synchronized to medium activity, whether or not the Rx PHY attempts decoding.

Offline

 

#9 2016-May-31 12:10:40

Edlmann
Member
Registered: 2015-Sep-09
Posts: 30

Re: High PER of PHY under Interference

With these changes, can we now only reset the RX-PHY block to fully clear the rx without side-effects? In 1.4 we had to clear all 3, RX, TX + Mac.

And would it be possible to change the Rx PHY in a way were it also reacts to hw_clear_rx by asserting RX_END, or would this be a more involved approach?

Offline

 

#10 2016-May-31 13:51:52

murphpo
Administrator
From: Mango Communications
Registered: 2006-Jul-03
Posts: 5159

Re: High PER of PHY under Interference

With these changes, can we now only reset the RX-PHY block to fully clear the rx without side-effects? In 1.4 we had to clear all 3, RX, TX + Mac.

I believe this is true. I added a path in the v1.5 Rx PHY connecting the software reset to the RX_END generation logic, so that an early PHY reset will still assert RX_END and RX_END_ERROR, clearing the "waiting for RX_END" state in the MAC core.

And would it be possible to change the Rx PHY in a way were it also reacts to hw_clear_rx by asserting RX_END, or would this be a more involved approach?

Certainly possible, but would require changes to the (at least) the Rx PHY core. The MAC core uses the RX_STARTED latch to block new Rx pkt det events via the "phy_rx_block_pktdet" port. You could modify the Rx PHY to treat the falling edge of that signal as a reset.

Offline

 

Board footer