802.11 Reference Design: Packet Flow
As packets move through the 802.11 reference design, the packet contents must be accessible by CPU High, CPU Low and the PHY cores. The design achieves this using the interconnects illustrated in the figure blow.
Each MicroBlaze has access to two AXI interconnects. For both CPUs the MicroBlaze DP port (non-cached peripheral memory access port) is connected to an AXI4 Lite interconnect. The peripheral cores connected to each AXI4 Lite interconnect are accessible by only one CPU. The cores are divided between CPUs based on which part of the MAC needs to access them. For example the radio_controller, w3_ad_controller and PHY configuration registers are all attached to the interconnect for CPU Low. Similarly the Ethernet cores are attached to the peripheral bus for CPU High. The mailbox and mutex ports for each CPU are also attached to their corresponding peripheral busses.
The MicroBlaze DC (cached memory access port) for both CPUs are attached to a shared AXI4 interconnect. The data cache is disabled in the Reference Design; all memory access via the DC ports access a slave memory device via the AXI4 interconnect. Both CPU DC ports are masters on this interconnect. The primary slave devices on this interconnect are the block RAMs used to implement the packet buffers (discussed below). Both CPUs can read and write any location in both memories. The AXI4 interconnect is a 64-bit crossbar clocked at 160MHz, which provides sufficient throughput to avoid contention between the CPUs.
Tx Packet Flow
The flow of packets leading to wireless transmissions is illustrated below. All wireless data transmissions pass through the upper-level MAC framework's Tx queuing system. The framework individually dequeues packets and copies them to a Tx packet buffer for processing and transmission by the lower-level MAC.
There are three sources of transmit packets in the 802.11 Reference Design:
- Ethernet Rx: data packets received on the ETH A interface may be encapsulated and enqueued for transmission if the address and packet types match filters implemented in the upper-level MAC code.
- Local: the upper-level MAC application can generate traffic internally. This includes management frames (probe request/response, etc) and arbitrary traffic generated by the framework's LTG system.
- Wireless Rx: packets received from the wireless interface can be enqueued for transmission. This path is used by the AP to relay packets from one associated STA to another.
Wireless transmissions may also originate in CPU Low. For the DCF MAC, control packets (RTS, CTS, ACK) and beacon packets are transmitted directly by the lower MAC without passing through CPU High's queuing framework.
Rx Packet Flow
The flow of packets received wirelessly is illustrated below. All wireless receptions are written to an Rx packet buffer by the Rx PHY. The lower-level MAC evaluates the received packet and may pass it up to the upper-level MAC for additional processing.
Received packets passed up to the upper-level MAC are routed to one of three sinks:
- Ethernet Tx: data packets may be de-encapsulated and transmitted via the ETH A interface
- Local: the upper-level MAC may directly consume a received packet. This includes management frames and arbitrary traffic generated by the LTG system at another node.
- Wireless Tx: as explained above, some MAC applications may re-transmit received packets via the wireless interface
Packet Buffers
The 802.11 Reference Design uses two dual-port 32KB on-chip RAMs as Tx and Rx packet buffers. One port of each RAM is attached to a BRAM interface controller (axi_bram_ctrl), which maps the RAM onto the address space of the two CPUs. The other port of each RAM is attached directly to the corresponding PHY core. These direct PHY connections do not traverse an AXI interconnect. The Tx and Rx PHY cores implement native 64-bit BRAM interfaces in logic.
The PHY cores divide each 32KB BRAM into 8 4KB buffers. Each 4KB buffer is referred to as a "packet buffer" in the MAC source code. The MAC and PHY number these buffers [0:7]. Each packet buffer has a fixed memory address which can be computed using the TX_PKT_BUF_TO_ADDR(x) and RX_PKT_BUF_TO_ADDR(x) macros. The packet buffers are accessible by both MicroBlaze CPUs and the central DMA (axi_cmda) controller. The Rx packet buffers are also accesible to the DMA controller (axi_dma) for the ETH A Ethernet interface.
Packet Buffer Contents
Each 4KB packet buffer is used to store packet payloads and related metadata. The MAC can store arbitrary metadata in each packet buffer before the actual packet bytes. Any metadata must be stored before the packet payload. The PHY must be configured with the size of the metadata so it knows where to read (Tx) and write (Rx) packet payloads. This configuration is handled at boot in wlan_mac_low.c:
The reference MAC code defines two C structs, tx_frame_info_t and rx_frame_info_t, to store the metadata for Tx and Rx packets. One instance of a metadata struct is stored at the base of each Tx and Rx packet buffer. The CPUs use the frame_info fields to set, store, and communicate per-packet parameters.
The PHY cores are configured to ignore the frame_info structs by setting "PHY Header Offset". In this context the "PHY Header" represents the first byte of the Tx or Rx packet buffer that is access by the PHY hardware.
//Move the PHY's starting address into the packet buffers by PHY_XX_PKT_BUF_PHY_HDR_OFFSET. //This accounts for the metadata located at the front of every packet buffer (Xx_mpdu_info) wlan_phy_rx_pkt_buf_phy_hdr_offset(PHY_RX_PKT_BUF_PHY_HDR_OFFSET); wlan_phy_tx_pkt_buf_phy_hdr_offset(PHY_TX_PKT_BUF_PHY_HDR_OFFSET);
The PHY_RX_PKT_BUF_PHY_HDR_OFFSET constant is defined as sizeof(rx_frame_info_t). Similarly, PHY_TX_PKT_BUF_PHY_HDR_OFFSET is defined as sizeof(tx_frame_info_t).
When MAC code accesses the packet buffers it is also necessary to know where the MAC payload (referred to as the MPDU) starts. In order to determine where the MPDU starts within the packet buffer, the constants PHY_RX_PKT_BUF_MPDU_OFFSET and PHY_TX_PKT_BUF_MPDU_OFFSET must be used. These include the PHY_HDR_OFFSET plus the PHY_HDR_SIZE.
The tx_frame_info_t and rx_frame_info_t structs are defined in wlan_mac_pkt_buf_util.h. The frame_info fields change with each release of the reference design as we add new MAC and PHY features. Refer to the source code for the current struct definitions.
Tx Packet Buffer States
The tx_frame_info_t struct has a tx_pkt_buf_state field used by the upper and lower MAC designs to coordinate access to the packet buffer contents. There are five possible states for a Tx packet buffer:
- UNINITIALIZED: the default state following a reconfiguration of the FPGA. The MAC code never sets a buffer to this state.
- HIGH_CTRL: the buffer is under control of CPU High. The MAC code in CPU Low must not read or write the buffer.
- LOW_CTRL: the buffer is under control of CPU Low and the Rx PHY. The MAC code in CPU High must not read or write the buffer.
- READY: the buffer contains a packet that is ready to be transmitted by the lower MAC and Tx PHY.
- DONE: the buffer contains a packet that has already been transmitted by the lower MAC and Tx PHY.
Each packet buffer transitions between these states in a rigid sequence. Each state transition is implemented by one CPU.
CPU High:
HIGH_CTRL → READY: after successful dequeue, immediately before sending MPDU_READY message to CPU Low
DONE → HIGH_CTRL: after receiving MPDU_DONE message, at start of MPDU_DONE handler
CPU Low:
READY → LOW_CTRL: after receipt of MPDU_READY message by CPU Low
LOW_CTRL → DONE: after return of MAC frame_transmit(), before sending MPDU_DONE message
This architecture is effectively a pipelined ping/pong scheme. Only two packet buffers are ever in the READY and/or LOW_CTRL states, minimizing disruption to the queue priorities implemented in CPU High. CPU High is allowed to dequeue a new packet into a HIGH_CTRL buffer before it runs post-Tx processing on the latest DONE buffer, minimizing the duration where CPU Low might be waiting for a new packet to transmit while CPU High is busy processing previous transmissions.
The figure to the right illustrates the sequence of state transitions and IPC messages for the preparation, transmission, and post-Tx handling of 2 packets.
Rx Packet Buffer States
The rx_frame_info_t struct has a rx_pkt_buf_state field used by the upper and lower MAC designs to coordinate access to the packet buffer contents. There are four possible states for an Rx packet buffer:
- UNINITIALIZED: the default state following a reconfiguration of the FPGA. The MAC code never sets a buffer to this state.
- HIGH_CTRL: the buffer is under control of CPU High. The MAC code in CPU Low must not read or write the buffer.
- LOW_CTRL: the buffer is under control of CPU Low and the Rx PHY. The MAC code in CPU High must not read or write the buffer.
- READY: the buffer contains a packet that has been processed by the lower MAC and is ready for further processing by the upper MAC. CPU Low must send a RX_MPDU_READY message to CPU High after setting an Rx buffer to this state.
Following at-boot initialization, there are 3 valid state transitions for Rx packet buffers. Each transition is only implemeted by one CPU.
CPU High:
READY → HIGH_CTRL: CPU High takes ownership of an Rx buffer after receiving the RX_MPDU_READY message from CPU Low
HIGH_CTRL → LOW_CTRL: CPU High finishes processing a reception and returns the buffer to control of CPU Low for use by a future reception
CPU Low:
LOW_CTRL → READY: CPU Low releases ownership of an Rx buffer after completing processing of a new reception, immediately before sending the RX_MPDU_READY message to CPU High
Attachments (1)
- tx_pkt_buf_pingpong_impl.png (122.3 KB) - added by murphpo 9 years ago.
Download all attachments as: .zip