{{{#!comment
[[Include(wiki:802.11/beta-note)]]
}}}

[[TracNav(802.11/TOC)]]

= 802.11 Reference Design: Packet Flow =

As packets move through the 802.11 reference design, the packet contents must be accessible by CPU High, CPU Low and the PHY cores. The design achieves this using the interconnects illustrated in the figure blow.

[[Image(wiki:802.11/files:wlan_ref_des_interconnects.png,width=350)]]

Each MicroBlaze has access to two AXI interconnects. For both CPUs the MicroBlaze DP port (non-cached peripheral memory access port) is connected to an AXI4 Lite interconnect. The peripheral cores connected to each AXI4 Lite interconnect are accessible by only one CPU. The cores are divided between CPUs based on which part of the MAC needs to access them. For example the radio_controller, w3_ad_controller and PHY configuration registers are all attached to the interconnect for CPU Low. Similarly the Ethernet cores are attached to the peripheral bus for CPU High. The mailbox and mutex ports for each CPU are also attached to their corresponding peripheral busses.

The MicroBlaze DC (cached memory access port) for both CPUs are attached to a shared AXI4 interconnect. The data cache is disabled in the Reference Design; all memory access via the DC ports access a slave memory device via the AXI4 interconnect. Both CPU DC ports are masters on this interconnect. The primary slave devices on this interconnect are the block RAMs used to implement the packet buffers (discussed below). Both CPUs can read and write any location in both memories. The AXI4 interconnect is a 64-bit crossbar clocked at 160MHz, which provides sufficient throughput to avoid contention between the CPUs.

== Tx Packet Flow ==

The flow of packets leading to wireless transmissions is illustrated below. All wireless data transmissions pass through the upper-level MAC framework's [wiki:../MAC/Upper/MACHighFramework/TX_queue Tx queuing system]. The framework individually dequeues packets and copies them to a Tx packet buffer for processing and transmission by the lower-level MAC.

[[Image(wiki:802.11/files:wlan_tx_pkt_flow.png)]]

There are three sources of transmit packets in the 802.11 Reference Design:
 * '''Ethernet Rx''': data packets received on the ETH A interface may be [wiki:../MAC/Upper/MACHighFramework/EthEncap encapsulated] and enqueued for transmission if the address and packet types match filters implemented in the upper-level MAC code.
 * '''Local''': the upper-level MAC application can generate traffic internally. This includes management frames (probe request/response, etc) and arbitrary traffic generated by the framework's [wiki:../MAC/Upper/MACHighFramework/LTG LTG system].
 * '''Wireless Rx''': packets received from the wireless interface can be enqueued for transmission. This path is used by the AP to relay packets from one associated STA to another.

Wireless transmissions may also originate in CPU Low. For the DCF MAC, control packets (RTS, CTS, ACK) and beacon packets are transmitted directly by the lower MAC without passing through CPU High's queuing framework.

== Rx Packet Flow ==

The flow of packets received wirelessly is illustrated below. All wireless receptions are written to an Rx packet buffer by the Rx PHY. The lower-level MAC evaluates the received packet and may pass it up to the upper-level MAC for additional processing.

[[Image(wiki:802.11/files:wlan_rx_pkt_flow.png)]]

Received packets passed up to the upper-level MAC are routed to one of three sinks:
 * '''Ethernet Tx''': data packets may be [wiki:../MAC/Upper/MACHighFramework/EthEncap de-encapsulated] and transmitted via the ETH A interface
 * '''Local''': the upper-level MAC may directly consume a received packet. This includes management frames and arbitrary traffic generated by the [wiki:../MAC/Upper/MACHighFramework/LTG LTG system] at another node.
 * '''Wireless Tx''': as explained above, some MAC applications may re-transmit received packets via the wireless interface

= Packet Buffers =

The 802.11 Reference Design uses two dual-port 32KB on-chip RAMs as Tx and Rx packet buffers. One port of each RAM is attached to a BRAM interface controller (axi_bram_ctrl), which maps the RAM onto the address space of the two CPUs. The other port of each RAM is attached directly to the corresponding PHY core. These direct PHY connections do not traverse an AXI interconnect. The Tx and Rx PHY cores implement native 64-bit BRAM interfaces in logic.

The PHY cores divide each 32KB BRAM into 8 4KB buffers. Each 4KB buffer is referred to as a "packet buffer" in the MAC source code. The MAC and PHY number these buffers ![0:7]. Each packet buffer has a fixed memory address which can be computed using the {{{TX_PKT_BUF_TO_ADDR(x)}}} and {{{RX_PKT_BUF_TO_ADDR(x)}}} macros. The packet buffers are accessible by both MicroBlaze CPUs and the central DMA (axi_cmda) controller. The Rx packet buffers are also accesible to the DMA controller (axi_dma) for the ETH A Ethernet interface.

== Packet Buffer Contents ==

Each 4KB packet buffer is used to store packet payloads and related metadata. The MAC can store arbitrary metadata in each packet buffer before the actual packet bytes. Any metadata must be stored before the packet payload. The PHY must be configured with the size of the metadata so it knows where to read (Tx) and write (Rx) packet payloads. This configuration is handled at boot in [browser:/ReferenceDesigns/w3_802.11/c/wlan_mac_low_framework/wlan_mac_low.c wlan_mac_low.c]:

The reference MAC code defines two C structs, {{{tx_frame_info_t}}} and {{{rx_frame_info_t}}}, to store the metadata for Tx and Rx packets. One instance of a metadata struct is stored at the base of each Tx and Rx packet buffer. The CPUs use the {{{frame_info}}} fields to set, store, and communicate per-packet parameters.

The PHY cores are configured to ignore the {{{frame_info}}} structs by setting "PHY Header Offset". In this context the "PHY Header" represents the first byte of the Tx or Rx packet buffer that is access by the PHY hardware.

{{{
#!c
//Move the PHY's starting address into the packet buffers by PHY_XX_PKT_BUF_PHY_HDR_OFFSET.
//This accounts for the metadata located at the front of every packet buffer (Xx_mpdu_info)
wlan_phy_rx_pkt_buf_phy_hdr_offset(PHY_RX_PKT_BUF_PHY_HDR_OFFSET);
wlan_phy_tx_pkt_buf_phy_hdr_offset(PHY_TX_PKT_BUF_PHY_HDR_OFFSET);
}}}

The {{{PHY_RX_PKT_BUF_PHY_HDR_OFFSET}}} constant is defined as {{{sizeof(rx_frame_info_t)}}}.  Similarly, 
{{{PHY_TX_PKT_BUF_PHY_HDR_OFFSET}}} is defined as {{{sizeof(tx_frame_info_t)}}}.  

When MAC code accesses the packet buffers it is also necessary to know where the MAC payload (referred to as the MPDU) starts.  In order to determine where the MPDU starts within the packet buffer, the constants {{{PHY_RX_PKT_BUF_MPDU_OFFSET}}} and {{{PHY_TX_PKT_BUF_MPDU_OFFSET}}} must be used.  These include the PHY_HDR_OFFSET plus the PHY_HDR_SIZE.

The {{{tx_frame_info_t}}} and {{{rx_frame_info_t}}} structs are defined in [browser:/ReferenceDesigns/w3_802.11/c/wlan_mac_common/include/wlan_mac_pkt_buf_util.h wlan_mac_pkt_buf_util.h]. The {{{frame_info}}} fields change with each release of the reference design as we add new MAC and PHY features. Refer to the source code for the current struct definitions.


== Tx Packet Buffer States ==
[[Image(tx_pkt_buf_pingpong_impl.png,width=300,align=right)]]

The {{{tx_frame_info_t}}} struct has a {{{tx_pkt_buf_state}}} field used by the upper and lower MAC designs to coordinate access to the packet buffer contents. There are five possible states for a Tx packet buffer:
 * {{{UNINITIALIZED}}}: the default state following a reconfiguration of the FPGA. The MAC code never sets a buffer to this state.
 * {{{HIGH_CTRL}}}: the buffer is under control of CPU High. The MAC code in CPU Low must not read or write the buffer.
 * {{{LOW_CTRL}}}:  the buffer is under control of CPU Low and the Rx PHY. The MAC code in CPU High must not read or write the buffer.
 * {{{READY}}}: the buffer contains a packet that is ready to be transmitted by the lower MAC and Tx PHY.
 * {{{DONE}}}: the buffer contains a packet that has already been transmitted by the lower MAC and Tx PHY.

Each packet buffer transitions between these states in a rigid sequence. Each state transition is implemented by one CPU.

'''CPU High:'''[[BR]]
{{{HIGH_CTRL → READY}}}: after successful dequeue, immediately before sending {{{MPDU_READY}}} message to CPU Low[[BR]]
{{{DONE → HIGH_CTRL}}}: after receiving {{{MPDU_DONE}}} message, at start of {{{MPDU_DONE}}} handler[[BR]]

'''CPU Low:'''[[BR]]
{{{READY → LOW_CTRL}}}: after receipt of {{{MPDU_READY}}} message by CPU Low[[BR]]
{{{LOW_CTRL → DONE}}}: after return of MAC frame_transmit(), before sending {{{MPDU_DONE}}} message[[BR]]

This architecture is effectively a pipelined ping/pong scheme. Only two packet buffers are ever in the {{{READY}}} and/or {{{LOW_CTRL}}} states, minimizing disruption to the queue priorities implemented in CPU High. CPU High is allowed to dequeue a new packet into a {{{HIGH_CTRL}}} buffer before it runs post-Tx processing on the latest {{{DONE}}} buffer, minimizing the duration where CPU Low might be waiting for a new packet to transmit while CPU High is busy processing previous transmissions.

The figure to the right illustrates the sequence of state transitions and IPC messages for the preparation, transmission, and post-Tx handling of 2 packets.

== Rx Packet Buffer States ==
The {{{rx_frame_info_t}}} struct has a {{{rx_pkt_buf_state}}} field used by the upper and lower MAC designs to coordinate access to the packet buffer contents. There are four possible states for an Rx packet buffer:
 * {{{UNINITIALIZED}}}: the default state following a reconfiguration of the FPGA. The MAC code never sets a buffer to this state.
 * {{{HIGH_CTRL}}}: the buffer is under control of CPU High. The MAC code in CPU Low must not read or write the buffer.
 * {{{LOW_CTRL}}}:  the buffer is under control of CPU Low and the Rx PHY. The MAC code in CPU High must not read or write the buffer.
 * {{{READY}}}: the buffer contains a packet that has been processed by the lower MAC and is ready for further processing by the upper MAC. CPU Low must send a {{{RX_MPDU_READY}}} message to CPU High after setting an Rx buffer to this state.

Following at-boot initialization, there are 3 valid state transitions for Rx packet buffers. Each transition is only implemeted by one CPU.

'''CPU High:'''[[BR]]
{{{READY → HIGH_CTRL}}}: CPU High takes ownership of an Rx buffer after receiving the {{{RX_MPDU_READY}}} message from CPU Low[[BR]]
{{{HIGH_CTRL → LOW_CTRL}}}: CPU High finishes processing a reception and returns the buffer to control of CPU Low for use by a future reception

'''CPU Low:'''[[BR]]
{{{LOW_CTRL → READY}}}: CPU Low releases ownership of an Rx buffer after completing processing of a new reception, immediately before sending the {{{RX_MPDU_READY}}} message to CPU High