Extending the Experiments Framework

The 802.11 Reference Design Experiments Framework provides a variety of commands to control and observe WARP v3 nodes in real time. However many experiments will require additional commands, especially when users extend the reference code with new behaviors.

Starting in 802.11 Reference Design v1.4, we have greatly simplified the process of implementing extensions to the experiments framework. Adding wlan_exp commands was more complicated in previous versions; refer to the archived HowToAddCommand page for details on adding commands to versions before v1.4.

There are two kinds of framework extensions:

  • User Commands: commands interact with the MAC High Framework and the upper-level MAC running in CPU High
  • Low Params: a write-only path for setting parameters in the MAC Low Framework and the lower-level MAC running in CPU Low

User Commands

The User Command flow provides a simple way to send custom commands and data from your Python script to the WARP v3 nodes in your experimental network. This flow presents a pipe for exchanging arbitrary commands and responses between a Python script and the upper-level MAC in a WARP node running the 802.11 Reference Design.


In Python user commands are constructed as:

  • Command ID: an arbitrary 24-bit integer
  • Command arguments: an optional list of u32 arguments supplied by the Python code that is passed to the C code handler

The command ID is used by the C code to select which code block should process the command payload. The Python and C code must use matching lists of command IDs. All user command IDs must be 24-bit integers (i.e. in [0, 224 -1]). The framework does not reserve any command ID values in this range for user commands.

The command arguments are an optional payload supplied by the Python script. The wlan_exp framework sends this payload to the node with no modifications. The arguments must be constructed as a list of unsigned 32-bit integers (u32 values). The Python and C code must implement complementary argument construction/parsing functions. The argument list may be omitted if a command requires no payload.

The WARP node must respond to every user command. The reference C code implements this response by default. Each command response may contain a payload supplied by the user C code. If the command handler supplies a response payload, this payload is conveyed back from the node to the Python code automatically. When present the response payload is always a list of u32 values. Your Python and C code must implement complementary response payload generation/processing functions.

User commands are passed to the wlan_exp framework using the send_user_command(cmd_id, args) method. For example:

# No command arguments, no response payload
my_node.send_user_command(CMDID_USER_MYCMD, args=None)

# 3 arguments, no response payload
my_node.send_user_command(CMDID_USER_MYCMD, args=[ARG0 ARG1 ARG2])

# 2 arguments, with response payload
cmd_resp = my_node.send_user_command(CMDID_USER_MYCMD, args=[ARG0 ARG1])

For simple commands your Python script can call send_user_command directly. For more complicated commands it is suggested you implement your custom command functionality in a separate method. This will simplify debugging your code and reusing your custom commands across multiple experiments. The sample code below adopts the approach of a custom method for sending a user command.

MAC C Code

The 802.11 MAC code running on the WARP node handles user commands at two levels.

  • Framework: If the command behavior is common to all high-level MAC applications, the command is handled in the process_user_cmd() function in wlan_exp_user.c. You should modify process_user_cmd() to handle any new framework-level commands required for your application. Any command IDs not handled in process_user_cmd() will be passed through to the high-level MAC application.
  • High-level MAC Application: If a command behavior depends on the high-level MAC application, the command is handled in the wlan_exp_process_user_cmd() callback function in the high-level MAC. You should modify the wlan_exp_process_user_cmd() function in the wlan_mac_ap.c, wlan_mac_sta.c and wlan_mac_ibss.c files to implement any MAC-specific command behaviors.

Command handlers in these functions share a common structure:

  • Examine the command ID and determine which code block (case) should handle the command
  • Implement the required command behavior using any supplied arguments in the cmd_args_32 array
  • If a response payload is required, populate the response values into the resp_args_32 array
  • Update the response header fields for response length (resp_hdr->length) and number of response arguments (resp_hdr->num_args)

The wlan_exp framework will encapsulate the command response payload in an Ethernet frame and return it to the calling Python script for processing. One thing to note is that a command response payload is limited to max_resp_len values.

Interrupt Safety
The process_user_cmd() and wlan_exp_process_user_cmd() functions executed outside the context of an interrupt service routine. As a result these functions may be interrupted at any time. Almost all processing in CPU High occurs in ISR contexts, including all processing of wireless and Ethernet Tx/Rx. If your command handler accesses or modifies data structures that might be modified in one of the upper-level MAC's interrupt service routines, you must take precautions to avoid bad states.

The easiest way to protect a command handler from bad states due to unexpected interrupts is to disable interrupts during execution of the handler. However disabling interrupts will have performance implications. When interrupts are disabled all packet processing is disabled in CPU High. This means no new Ethernet receptions can be processed, no new MPDUs can be submitted to CPU Low for transmission, and no wireless receptions will be processed. It also means all scheduled events will be postponed, including all LTG callbacks and beacon transmissions. You should always minimize the length of time interrupts are disabled.

The MAC High Framework provides helper functions for disabling and restoring the interrupt controller state:

interrupt_state_t curr_interrupt_state;

// Record the current interrupt enable state, then disable all interrupts
curr_interrupt_state = wlan_mac_high_interrupt_stop();

 * Do any interrupt-sensitive processing here

// Restore the interrupt enable state


Both the C code on the node and the Python code on the host operate on little-endian data. However, the wlan_exp transport requires command and response arguments to be big-endian for transfer within the Ethernet frame. Xilinx provides two functions to assist in converting data between big-endian and little-endian:

  • Xil_Ntohl() - "Network to Host" conversion (i.e. converts big-endian to little-endian)
  • Xil_Htonl() - "Host to Network" conversion (i.e. converts little-endian to big-endian)

Therefore, in order to use command arguments within your C code, you must convert each argument from big-endian to little-endian:

arg_0       = Xil_Ntohl(cmd_args_32[0]);              // Swap endianness of command argument

Similarly, in order to send response arguments to the host, you must convert each argument from little-endian to big-endian:

resp_args_32[resp_index++] = Xil_Htonl(status);       // Swap endianness of response arguments

Example Command: Reading Tx Queue Status

This example shows how to implement a custom wlan_exp command, including the required Python and C code. The example command retrieves the status of the transmit (Tx) queues from the WARP node. The upper-level MAC responds to the command with the number of packets currently enqueued in each Tx queue. Because each upper-level MAC (AP, STA, IBSS) handles Tx queues differently the command handler is implemented in the wlan_exp_process_user_cmd() function of the top-level MAC.

Python Code

The first step is to implement a Python method to send the command and process the results. This method is shown below.

# Define a command ID (must be 24-bit integer)

# Define method for sending command and processing the response
def get_tx_queue_status(node, queue_id=None):

    if(queue_id is not None):
        # Arguments must be a list of ints - listify/cast the scaler queue_id
        args = [int(queue_id)]
        args = None

    resp = node.send_user_command(CMDID_USER_TX_QUEUE_STATUS, args)

    # Response should be list of ints with 2 values per queue
    #   - Each pair of response values is (queue_id, queue_occupancy)
    #   - Sanity check the response, then convert to a list of 2-tuples and return
    if(len(resp) % 2 != 0):
        print('ERROR: Tx Queue Status command response lenth ({0} values) was not even!'.format(len(resp)))
        # Group each pair of response values into 2-tuple and return list of 2-tuples
        ret = zip(*[iter(resp)]*2)

    return ret

This method takes one node object as its argument, sends the CMDID_USER_TX_QUEUE_STATUS command to the node, then parses the response. If the response is valid it returns the list of queue id/occupancy values.

You can implement this method wherever it is callable from your Python script. The easiest place is to include the method at the top of your script, then call it directly during the experiment.

C Code

Each high-level MAC application implements its own callback function for handling application-specific user commands. By default these functions are empty. You must modify the C code to implement the behaviors required for your custom command. The code below is for the AP. It is left to the user to implement similar functionality the other MAC applications so that the command works on all types of nodes.

Modify the wlan_exp_process_user_cmd() function in wlan_mac_ap.c. Start by defining the command ID before the function:


Next, add a new case for the your command ID to the switch(cmd_id) statement:

        int            iter;
        u32            req_queue_id;
        u32            q_id, q_occ;
        dl_entry     * curr_station_info_entry;
        station_info * curr_station_info;

        xil_printf("Got Tx queue status cmd\n");

        if(command->header->num_args > 0) {
            req_queue_id = Xil_Ntohl(cmd_args_32[0]);
            xil_printf("Queue ID %d requested\n", req_queue_id);

            resp_args_32[resp_index++] = Xil_Htonl(req_queue_id);
            resp_args_32[resp_index++] = Xil_Htonl(queue_num_queued(req_queue_id));

        } else {
            xil_printf("No queue ID requested - returning status of all queues\n");

            // Gather status of all Tx queues:
            //   Multicast queue (MCAST_QID)
            //   Managment queue (MANAGEMENT_QID)
            //   Queue per AID
            resp_args_32[resp_index++] = Xil_Htonl(MCAST_QID);
            resp_args_32[resp_index++] = Xil_Htonl(queue_num_queued(MCAST_QID));
            resp_args_32[resp_index++] = Xil_Htonl(MANAGEMENT_QID);
            resp_args_32[resp_index++] = Xil_Htonl(queue_num_queued(MANAGEMENT_QID));

            // Note: the following block of code only applies to an AP implementation since
            // the AP maintains a Tx queue for each outgoing station. The STA implementation
            // has a single unicast data queue for all outgoing data transmissions.
            if(my_bss_info->associated_stations.length > 0) {
                iter                    = my_bss_info->associated_stations.length;
                curr_station_info_entry = my_bss_info->associated_stations.first;

                // Iterating over the dl_list is potentially dangerous, as the list itself might change
                // if this function is interrupted. We protect against this by iterating over (at most)
                // the number of entries in the list at the time this loop starts. The iteration variable 
                // is *not* used for indexing the list - we still traverse entries with
                while((curr_station_info_entry != NULL) && (iter-- > 0)) {

                    // Get the station info pointer from the list entry
                    curr_station_info = (station_info*)(curr_station_info_entry->data);

                    // Get the queue ID and queue occupancy
                    q_id = AID_TO_QID(curr_station_info->AID);
                    q_occ = queue_num_queued(q_id);
                    xil_printf("Q: %2d %3d\n", q_id, q_occ);

                    // Add the queue info to the response payload
                    resp_args_32[resp_index++] = Xil_Htonl(q_id);
                    resp_args_32[resp_index++] = Xil_Htonl(q_occ);

                    // Get the next entry in the dl_list
                    curr_station_info_entry = dl_entry_next(curr_station_info_entry);

                    // Break out of the for loop if our response is already reached the max size allowed
                    // by the framework
                    if(resp_index >= max_resp_len) {

        // All done - resp_args_32 now contains queue status, res_index is length of response arguments array
        // Update the response header struct with the length of the response and the number of arguments
        resp_hdr->length  += (resp_index * sizeof(resp_args_32[0]));
        resp_hdr->num_args = resp_index;

Finally, to utilize this new command from your Python script, initialize a node object and call the new get_tx_queue_status(node) method:

# Assume n0 is a wlan_exp node object that has already been initialized
q_stat = get_tx_queue_status(n0)

# Print the queue status response
  print('Tx Queue Status for node {0}'.format(n0.sn_str))
  print('ID Occupancy')

  for q_id,q_occ in q_stat:
      print('{0:2d} {1:3d}'.format(q_id, q_occ))
  print('No Tx Queue Status received for {0}'.format(n0_str))

Low Params

The second flow for extending the experiments framework provides a path for setting parameters in CPU Low. The default 802.11 Reference Design code uses this flow for setting low-level MAC and PHY parameters, such as the RTS Threshold and min/max Contention Window.

A Python script can only write parameters in CPU Low. The script cannot read any data directly from CPU Low. This is by design. The experiments framework C code runs in CPU High. CPU High communicates with CPU Low via the IPC Mailbox. This inter-CPU communication is asynchronous. CPU Low can take an arbitrarily long time to service any new mailbox message from CPU High. The experiments framework cannot block processing in CPU High waiting for a response from CPU Low.

Adding Parameters

Adding new parameters to this flow requires modifying your Python script and the C code for CPU Low.

Each parameter is identified by a unique 32-bit ID value. The Reference Design reserves some parameter IDs. These reserved values are defined in by constant values named with the prefix CMD_PARAM_LOW_PARAM_. User-defined parameters must not collide with any of the existing CMD_PARAM_LOW_PARAM_ values.

We suggest you set the 4 MSB of any new parameters to 0xF. We will never use this range for parameters in the reference code. Do this by defining your new parameters with the form:

// C code
#define CMD_PARAM_LOW_PARAM_NEW_PARAM0 0xF00000000
#define CMD_PARAM_LOW_PARAM_NEW_PARAM1 0xF00000001
// etc...

# Python code
# etc...

The wlan_exp Python node object implements a node.set_low_param(param_id, param_values) method. Your Python script can call this method for any initialized node. The parameter ID and list of values will be sent to the node via Ethernet, passed from CPU High to CPU Low via the mailbox and (eventually) processed by the MAC Low Framework. CPU High acknowledges the command immediately, allowing the Python script to continue execution. There are no guarantees on how long CPU Low might take to actually apply the parameter update.

C Code
Adding a new Low Parameter requires new C code for CPU Low. No changes are required in CPU High.

CPU Low handles Low Parameter messages in two places.

  • Framework: the MAC Low Framework handles parameters in the case IPC_MBOX_LOW_PARAM: clause of the IPC mailbox reception handler. This code is responsible for any framework or PHY parameters which are used by any low-level MAC. In the Reference Design code this handler is implemented in the wlan_mac_low_process_ipc_msg() function in wlan_mac_low.c.
  • MAC: the lower MAC handles parameters specific to the MAC application. Each lower-level MAC implements a callback function which processes any MAC-specific parameters. The framework executes this callback for any parameter not handled by the framework itself. The Reference Design code implements these callbacks in the process_low_param() function in wlan_mac_dcf.c and wlan_mac_nomac.c.

When adding a new parameter you should add a new case to either the framework handler or the MAC code's handler.

Last modified 15 months ago Last modified on Jan 7, 2016, 10:09:18 AM