WARP Project Forums - Wireless Open-Access Research Platform

You are not logged in.

#1 2015-Feb-28 13:22:53

adherentx
Member
Registered: 2014-Oct-23
Posts: 45

A problem in extending log entries

I am working based on 802.11 ref. design v1.0 on WARP v3 boards.

I want to log extra information when receiving a packet, so I add a log entry type like the following.

Code:

typedef struct{
	rx_common_entry     rx_entry;
	u32	                channel_est[64];         // Channel estimates for the packet
	u32			mydata1[256];
	u32			mydata2[256];
	u32                 mac_payload_log_len;     // Number of payload bytes actually recorded in log entry
        u32                 mac_payload[MIN_MAC_PAYLOAD_LOG_LEN/4];
	
} rx_ofdm_with_my_data_entry;

When I receive a packet, I followed the following order to log the packet.

Code:

1. copy payload (CDMA transfer)
2. fill in Log Entries like timestamp (one-by-one assignment)
3. copy channel estimate (CDMA transfer)
4. copy my data (CDMA transfer)

(CDMA transfer)
I use two lines of code like the following to copy my data from brams to log buffer.

Code:

wlan_mac_high_cdma_start_transfer((u8*)(((rx_ofdm_with_my_data_entry*)rx_event_log_entry)->mydata1), (void*)mydata1_bram_addr, 256*4);
wlan_mac_high_cdma_start_transfer((u8*)(((rx_ofdm_with_my_data_entry*)rx_event_log_entry)->mydata2), (void*)mydata2_bram_addr, 256*4);

I think it is trivial, but things don't run correctly. When I turn two WARP nodes on, they cannot even get associated.
I tried to dig into this problem. I found that CDMA errors occurred in the wlan_exp_log_create_tx_entry(), with error code 15 (I checked and found that it is because the TX packet length is set as 0). Very strangely, error in TX rather than RX. But I am sure the problem lies in the newly added custom RX log entry, because there are no problems when I don't use the new entry.
Then, I tried to use a for-loop to copy my data one by one like the following.

Code:

for(li = 0; li < 256; li++){
	((rx_ofdm_with_my_data_entry*)rx_event_log_entry)->mydata1[li] = ((u32*)mydata1_bram_addr)[li];
	((rx_ofdm_with_my_data_entry*)rx_event_log_entry)->mydata2[li] = ((u32*)mydata2_bram_addr)[li];
}

It works well.
I think the problem should lie in the CDMA transfer. I cannot figure out what is wrong. Do you have any ideas? Thanks.

Last edited by adherentx (2015-Mar-01 00:02:22)

Offline

 

#2 2015-Feb-28 13:42:56

murphpo
Administrator
From: Mango Communications
Registered: 2006-Jul-03
Posts: 5159

Re: A problem in extending log entries

What is the memory address for mydata1_bram_addr and mydata2_bram_addr?

One possibility is that the axi_cdma core cannot access the memory where you've stored these arrays. Specifically the cdma cannot access the ILMB/DLMB memory areas. These are only accessible by the MicroBlaze memory interface.

Do you see any output on the UART when the DMA transfer fails?

Offline

 

#3 2015-Feb-28 14:00:53

murphpo
Administrator
From: Mango Communications
Registered: 2006-Jul-03
Posts: 5159

Re: A problem in extending log entries

One other thought- are the mydata1_bram_addr and mydata2_bram_addr arrays created by CPU Low? If so, in which memory block? The only memory block accessible by both CPU Low and the axi_cmda are the Tx and Rx packet buffers.

Offline

 

#4 2015-Feb-28 14:11:37

adherentx
Member
Registered: 2014-Oct-23
Posts: 45

Re: A problem in extending log entries

murphpo wrote:

One other thought- are the mydata1_bram_addr and mydata2_bram_addr arrays created by CPU Low? If so, in which memory block? The only memory block accessible by both CPU Low and the axi_cmda are the Tx and Rx packet buffers.

I added the new brams in XPS, following the same setting as TX and RX brams. I connected the brams to a bus called **shared**(sorry, I forget the name and I don't have access to my working computer now.), just like the RX and TX buffers.
I think both CPU high and CPU low can access the two brams since I see then listed both in high and low in the 'address' page of XPS. However, I am not sure whether axi_cdma have access to the two new brams.
So, how can I know whether axi_cdma have access to the brams? How can I grant axi_cdma the access to the brams?

Last edited by adherentx (2015-Feb-28 14:36:44)

Offline

 

#5 2015-Feb-28 14:14:59

adherentx
Member
Registered: 2014-Oct-23
Posts: 45

Re: A problem in extending log entries

murphpo wrote:

What is the memory address for mydata1_bram_addr and mydata2_bram_addr?

One possibility is that the axi_cdma core cannot access the memory where you've stored these arrays. Specifically the cdma cannot access the ILMB/DLMB memory areas. These are only accessible by the MicroBlaze memory interface.

Do you see any output on the UART when the DMA transfer fails?

Actually, I don't know whether the DMA transfer fails or not. There is no output about the DMA transfer.
BTW. I changed my program a bit before posting it to make it easier to read. My original program is aimed for recoding phy raw datas, so the 'my_data' shown in my code is actually of size 16kB. Does the size matter?

Last edited by adherentx (2015-Feb-28 14:40:38)

Offline

 

#6 2015-Feb-28 15:35:05

murphpo
Administrator
From: Mango Communications
Registered: 2006-Jul-03
Posts: 5159

Re: A problem in extending log entries

The problem might be that the axi_cdma cannot access the new memory blocks because of the AXI interconnect settings.

The 'mb_shared_axi' instance of the axi_interconnect core is implemented as a crossbar. You must explicitly set which masters are able to access each slave. In the reference design we do not connect every master to every slave, to improve resource usage and timing. You should follow this approach when adding custom cores to the mb_shared_axi interconnect- only make the necessary master/slave connections.

To modify which masters can access a given slave:
-Find your slave core instance in the Bus Interfaces tab
-Expand the slave device entry (click the + to the left of the core instance)
-Find the S_AXI interface under your core
-Click the button labeled 'mb_shared_axi...' under the Bus Name column

This will show a dialog box with check boxes for all the master devices on the mb_shared_axi interconnect, including the 'axi_cdma_0.M_AXI' master. Look at the bus interface settings for the Tx packet buffer ('pkt_buff_TX_bram_ctrl) for a good example.

You can make the same connections by manually editing the system.mhs file (System Assembly View in XPS is just a GUI for editing system.mhs). Look at the parameter named C_INTERCONNECT_S_AXI_MASTERS in your new memory block instances.

Offline

 

#7 2015-Mar-01 00:41:51

adherentx
Member
Registered: 2014-Oct-23
Posts: 45

Re: A problem in extending log entries

murphpo wrote:

The problem might be that the axi_cdma cannot access the new memory blocks because of the AXI interconnect settings.

The 'mb_shared_axi' instance of the axi_interconnect core is implemented as a crossbar. You must explicitly set which masters are able to access each slave. In the reference design we do not connect every master to every slave, to improve resource usage and timing. You should follow this approach when adding custom cores to the mb_shared_axi interconnect- only make the necessary master/slave connections.

To modify which masters can access a given slave:
-Find your slave core instance in the Bus Interfaces tab
-Expand the slave device entry (click the + to the left of the core instance)
-Find the S_AXI interface under your core
-Click the button labeled 'mb_shared_axi...' under the Bus Name column

This will show a dialog box with check boxes for all the master devices on the mb_shared_axi interconnect, including the 'axi_cdma_0.M_AXI' master. Look at the bus interface settings for the Tx packet buffer ('pkt_buff_TX_bram_ctrl) for a good example.

You can make the same connections by manually editing the system.mhs file (System Assembly View in XPS is just a GUI for editing system.mhs). Look at the parameter named C_INTERCONNECT_S_AXI_MASTERS in your new memory block instances.

Problem solved. Thanks.
I checked mb_low, mb_high, and mb_high_eth when I was configuring bus connection of the new brams in XPS. I did notice the CDMA, but I didn't check it because I thought RX and TX buffer may need CDMA but my new brams may not need it. I was so foolish that I didn't recall it when I met with the CDMA transfer problem.
Now, a new question:
I think the CDMA transfer is non-blocking: after I call the function wlan_mac_high_cdma_start_transfer(), it will return in a short time, before the real transfer is completed. Am I right? If so, then write/read timing becomes a problem. I stop writing in the brams before I start CDMA transfer, to ensure may data is not changing when I copy the data from the brams to log buffer. However, when should I re-enable writing to the brams?

Offline

 

#8 2015-Mar-01 11:12:22

adherentx
Member
Registered: 2014-Oct-23
Posts: 45

Re: A problem in extending log entries

deleted.

Last edited by adherentx (2015-Mar-02 08:49:29)

Offline

 

#9 2015-Mar-02 09:27:05

welsh
Administrator
From: Mango Communications
Registered: 2013-May-15
Posts: 612

Re: A problem in extending log entries

If you look at the CDMA code, you can see that the calls to wlan_mac_high_cdma_start_transfer() will return before the transfer is done.  However, subsequent calls to the function will wait until the CDMA is done with its current transfer before starting the next transfer.  This was so that you could have the CPU work on other code while the DMA is transferring data.

If you are concerned about read / write timing issues between the DMA and other HW, then you can just add another wlan_mac_high_cdma_finish_transfer() after your call to wlan_mac_high_cdma_start_transfer() to ensure that the DMA is complete and this will cause the CPU to wait until the DMA transfer is complete (or you can add the call right before you want to re-enable writing to the memory so that the CPU can do as much as possible before waiting on the DMA).

Offline

 

#10 2015-Mar-02 10:24:36

adherentx
Member
Registered: 2014-Oct-23
Posts: 45

Re: A problem in extending log entries

Ok Thanks. Another 2 problems.
1) I find that the rx_common_entry can be logged correctly when the size of phy_raw_I[] and phy_raw_Q[] are too large, like 64kB. Things starts to be messed up from phy_raw_I. I used wlan_exp to extract the logged data and plot in Matlab. The 'phy_raw_I' is like the following figure. You can see that from somewhere all data become zero, and actually all phy_raw_Q data are zero. All mac_payload data are zero as well. However, when the size of phy_raw_I[] and phy_raw_Q[] are not too large, say 5kB, there is no this problem.
2) For every packet I receive, I log the raw data captured in HW when receiving the packet, with length N_raw. I make sure N_raw is larger than the packet length, and I take the decoding time into account as well. I also make sure the interval between two packet transmission is very large, such that I will not log the raw data from an old packet. After extracting the raw data in wlan_exp, I plot the raw data of each packet reception in Matlab. I expected each one should be a figure where some parts are nearly zero (only noise), and only one continuous interval has values much larger than 0 (and less than 1). However, what I get is the following. The N_raw is 1000 samples. LTG payload length is 50 bytes. The rate is 18Mbps - QPSK 3/4.
http://s3.postimg.org/ghb1li5s1/raw_I.jpg
I have tried N_raw = 5000, and payload is still 50 bytes. The result is similar to the above figure.
In the RX hardware, I capture the raw data like the following figure.
http://s23.postimg.org/q7byjomtl/diagram.jpg
Inside the 'Raw Data Capture' block is the following.
http://s12.postimg.org/tanw61l0c/raw_hw.jpg

Last edited by adherentx (2015-Mar-02 11:15:55)

Offline

 

#11 2015-Mar-02 11:45:16

welsh
Administrator
From: Mango Communications
Registered: 2013-May-15
Posts: 612

Re: A problem in extending log entries

1)  Unfortunately, you are running into a fundamental size limitation within the logging framework.  Each log entry has a header that describes the log entry.  As you can see from the definition, we made the length field 16 bits (ie each log entry can be up to 64 kB).  Therefore, when you try to create log entries greater than 64 kB, things break down.  To get around this limitation, just define your own entry type that has a sequence number and total number of items and then break up your data across multiple log entries.  For example, have a u32 with total data packets, and a u32 with current data packet, then have have enough data that the total entry size stays under 64 kB:

Code:

typedef struct{
	rx_common_entry     rx_entry;
	u32                 channel_est[64];         // Channel estimates for the packet
        u32                 packet_num;
        u32                 total_packets; 
	u32                 mydata1[4096];         // 16 kB of my data
	u32                 mydata2[4096];         // 16 kB of my data
	u32                 mac_payload_log_len;     // Number of payload bytes actually recorded in log entry
        u32                 mac_payload[MIN_MAC_PAYLOAD_LOG_LEN/4];
	
} rx_ofdm_with_my_data_entry;

Obviously, you could make the mydata* entries a bit larger, you just need to make sure that the size of the entire entry is under 64kB.  Also, I would not try to pack as much as possible into a single log entry and instead opt for breaking the data on larger alignment boundaries, eg. 512B boundaries or larger so you maintain DMA transfer efficiencies (64 beat burst at 64 bits per burst).  Then in python, you can re-create the longer packet data when you process your log entries. 


2) We are not able to see your figures.  The forum site cannot host images and you must post images somewhere else and then link to them in the forum post.  I'm not sure but this thread might be helpful.

Offline

 

#12 2015-Mar-02 11:55:17

adherentx
Member
Registered: 2014-Oct-23
Posts: 45

Re: A problem in extending log entries

I think that thread is irrelevant.
I did post my figures on another website and link to them here. Can you try clicking the following three Links?
http://s3.postimg.org/ghb1li5s1/raw_I.jpg
http://s23.postimg.org/q7byjomtl/diagram.jpg
http://s12.postimg.org/tanw61l0c/raw_hw.jpg

Last edited by adherentx (2015-Mar-02 11:55:38)

Offline

 

#13 2015-Mar-02 12:49:51

welsh
Administrator
From: Mango Communications
Registered: 2013-May-15
Posts: 612

Re: A problem in extending log entries

Sorry.  Didn't realize that your figure just looked like noise :)

I think you are having issues with data formatting.  If you look, the output of the "Sync & Antenna Sel" block is "Fix_16_15".  From the pictures, you take this Fix_16_15 data and put it into the memory as UFix_32_0 (you can check the data types but I assume you are padding zeros in the upper 16 bits as part of 'Register 6' and 'Register 7').  At the host side, you need to then convert the UFix_32_0 to Fix_16_15 (mask off the upper 16 bits) and then convert the Fix_16_15 to a double.  You can see an example of how to do this in in C code or in M code.  If you are doing the conversion correctly, one other thing to check is that the endianness of your data is correct.  We have found that using a counter to generate known data is a good way to test schemes like this (you can see an example of this in the WARPLab Buffers sysgen core; See the "COUNTER_DATA_SEL" line in w3_warplab_buffers/RFA Inputs).

Offline

 

#14 2015-Mar-02 12:57:16

adherentx
Member
Registered: 2014-Oct-23
Posts: 45

Re: A problem in extending log entries

welsh wrote:

Sorry.  Didn't realize that your figure just looked like noise :)

I think you are having issues with data formatting.  If you look, the output of the "Sync & Antenna Sel" block is "Fix_16_15".  From the pictures, you take this Fix_16_15 data and put it into the memory as UFix_32_0 (you can check the data types but I assume you are padding zeros in the upper 16 bits as part of 'Register 6' and 'Register 7').  At the host side, you need to then convert the UFix_32_0 to Fix_16_15 (mask off the upper 16 bits) and then convert the Fix_16_15 to a double.  You can see an example of how to do this in in C code or in M code.  If you are doing the conversion correctly, one other thing to check is that the endianness of your data is correct.  We have found that using a counter to generate known data is a good way to test schemes like this (you can see an example of this in the WARPLab Buffers sysgen core; See the "COUNTER_DATA_SEL" line in w3_warplab_buffers/RFA Inputs).

But it is signed 16_15. I cast it to be 32_31. I think zeroes will be padded after 16_15 for positive number? And for negative number, the converter will also work correctly to make it 32_31.

Last edited by adherentx (2015-Mar-02 13:22:17)

Offline

 

#15 2015-Mar-02 14:56:17

welsh
Administrator
From: Mango Communications
Registered: 2013-May-15
Posts: 612

Re: A problem in extending log entries

I couldn't see the conversion in the picture.  If you converted it to Fix_32_31, then you shouldn't have to worry about masking any bits and you are correct that it will just pad the bottom 16 bits.  Just make sure that you correctly convert back to a double using a similar method that I referred to in the previous post.

If you are already converting correctly, then there is probably an endian problem.

Offline

 

Board footer