Changes between Version 9 and Version 10 of WARPLab/Debugging

Timestamp:

Nov 18, 2015, 3:42:22 PM (8 years ago)

Author:

welsh

Comment:

--

WARPLab/Debugging

@@ -193 +193 @@
 where X.Y.Z shows your WARPLab version; AAAAA is your hardware serial number; and NN:NN:NN are the last six digits of your MAC address.
 
-  
-
-
 
 ----
@@ -235 +232 @@
 }}}
 
+
 ----
 == ERROR:  Broadcast Triggers ==
@@ -257 +255 @@
 }}}
 
+
 ----
 == ERROR:  DMA transfers ==
@@ -270 +269 @@
 To resolve this error update your linker script to assign data sections to a memory area accessible by the AXI interconnect. The axi_bram (listed as {{{axi_bram_0_S_AXI_BASEADDR}}} in the current design) is a good choice.
 
-----
-
-
+
+----
+== ERROR:  MEX Transport out of memory ==
+
+When using [http://warpproject.org/trac/wiki/WARPLab/BufferSizes large buffers], it is possible for Matlab to run out of memory when trying to process a Read IQ request: 
+
+{{{
+Error using wl_mex_udp_transport
+Out of memory. Type HELP MEMORY for your options.
+
+Error in wl_transport_eth_udp_mex/read_buffers (line 545)
+                    [num_rcvd_samples, cmds_used, rx_samples]  =
+                    wl_mex_udp_transport('read_iq', obj.sock, data8,
+                    length(data8), obj.address, obj.port, num_samples,
+                    buffer_ids, start_sample, obj.maxS
+Error in wl_baseband_buffers>readIQ (line 1393)
+        rxSamples_IQ = node.transport.read_buffers('IQ', numSamps, buffSel,
+        offset, myCmd, 0);
+
+Error in wl_baseband_buffers/procCmd (line 576)
+                    out = readIQ(obj, node, buffSel, cmdStr, varargin{:});
+}}}
+
+Each IQ sample is stored in Matlab as a complex double.  This means that each sample requires 16 bytes of memory to be stored in Matlab.  Now, if a Read IQ request of a single buffer is transferring 2^25^ samples, then this implies that an array of those samples requires 2^29^ bytes (i.e. 512 MB).  Similarly, if a Read IQ request of 2 buffers is transferring 2^25^ samples per buffer, then this implies that an array of those samples requires 2^30^ bytes (i.e. 1 GB).  By using the ''''memory'''' command in Matlab, you can see the maximum possible array size for your installation:
+{{{
+Maximum possible array:              XXXXX MB (Y.YYYe+ZZZ bytes)
+}}}
+This will allow you to understand the maximum number of samples you can request in a single Read IQ command.
+
+
+----
+== ERROR: Transport only supports X samples==
+
+When using [http://warpproject.org/trac/wiki/WARPLab/BufferSizes large buffers], even though WARPLab allows for up to 2^27^ samples (ie 128M samples), requesting greater than 2^25^ samples when using the MEX transport or greater than 2^20^ samples when using the Java transport will result in the following error messages:
+
+{{{
+??? Error using ==> wl_baseband_buffers>readIQ at 1989
+read_iq: Requested 67108864 samples.  Due to Matlab memory limitations, the mex transport only supports 33554432 samples.
+    If your computer has enough physical memory, you can adjust this limit using node.baseband.MEX_TRANSPORT_MAX_IQ.
+}}}
+
+{{{
+??? Error using ==> wl_baseband_buffers>readIQ at 2003
+read_iq: Requested 67108864 samples.  Due to performance reasons, the java transport only supports 1048576 samples.
+    Please use the MEX transport for larger requests.
+}}}
+
+The error in the Java transport is issued because, as seen in the [http://warpproject.org/trac/wiki/WARPLab/Benchmarks benchmarks], the Java transport is much slower than the MEX transport so larger requests can take a "long time".  Therefore, it is recommended to move to the MEX transport for larger requests.
+
+The error in the MEX transport is issued because many host computers do not have enough memory to handle longer transfers.  As discussed above, it requires 2^29^ bytes (ie 512 MB) to hold 2^25^ samples.  As noted in the error message, if your host computer has more memory (can be seen using the Matlab ''''memory'''' command), then you can adjust the maximum using the MEX_TRANSPORT_MAX_IQ variable.
+
+
+----
+== WARNING: Read IQ / Read RSSI request timed out ==
+
+When performing a Read IQ or Read RSSI command, you might see the following warning:
+
+{{{
+WARNING:  Read IQ / Read RSSI request timed out.  Retrying remaining samples. 
+          If this message occurs frequently, please adjust the Read IQ 
+          maximum request size (in bytes) for the transport using the 
+          M code function:  
+              wl_mex_udp_transport('read_iq_set_max_request_size', size)  
+          Defaults to 80 percent of the receive buffer allocated by the OS. 
+
+          To suppress all IQ warnings for the transport use the M code function: 
+              wl_mex_udp_transport('suppress_iq_warnings')
+}}}
+
+When Matlab initializes the socket during initNodes, the code requests an Ethernet receive buffer from the OS for 2^22^ bytes, ie 4 MB (see REQUESTED_BUF_SIZE in the open() function of the transport, for example wl_transport_eth_udp_mex.m or wl_transport_eth_udp_java.m).  This value was chosen because it gives good performance and most OSes are able to allocate an Ethernet receive buffer that large.  Then, by default, WARPLab only allows requests to the node that are 80% of that receive buffer (~3.35 MB) so that the node cannot overwhelm the host with data.  
+
+However, if you see the warning above, this means that packets are being dropped during the Read IQ / Read RSSI command.  This could come from a number of sources:
+
+  1. Intermittent / flaky Ethernet hardware such as a bad Ethernet cable, Ethernet switch, or NIC.  These can be tested by swapping components and seeing if that resolves the warning.  
+  2. Even with the built in buffer overhead, the host still cannot keep up with the flow of data packets from the node and begins dropping packets.  This could be due to the hardware specifications of the host.  In most systems, if WARPLab requests a 4 MB socket buffer from the OS, and the OS cannot provide that much buffer space, then the OS will allocate a smaller buffer and tell WARPLab how much buffer space it can actually use.  Then, the maximum request to the node that transport issues will be adjusted accordingly and there are no issues.  However, if the OS tells WARPLab that a 4 MB socket buffer is available but in reality there is much less than 4 MB available (i.e. the OS begins to drop Ethernet packets way before 4 MB of data is transferred), then the transport can run into problems.  When a situation like this occurs, you can use the command '''wl_mex_udp_transport('read_iq_set_max_request_size', size)''' to set the size of the request, in bytes.  This should be less that (0.8 * 2^22^) bytes, or (0.8 * <value OS reduced recv buffer printed during initNodes>), since you want to reduce the maximum size of the request.  
+
+As long as you just receive the warning and there are no Read IQ / Read RSSI errors, then all this does is slow down the commands (ie you will not be getting the best performance).  If you are ok with the reduced performance and wish to remove the warning, you can use the command '''wl_mex_udp_transport('suppress_iq_warnings')'''.
+
+
+