Changes between Version 9 and Version 10 of WARPLab/Benchmarks


Ignore:
Timestamp:
Aug 22, 2013, 4:38:01 PM (11 years ago)
Author:
welsh
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • WARPLab/Benchmarks

    v9 v10  
    33= WARPLab 7: Benchmarks =
    44
    5 == Jumbo Frames ==
    6 WARPLab 7 supports "jumbo frames," which are Ethernet packets larger than 1500 bytes. WARPLab 7 supports packet sizes up to 9014 bytes long. The benefit of longer packets is that fewer packet exchanges are required in the burst of transmissions that occur when reading or writing large numbers of samples. On our computers, jumbo packets improve the speed for the [wiki:WARPLab/Reference/Commands/Baseband#read_iq read_IQ] and [wiki:WARPLab/Reference/Commands/Baseband#write_iq write_IQ] commands significantly:
     5In order to get a better understanding of WARPLab performance, we have added [wiki:Reference/Utility/#wl_benchmark wl_benchmark] so that you can perform some basic performance testing on WARPLab setup. 
    76
    8 [[Image(jumbo_char.png,width=600)]]
     7For the WARPLab 7.3.0 release, we have provided some Ethernet performance benchmarks since one of the biggest bottlenecks is moving data into and out of the node. 
    98
    10 This plot shows the amount of time each command takes to read or write a full 32768 (2^15^) samples to a buffer. The WARPLab 7 script used to generate the above plot is provided here:
    119
     10== Ethernet Performance ==
     11
     12As part of our on-going development to make WARPLab better and faster, we have introduced a new [wiki:WARPLab/Framework/Modules#Transport Transport] (ie, the [source:ResearchApps/PHY/WARPLAB/WARPLab7/M_Code_Reference/classes/wl_transport_eth_udp_mex.m WARPLab MEX UDP Transport]) to boost communication performance between a host and a node.  For more information on how to install and set up the WARPLab MEX UDP Transport, please refer to the [wiki:WARPLab/QuickStart QuickStart Guide].
     13
     14=== Hardware and Software ===
     15
     16  * Netgear ProSafe 5-port Gigabit Desktop Switch (GS105)
     17  * PC Hardware
     18    * Dell Optiplex 790
     19    * Intel Core i7-2600 3.4 GHz
     20    * 16 GB RAM
     21    * Window 7 Professional SP1 64-bit
     22    * Intel 82579LM Gigabit Network Connection
     23      * Configured to support Jumbo Packets of up to 9014 Bytes
     24  * PC Software
     25    * Matlab 7.12.0 (R2011a)
     26    * Microsoft Visual C++ 2010 Express (compiler for MEX)
     27    * Microsoft Windows 7 SDK
     28[[BR]]
     29=== Procedure ===
     30
     31  * Download Reference bit stream w/ iMPACT
     32  * Open MATLAB
     33    * Set current folder within MATLAB to the M_Code_Reference folder of the appropriate WARPLab release
     34    * Run the following commands
     35      * wl_setup
     36        * NOTE:  This allows us to configure the Transport type and the Ethernet Packet Size
     37      * nodes = wl_initNodes(1)
     38      * results = wl_benchmark( nodes, 'write_iq', 10000, 'max' );
     39      * results = wl_benchmark( nodes, 'read_iq', 10000, 'max' );
     40[[BR]]
     41=== Results ===
     42
     43''' WARP v3 Hardware '''
    1244{{{
    13 clear node
    14 node = wl_initNodes(1);
    15 [RFA,RFB] = wl_getInterfaceIDs(node);
     45------------------------------------------------------------------------------------------------------------------------------------
     46|  ID |  WLVER |    Serial # |   Transport | Pkt Size (B) | Num Samples | Num Trials | Avg Write IQ per sec |  Avg Read IQ per sec |
     47------------------------------------------------------------------------------------------------------------------------------------
     48|   0 |  7.3.0 |  W3-a-00027 |  WL Mex UDP |         1464 |       32768 |      10000 |             210.4994 |             232.6682 |
     49|   0 |  7.3.0 |  W3-a-00027 |    Java UDP |         1464 |       32768 |      10000 |              17.0187 |              16.9429 |
     50|   0 |  7.1.0 |  W3-a-00027 |    Java UDP |         1464 |       32768 |      10000 |              16.9390 |              16.6640 |
     51------------------------------------------------------------------------------------------------------------------------------------
     52|   0 |  7.3.0 |  W3-a-00027 |  WL Mex UDP |         8960 |       32768 |      10000 |             445.8885 |             466.6237 |
     53|   0 |  7.3.0 |  W3-a-00027 |    Java UDP |         8960 |       32768 |      10000 |              82.0430 |              71.2319 |
     54|   0 |  7.1.0 |  W3-a-00027 |    Java UDP |         8960 |       32768 |      10000 |              73.1989 |              44.6655 |
     55------------------------------------------------------------------------------------------------------------------------------------
     56}}}
    1657
    17 NUMITER = 100;
    1858
    19 NUMLENGTHS = 6;
    20 maxPayloadVec = [1000,floor(linspace(1470,double(node.transport.maxPayload),NUMLENGTHS-1))];
    2159
    22 IQvec = complex(randn(node.baseband.txIQLen,1),randn(node.baseband.txIQLen,1));
    23 IQvec = IQvec./(max(abs(IQvec)));
    2460
    25 duration_write = zeros(NUMITER,NUMLENGTHS);
    26 duration_read = zeros(NUMITER,NUMLENGTHS);
    27 mean_duration_write = zeros(1,NUMLENGTHS);
    28 std_duration_write = zeros(1,NUMLENGTHS);
    29 mean_duration_read = zeros(1,NUMLENGTHS);
    30 std_duration_read = zeros(1,NUMLENGTHS);
    31 for k = NUMLENGTHS:-1:1
    32     fprintf('Current maxPayload: %d\n',maxPayloadVec(k));
    33     node.transport.maxPayload = maxPayloadVec(k);
    34     for m = NUMITER:-1:1
    35         currTime = tic;
    36         wl_basebandCmd(node,RFA,'write_IQ',IQvec);
    37         duration_write(m,k) = toc(currTime);
    38         currTime = tic;
    39         wl_basebandCmd(node,RFA,'read_IQ',0,node.baseband.rxIQLen);
    40         duration_read(m,k) = toc(currTime);
    41     end
    42     mean_duration_write(k) = mean(duration_write(:,k));
    43     std_duration_write(k) = std(duration_write(:,k));
    44     mean_duration_read(k) = mean(duration_read(:,k));
    45     std_duration_read(k) = std(duration_read(:,k));
    46 end
     61''' WARP v2 Hardware '''
     62{{{
     63------------------------------------------------------------------------------------------------------------------------------------
     64|  ID |  WLVER |    Serial # |   Transport | Pkt Size (B) | Num Samples | Num Trials | Avg Write IQ per sec |  Avg Read IQ per sec |
     65------------------------------------------------------------------------------------------------------------------------------------
     66|   0 |  7.3.0 |         N/A |  WL Mex UDP |         1464 |       16384 |      10000 |             128.1127 |             133.0076 |
     67|   0 |  7.3.0 |         N/A |    Java UDP |         1464 |       16384 |      10000 |              32.9602 |              31.1658 |
     68|   0 |  7.1.0 |         N/A |    Java UDP |         1464 |       16384 |      10000 |              32.5807 |              22.0276 |
     69------------------------------------------------------------------------------------------------------------------------------------
     70}}}
    4771
    48 %% Plotting
    49 figure(1);clf
    50 errorbar(maxPayloadVec,mean_duration_write,std_duration_write,'LineWidth',2)
    51 hold on
    52 errorbar(maxPayloadVec,mean_duration_read,std_duration_read,'r','LineWidth',2)
    53 grid on
    54 xlabel('Payload Size')
    55 ylabel('Duration (Seconds)')
    56 title('Duration of Commands vs Payload Size')
    57 legend('write_IQ','read_IQ')
    58 myAxis = axis;
    59 myAxis(3) = 0;
    60 axis(myAxis);
    61 
    62 standardPayload = 1470;
    63 H = line([standardPayload standardPayload],[myAxis(3),myAxis(4)]);
    64 set(H,'Color',[0 0 0])
    65 set(H,'LineWidth',2)
    66 H = text(standardPayload + .01*myAxis(2),.9*myAxis(4),'Standard Payload Size');
    67 set(H,'FontName','Ariel')
    68 set(H,'FontWeight','bold')
    69 hold off
    70 
    71 }}}