|
|
|
|
HPM Counter Statistics | ||||
---|---|---|---|---|
Event | Ntasks | Avg | Min(rank) | Max(rank) |
Communication Event Statistics (0.00% detail, 2.8688e+01 error) | |||||||
---|---|---|---|---|---|---|---|
Buffer Size | Ncalls | Total Time | Min Time | Max Time | %MPI | %Wall | |
MPI_Allreduce | 8 | 23544 | 23.619 | 1.383e-05 | 6.081e-02 | 82.33 | 23.35 |
MPI_Testall | 0 | 20355863 | 13.267 | 0.000e+00 | 8.636e-03 | 46.25 | 13.12 |
MPI_Iprobe | 0 | 38950459 | 10.331 | 0.000e+00 | 4.790e-03 | 36.01 | 10.21 |
MPI_Waitall | 57344 | 5375 | 6.113 | 2.790e-05 | 1.298e-02 | 21.31 | 6.04 |
MPI_Waitall | 65536 | 3891 | 4.999 | 2.289e-05 | 1.253e-02 | 17.43 | 4.94 |
MPI_Waitall | 49152 | 2774 | 3.234 | 2.909e-05 | 1.308e-02 | 11.27 | 3.20 |
MPI_Test | 0 | 18370562 | 3.490 | 0.000e+00 | 1.199e-04 | 12.17 | 3.45 |
MPI_Waitall | 8192 | 3077 | 0.284 | 3.099e-06 | 2.241e-03 | 0.99 | 0.28 |
MPI_Waitall | 16384 | 959 | 0.339 | 4.053e-06 | 6.507e-03 | 1.18 | 0.34 |
MPI_Allreduce | 4 | 15336 | 2.413 | 1.192e-05 | 1.048e-02 | 8.41 | 2.39 |
MPI_Waitall | 7168 | 1249 | 0.112 | 1.907e-06 | 1.164e-03 | 0.39 | 0.11 |
MPI_Send | 28 | 120308 | 1.662 | 0.000e+00 | 2.702e-03 | 5.79 | 1.64 |
MPI_Waitall | 20480 | 432 | 0.367 | 9.060e-06 | 7.369e-03 | 1.28 | 0.36 |
MPI_Recv | 8 | 138614 | 1.296 | 0.000e+00 | 1.992e-02 | 4.52 | 1.28 |
MPI_Waitall | 14336 | 252 | 0.064 | 3.815e-06 | 1.126e-03 | 0.22 | 0.06 |
MPI_Waitall | 32768 | 3921 | 1.006 | 4.053e-06 | 1.209e-02 | 3.51 | 0.99 |
MPI_Isend | 8 | 267070 | 0.675 | 0.000e+00 | 4.550e-03 | 2.35 | 0.67 |
MPI_Waitall | 28672 | 4815 | 0.904 | 3.815e-06 | 5.073e-03 | 3.15 | 0.89 |
MPI_Waitall | 10240 | 2268 | 0.256 | 4.053e-06 | 2.257e-03 | 0.89 | 0.25 |
MPI_Waitall | 2048 | 2091 | 0.124 | 2.146e-06 | 3.869e-03 | 0.43 | 0.12 |
MPI_Waitall | 512 | 1641 | 0.165 | 2.861e-06 | 4.759e-04 | 0.57 | 0.16 |
MPI_Waitall | 6144 | 1346 | 0.077 | 1.907e-06 | 1.990e-03 | 0.27 | 0.08 |
MPI_Isend | 12288 | 59452 | 0.480 | 0.000e+00 | 7.801e-04 | 1.67 | 0.47 |
MPI_Waitall | 4096 | 2881 | 0.320 | 3.099e-06 | 4.246e-03 | 1.12 | 0.32 |
MPI_Waitall | 640 | 1647 | 0.173 | 2.861e-06 | 1.353e-03 | 0.60 | 0.17 |
MPI_Waitall | 12288 | 469 | 0.136 | 3.815e-06 | 5.052e-03 | 0.47 | 0.13 |
MPI_Waitall | 2560 | 1055 | 0.072 | 2.146e-06 | 2.028e-03 | 0.25 | 0.07 |
MPI_Waitall | 24576 | 2491 | 0.598 | 3.099e-06 | 7.716e-03 | 2.08 | 0.59 |
MPI_Waitall | 3584 | 1574 | 0.252 | 5.960e-06 | 4.132e-03 | 0.88 | 0.25 |
MPI_Waitall | 1792 | 2182 | 0.112 | 2.146e-06 | 6.921e-04 | 0.39 | 0.11 |
MPI_Waitall | 1024 | 1217 | 0.169 | 1.907e-06 | 1.794e-03 | 0.59 | 0.17 |
MPI_Waitall | 1280 | 1120 | 0.131 | 9.537e-07 | 1.914e-03 | 0.46 | 0.13 |
MPI_Waitall | 64 | 190 | 0.023 | 9.537e-07 | 4.280e-04 | 0.08 | 0.02 |
MPI_Waitall | 768 | 1616 | 0.153 | 1.097e-05 | 1.114e-03 | 0.54 | 0.15 |
MPI_Waitall | 448 | 526 | 0.061 | 4.053e-06 | 6.230e-04 | 0.21 | 0.06 |
MPI_Send | 4 | 103680 | 0.357 | 0.000e+00 | 3.490e-04 | 1.24 | 0.35 |
MPI_Waitall | 80 | 260 | 0.029 | 8.106e-06 | 5.989e-04 | 0.10 | 0.03 |
MPI_Waitall | 320 | 741 | 0.150 | 6.914e-06 | 1.392e-03 | 0.52 | 0.15 |
MPI_Waitall | 256 | 858 | 0.175 | 2.861e-06 | 1.223e-03 | 0.61 | 0.17 |
MPI_Waitall | 1536 | 1006 | 0.072 | 4.053e-06 | 1.288e-03 | 0.25 | 0.07 |
MPI_Waitall | 384 | 649 | 0.100 | 1.192e-06 | 1.122e-03 | 0.35 | 0.10 |
MPI_Isend | 16 | 43940 | 0.044 | 0.000e+00 | 7.606e-05 | 0.15 | 0.04 |
MPI_Waitall | 3072 | 281 | 0.131 | 1.907e-06 | 4.921e-03 | 0.46 | 0.13 |
MPI_Isend | 4 | 261319 | 0.258 | 0.000e+00 | 8.769e-04 | 0.90 | 0.25 |
MPI_Waitall | 5120 | 1468 | 0.083 | 5.007e-06 | 1.235e-03 | 0.29 | 0.08 |
MPI_Waitall | 896 | 1034 | 0.129 | 2.146e-06 | 1.600e-03 | 0.45 | 0.13 |
MPI_Isend | 6144 | 55940 | 0.236 | 0.000e+00 | 9.513e-05 | 0.82 | 0.23 |
MPI_Waitall | 81920 | 427 | 0.230 | 2.098e-05 | 2.633e-03 | 0.80 | 0.23 |
MPI_Waitall | 160 | 371 | 0.060 | 5.960e-06 | 7.610e-04 | 0.21 | 0.06 |
MPI_Waitall | 393216 | 191 | 0.207 | 3.140e-04 | 2.777e-03 | 0.72 | 0.20 |
MPI_Waitall | 262144 | 213 | 0.198 | 4.506e-05 | 2.420e-03 | 0.69 | 0.20 |
MPI_Waitall | 192 | 329 | 0.051 | 9.537e-07 | 8.619e-04 | 0.18 | 0.05 |
MPI_Scan | 4 | 1728 | 0.191 | 2.289e-05 | 1.190e-03 | 0.67 | 0.19 |
MPI_Waitall | 128 | 325 | 0.047 | 1.192e-06 | 5.631e-04 | 0.16 | 0.05 |
MPI_Waitall | 96 | 208 | 0.025 | 9.537e-07 | 5.650e-04 | 0.09 | 0.02 |
MPI_Waitall | 224 | 540 | 0.095 | 4.053e-06 | 1.039e-03 | 0.33 | 0.09 |
MPI_Irecv | 12288 | 59452 | 0.106 | 0.000e+00 | 4.969e-04 | 0.37 | 0.10 |
MPI_Waitall | 196608 | 238 | 0.154 | 1.290e-04 | 1.776e-03 | 0.54 | 0.15 |
MPI_Isend | 24 | 24991 | 0.027 | 0.000e+00 | 7.701e-05 | 0.09 | 0.03 |
MPI_Waitall | 524288 | 96 | 0.143 | 5.889e-04 | 2.488e-03 | 0.50 | 0.14 |
MPI_Waitall | 98304 | 418 | 0.142 | 2.217e-05 | 1.856e-03 | 0.50 | 0.14 |
MPI_Waitall | 131072 | 305 | 0.142 | 2.909e-05 | 1.582e-03 | 0.49 | 0.14 |
MPI_Waitall | 327680 | 101 | 0.120 | 3.359e-04 | 1.952e-03 | 0.42 | 0.12 |
MPI_Isend | 1536 | 31636 | 0.039 | 0.000e+00 | 7.987e-05 | 0.14 | 0.04 |
MPI_Waitall | 112 | 159 | 0.018 | 1.192e-05 | 5.190e-04 | 0.06 | 0.02 |
MPI_Isend | 32 | 54145 | 0.051 | 0.000e+00 | 3.400e-04 | 0.18 | 0.05 |
MPI_Waitall | 40960 | 393 | 0.102 | 1.288e-05 | 3.662e-03 | 0.35 | 0.10 |
MPI_Irecv | 8 | 128456 | 0.015 | 0.000e+00 | 3.099e-06 | 0.05 | 0.02 |
MPI_Waitall | 655360 | 64 | 0.089 | 5.610e-04 | 2.436e-03 | 0.31 | 0.09 |
MPI_Isend | 64 | 49592 | 0.043 | 0.000e+00 | 1.810e-04 | 0.15 | 0.04 |
MPI_Waitall | 229376 | 121 | 0.082 | 2.401e-04 | 1.438e-03 | 0.29 | 0.08 |
MPI_Isend | 40 | 46524 | 0.042 | 0.000e+00 | 7.820e-05 | 0.15 | 0.04 |
MPI_Bcast | 4 | 3240 | 0.078 | 9.537e-07 | 2.410e-04 | 0.27 | 0.08 |
MPI_Isend | 48 | 50531 | 0.049 | 0.000e+00 | 3.283e-03 | 0.17 | 0.05 |
MPI_Isend | 128 | 26552 | 0.024 | 0.000e+00 | 8.106e-05 | 0.08 | 0.02 |
MPI_Waitall | 458752 | 80 | 0.074 | 2.561e-04 | 1.430e-03 | 0.26 | 0.07 |
MPI_Isend | 56 | 46426 | 0.044 | 0.000e+00 | 7.701e-05 | 0.15 | 0.04 |
MPI_Isend | 96 | 27756 | 0.024 | 0.000e+00 | 5.794e-05 | 0.08 | 0.02 |
MPI_Isend | 80 | 25496 | 0.022 | 0.000e+00 | 3.815e-05 | 0.08 | 0.02 |
MPI_Waitall | 163840 | 129 | 0.064 | 2.813e-05 | 1.514e-03 | 0.22 | 0.06 |
MPI_Isend | 112 | 24644 | 0.022 | 0.000e+00 | 1.907e-05 | 0.08 | 0.02 |
MPI_Isend | 12 | 59803 | 0.060 | 0.000e+00 | 7.892e-05 | 0.21 | 0.06 |
MPI_Waitall | 114688 | 173 | 0.050 | 6.199e-05 | 6.921e-04 | 0.17 | 0.05 |
MPI_Isend | 256 | 23315 | 0.019 | 0.000e+00 | 9.611e-04 | 0.07 | 0.02 |
MPI_Isend | 224 | 24298 | 0.017 | 0.000e+00 | 6.914e-06 | 0.06 | 0.02 |
MPI_Isend | 192 | 17334 | 0.012 | 0.000e+00 | 6.914e-06 | 0.04 | 0.01 |
MPI_Irecv | 4 | 336589 | 0.039 | 0.000e+00 | 1.001e-05 | 0.14 | 0.04 |
MPI_Comm_size | 0 | 357072 | 0.022 | 0.000e+00 | 1.624e-03 | 0.08 | 0.02 |
MPI_Comm_rank | 0 | 336809 | 0.018 | 0.000e+00 | 5.960e-06 | 0.06 | 0.02 |
MPI_Isend | 16384 | 4618 | 0.029 | 0.000e+00 | 8.209e-04 | 0.10 | 0.03 |
MPI_Irecv | 28 | 142573 | 0.029 | 0.000e+00 | 4.699e-04 | 0.10 | 0.03 |
MPI_Isend | 20 | 24020 | 0.028 | 0.000e+00 | 1.270e-03 | 0.10 | 0.03 |
MPI_Isend | 768 | 28312 | 0.026 | 0.000e+00 | 7.606e-05 | 0.09 | 0.03 |
MPI_Isend | 28 | 25009 | 0.025 | 0.000e+00 | 7.796e-05 | 0.09 | 0.03 |
MPI_Isend | 10240 | 1981 | 0.025 | 0.000e+00 | 8.328e-04 | 0.09 | 0.02 |
MPI_Irecv | 81920 | 1159 | 0.024 | 0.000e+00 | 5.119e-04 | 0.08 | 0.02 |
MPI_Irecv | 16384 | 4618 | 0.024 | 0.000e+00 | 8.731e-04 | 0.08 | 0.02 |
MPI_Waitall | 0 | 12291 | 0.013 | 0.000e+00 | 2.370e-04 | 0.05 | 0.01 |
MPI_Irecv | 0 | 22809 | 0.022 | 0.000e+00 | 9.110e-04 | 0.08 | 0.02 |
MPI_Isend | 14336 | 2674 | 0.020 | 0.000e+00 | 8.550e-04 | 0.07 | 0.02 |
MPI_Isend | 5120 | 5005 | 0.018 | 0.000e+00 | 8.917e-05 | 0.06 | 0.02 |
MPI_Irecv | 14336 | 2674 | 0.017 | 0.000e+00 | 6.800e-04 | 0.06 | 0.02 |
MPI_Irecv | 6144 | 52725 | 0.016 | 0.000e+00 | 1.311e-05 | 0.06 | 0.02 |
MPI_Isend | 7168 | 3128 | 0.016 | 0.000e+00 | 2.150e-04 | 0.06 | 0.02 |
MPI_Irecv | 65536 | 995 | 0.015 | 0.000e+00 | 3.688e-04 | 0.05 | 0.01 |
MPI_Isend | 32768 | 1884 | 0.013 | 0.000e+00 | 8.011e-05 | 0.05 | 0.01 |
MPI_Isend | 4096 | 3529 | 0.012 | 0.000e+00 | 8.702e-05 | 0.04 | 0.01 |
MPI_Isend | 8192 | 2178 | 0.012 | 0.000e+00 | 3.061e-04 | 0.04 | 0.01 |
MPI_Isend | 640 | 13577 | 0.012 | 0.000e+00 | 8.011e-05 | 0.04 | 0.01 |
MPI_Isend | 81920 | 1159 | 0.011 | 0.000e+00 | 6.821e-04 | 0.04 | 0.01 |
Load balance by task: HPM counters |
---|
by MPI rank, by MPI time |
Load balance by task: memory, flops, timings |
by MPI rank, by MPI time |
Communication balance by task (sorted by MPI time) |
by MPI rank , time detail by MPI time , time detail by rank , call list |
Message Buffer Size Distributions: time |
|
Message Buffer Size Distributions: Ncalls |
|
Communication Topology : point to point data flow |
|