|
|
Regions | |||||
---|---|---|---|---|---|
Label | Ntasks | <MPI sec> | <Wall sec> | %Wall | [gflop/sec] |
ipm_noregion | 1728 | 0.9911 | 1.6437 | 74.45 | 0.0000e+00 |
hypre_BoomerAMGCycle | 1728 | 0.1768 | 0.4601 | 20.84 | 0.0000e+00 |
|
|
HPM Counter Statistics | ||||
---|---|---|---|---|
Event | Ntasks | Avg | Min(rank) | Max(rank) |
Communication Event Statistics (0.00% detail, 3.0550e+02 error) | |||||||
---|---|---|---|---|---|---|---|
Buffer Size | Ncalls | Total Time | Min Time | Max Time | %MPI | %Wall | |
MPI_Iprobe | 0 | 1679164518 | 450.628 | 0.000e+00 | 8.317e-02 | 147.51 | 48.82 |
MPI_Testall | 0 | 428912130 | 442.731 | 0.000e+00 | 5.592e-02 | 144.92 | 47.96 |
MPI_Test | 0 | 1247462416 | 235.374 | 0.000e+00 | 2.737e-03 | 77.05 | 25.50 |
MPI_Allreduce | 8 | 210816 | 209.537 | 1.883e-05 | 6.221e-02 | 68.59 | 22.70 |
MPI_Waitall | 65536 | 62502 | 68.605 | 1.788e-05 | 1.494e-02 | 22.46 | 7.43 |
MPI_Waitall | 57344 | 37235 | 37.978 | 3.386e-05 | 1.265e-02 | 12.43 | 4.11 |
MPI_Allreduce | 4 | 141696 | 49.298 | 1.717e-05 | 1.275e-02 | 16.14 | 5.34 |
MPI_Send | 28 | 1517419 | 32.238 | 0.000e+00 | 8.009e-02 | 10.55 | 3.49 |
MPI_Waitall | 8192 | 21514 | 2.192 | 1.907e-06 | 4.822e-03 | 0.72 | 0.24 |
MPI_Waitall | 20480 | 4963 | 2.981 | 3.099e-06 | 7.661e-03 | 0.98 | 0.32 |
MPI_Send | 4 | 1272501 | 20.830 | 0.000e+00 | 4.012e-03 | 6.82 | 2.26 |
MPI_Isend | 8 | 3235039 | 11.375 | 0.000e+00 | 6.276e-02 | 3.72 | 1.23 |
MPI_Recv | 8 | 1743898 | 16.614 | 0.000e+00 | 2.846e-02 | 5.44 | 1.80 |
MPI_Waitall | 16384 | 4780 | 1.461 | 3.815e-06 | 6.749e-03 | 0.48 | 0.16 |
MPI_Waitall | 64 | 983 | 0.170 | 9.537e-07 | 1.318e-03 | 0.06 | 0.02 |
MPI_Waitall | 32768 | 54122 | 12.237 | 4.053e-06 | 1.028e-02 | 4.01 | 1.33 |
MPI_Waitall | 80 | 1248 | 0.250 | 9.537e-07 | 1.310e-03 | 0.08 | 0.03 |
MPI_Waitall | 49152 | 8710 | 7.104 | 1.383e-05 | 1.201e-02 | 2.33 | 0.77 |
MPI_Waitall | 2048 | 26611 | 2.998 | 9.537e-07 | 1.051e-02 | 0.98 | 0.32 |
MPI_Waitall | 4096 | 23265 | 4.346 | 9.537e-07 | 6.098e-03 | 1.42 | 0.47 |
MPI_Waitall | 10240 | 33806 | 3.296 | 1.907e-06 | 3.728e-03 | 1.08 | 0.36 |
MPI_Waitall | 2560 | 10168 | 1.206 | 0.000e+00 | 6.115e-03 | 0.39 | 0.13 |
MPI_Waitall | 7168 | 4380 | 0.524 | 1.907e-06 | 4.820e-03 | 0.17 | 0.06 |
MPI_Waitall | 640 | 11164 | 1.670 | 1.907e-06 | 5.688e-03 | 0.55 | 0.18 |
MPI_Waitall | 512 | 10443 | 2.125 | 9.537e-07 | 5.291e-03 | 0.70 | 0.23 |
MPI_Isend | 12288 | 580111 | 4.441 | 0.000e+00 | 7.889e-04 | 1.45 | 0.48 |
MPI_Waitall | 12288 | 5101 | 1.587 | 1.907e-06 | 4.436e-03 | 0.52 | 0.17 |
MPI_Waitall | 28672 | 31390 | 6.371 | 5.007e-06 | 6.316e-03 | 2.09 | 0.69 |
MPI_Waitall | 1024 | 12340 | 2.759 | 9.537e-07 | 6.763e-03 | 0.90 | 0.30 |
MPI_Waitall | 96 | 1316 | 0.286 | 0.000e+00 | 1.265e-03 | 0.09 | 0.03 |
MPI_Waitall | 768 | 12941 | 1.749 | 2.146e-06 | 6.184e-03 | 0.57 | 0.19 |
MPI_Waitall | 1792 | 13853 | 2.151 | 0.000e+00 | 6.573e-03 | 0.70 | 0.23 |
MPI_Waitall | 1280 | 5325 | 2.557 | 0.000e+00 | 6.713e-03 | 0.84 | 0.28 |
MPI_Waitall | 1536 | 4128 | 2.013 | 9.537e-07 | 6.676e-03 | 0.66 | 0.22 |
MPI_Waitall | 3584 | 6749 | 1.958 | 9.537e-07 | 1.520e-02 | 0.64 | 0.21 |
MPI_Isend | 4 | 3718975 | 4.652 | 0.000e+00 | 2.889e-02 | 1.52 | 0.50 |
MPI_Waitall | 896 | 12269 | 1.833 | 0.000e+00 | 6.209e-03 | 0.60 | 0.20 |
MPI_Waitall | 24576 | 7775 | 3.747 | 5.007e-06 | 8.146e-03 | 1.23 | 0.41 |
MPI_Waitall | 256 | 6521 | 1.639 | 9.537e-07 | 4.820e-03 | 0.54 | 0.18 |
MPI_Waitall | 320 | 6576 | 1.809 | 1.907e-06 | 4.403e-03 | 0.59 | 0.20 |
MPI_Isend | 16 | 476147 | 0.713 | 0.000e+00 | 1.200e-02 | 0.23 | 0.08 |
MPI_Waitall | 5120 | 19379 | 1.550 | 9.537e-07 | 4.808e-03 | 0.51 | 0.17 |
MPI_Waitall | 384 | 5432 | 1.408 | 1.907e-06 | 4.830e-03 | 0.46 | 0.15 |
MPI_Waitall | 6144 | 19232 | 1.553 | 9.537e-07 | 6.007e-03 | 0.51 | 0.17 |
MPI_Waitall | 448 | 5138 | 1.165 | 9.537e-07 | 4.832e-03 | 0.38 | 0.13 |
MPI_Waitall | 81920 | 3956 | 2.839 | 2.098e-05 | 3.716e-03 | 0.93 | 0.31 |
MPI_Scan | 4 | 15552 | 2.757 | 3.386e-05 | 1.498e-03 | 0.90 | 0.30 |
MPI_Waitall | 14336 | 1408 | 0.322 | 4.053e-06 | 1.203e-03 | 0.11 | 0.03 |
MPI_Waitall | 192 | 2253 | 0.517 | 0.000e+00 | 2.016e-03 | 0.17 | 0.06 |
MPI_Waitall | 160 | 2122 | 0.488 | 1.192e-06 | 4.491e-03 | 0.16 | 0.05 |
MPI_Waitall | 327680 | 1300 | 2.361 | 1.719e-04 | 3.245e-03 | 0.77 | 0.26 |
MPI_Waitall | 224 | 2772 | 0.611 | 9.537e-07 | 4.497e-03 | 0.20 | 0.07 |
MPI_Waitall | 393216 | 1459 | 2.314 | 3.569e-04 | 3.412e-03 | 0.76 | 0.25 |
MPI_Waitall | 655360 | 1000 | 2.138 | 9.179e-04 | 3.603e-03 | 0.70 | 0.23 |
MPI_Waitall | 128 | 2029 | 0.467 | 9.537e-07 | 1.339e-03 | 0.15 | 0.05 |
MPI_Waitall | 112 | 1085 | 0.264 | 0.000e+00 | 4.828e-03 | 0.09 | 0.03 |
MPI_Waitall | 196608 | 1945 | 1.762 | 1.969e-04 | 2.419e-03 | 0.58 | 0.19 |
MPI_Waitall | 458752 | 1227 | 1.745 | 1.891e-04 | 2.396e-03 | 0.57 | 0.19 |
MPI_Isend | 24 | 271470 | 0.443 | 0.000e+00 | 7.522e-03 | 0.15 | 0.05 |
MPI_Waitall | 98304 | 3705 | 1.677 | 1.788e-05 | 1.831e-03 | 0.55 | 0.18 |
MPI_Waitall | 262144 | 1093 | 1.656 | 1.569e-04 | 2.896e-03 | 0.54 | 0.18 |
MPI_Waitall | 3072 | 1897 | 0.762 | 9.537e-07 | 6.114e-03 | 0.25 | 0.08 |
MPI_Isend | 32 | 643982 | 0.876 | 0.000e+00 | 1.564e-02 | 0.29 | 0.09 |
MPI_Irecv | 12288 | 580111 | 0.943 | 0.000e+00 | 6.978e-04 | 0.31 | 0.10 |
MPI_Waitall | 229376 | 1489 | 1.351 | 2.508e-04 | 2.413e-03 | 0.44 | 0.15 |
MPI_Waitall | 0 | 101707 | 1.229 | 0.000e+00 | 1.318e-02 | 0.40 | 0.13 |
MPI_Waitall | 40960 | 4093 | 1.338 | 1.502e-05 | 3.585e-03 | 0.44 | 0.14 |
MPI_Waitall | 131072 | 2047 | 1.284 | 2.098e-05 | 1.961e-03 | 0.42 | 0.14 |
MPI_Irecv | 8 | 1491141 | 0.201 | 0.000e+00 | 1.717e-05 | 0.07 | 0.02 |
MPI_Isend | 6144 | 516139 | 1.188 | 0.000e+00 | 9.894e-05 | 0.39 | 0.13 |
MPI_Waitall | 524288 | 600 | 1.152 | 8.001e-04 | 3.887e-03 | 0.38 | 0.12 |
MPI_Isend | 64 | 482168 | 0.509 | 0.000e+00 | 3.002e-04 | 0.17 | 0.06 |
MPI_Waitall | 163840 | 1612 | 1.110 | 2.313e-05 | 1.980e-03 | 0.36 | 0.12 |
MPI_Isend | 1536 | 315362 | 0.317 | 0.000e+00 | 1.781e-04 | 0.10 | 0.03 |
MPI_Isend | 40 | 454141 | 0.572 | 0.000e+00 | 1.095e-02 | 0.19 | 0.06 |
MPI_Bcast | 4 | 29376 | 1.012 | 3.099e-06 | 5.860e-04 | 0.33 | 0.11 |
MPI_Isend | 12 | 611389 | 0.911 | 0.000e+00 | 2.360e-04 | 0.30 | 0.10 |
MPI_Waitall | 114688 | 2061 | 0.854 | 6.795e-05 | 1.431e-03 | 0.28 | 0.09 |
MPI_Isend | 48 | 470495 | 0.517 | 0.000e+00 | 3.571e-04 | 0.17 | 0.06 |
MPI_Isend | 80 | 269135 | 0.290 | 0.000e+00 | 5.142e-03 | 0.09 | 0.03 |
MPI_Isend | 128 | 285561 | 0.284 | 0.000e+00 | 2.098e-05 | 0.09 | 0.03 |
MPI_Isend | 56 | 447837 | 0.473 | 0.000e+00 | 2.849e-04 | 0.15 | 0.05 |
MPI_Isend | 96 | 297427 | 0.304 | 0.000e+00 | 9.394e-05 | 0.10 | 0.03 |
MPI_Irecv | 4 | 4522147 | 0.736 | 0.000e+00 | 3.501e-02 | 0.24 | 0.08 |
MPI_Isend | 112 | 260550 | 0.266 | 0.000e+00 | 8.202e-05 | 0.09 | 0.03 |
MPI_Isend | 192 | 184302 | 0.152 | 0.000e+00 | 1.693e-05 | 0.05 | 0.02 |
MPI_Isend | 224 | 225722 | 0.186 | 0.000e+00 | 1.717e-05 | 0.06 | 0.02 |
MPI_Isend | 20 | 261933 | 0.455 | 0.000e+00 | 2.714e-03 | 0.15 | 0.05 |
MPI_Isend | 256 | 179950 | 0.164 | 0.000e+00 | 1.812e-05 | 0.05 | 0.02 |
MPI_Isend | 160 | 110416 | 0.104 | 0.000e+00 | 3.061e-03 | 0.03 | 0.01 |
MPI_Isend | 28 | 253580 | 0.404 | 0.000e+00 | 9.209e-03 | 0.13 | 0.04 |
MPI_Irecv | 28 | 1739005 | 0.381 | 0.000e+00 | 3.300e-03 | 0.12 | 0.04 |
MPI_Comm_size | 0 | 4099616 | 0.221 | 0.000e+00 | 8.446e-03 | 0.07 | 0.02 |
MPI_Isend | 16384 | 41933 | 0.325 | 0.000e+00 | 7.279e-04 | 0.11 | 0.04 |
MPI_Comm_rank | 0 | 3815323 | 0.201 | 0.000e+00 | 6.890e-05 | 0.07 | 0.02 |
MPI_Isend | 768 | 278063 | 0.274 | 0.000e+00 | 7.892e-05 | 0.09 | 0.03 |
MPI_Isend | 10240 | 19004 | 0.255 | 0.000e+00 | 7.961e-04 | 0.08 | 0.03 |
MPI_Irecv | 81920 | 10340 | 0.207 | 0.000e+00 | 7.012e-04 | 0.07 | 0.02 |
MPI_Isend | 14336 | 23927 | 0.207 | 0.000e+00 | 7.401e-04 | 0.07 | 0.02 |
MPI_Irecv | 16384 | 41933 | 0.178 | 0.000e+00 | 6.828e-04 | 0.06 | 0.02 |
MPI_Isend | 32768 | 16218 | 0.165 | 0.000e+00 | 9.179e-05 | 0.05 | 0.02 |
MPI_Irecv | 6144 | 488012 | 0.156 | 0.000e+00 | 1.407e-05 | 0.05 | 0.02 |
MPI_Isend | 512 | 87253 | 0.097 | 0.000e+00 | 1.693e-05 | 0.03 | 0.01 |
MPI_Isend | 8192 | 21142 | 0.145 | 0.000e+00 | 7.429e-04 | 0.05 | 0.02 |
MPI_Isend | 640 | 160176 | 0.144 | 0.000e+00 | 1.383e-05 | 0.05 | 0.02 |
MPI_Irecv | 14336 | 23927 | 0.138 | 0.000e+00 | 6.771e-04 | 0.05 | 0.01 |
MPI_Irecv | 65536 | 8603 | 0.130 | 0.000e+00 | 4.299e-04 | 0.04 | 0.01 |
MPI_Isend | 40960 | 12002 | 0.118 | 0.000e+00 | 8.140e-04 | 0.04 | 0.01 |
MPI_Isend | 81920 | 10340 | 0.107 | 0.000e+00 | 6.089e-04 | 0.03 | 0.01 |
MPI_Recv | 4 | 469329 | 0.101 | 0.000e+00 | 8.416e-05 | 0.03 | 0.01 |
MPI_Irecv | 8192 | 21142 | 0.098 | 0.000e+00 | 8.860e-04 | 0.03 | 0.01 |
MPI_Irecv | 40960 | 12002 | 0.094 | 0.000e+00 | 1.001e-03 | 0.03 | 0.01 |
Load balance by task: HPM counters |
---|
by MPI rank, by MPI time |
Load balance by task: memory, flops, timings |
by MPI rank, by MPI time |
Communication balance by task (sorted by MPI time) |
by MPI rank , time detail by MPI time , time detail by rank , call list |
Message Buffer Size Distributions: time |
|
Message Buffer Size Distributions: Ncalls |
|
Communication Topology : point to point data flow |
|
Switch Traffic (volume by node) |
|
Memory usage by node |
|