This is a small wrapper class around a BLACS grid and a BLACS context. More...

#include <BLACSGrid.hpp>

Public Member Functions
	BLACSGrid ()

	BLACSGrid (const MPIComm &comm)

	BLACSGrid (MPIComm &&comm)

	BLACSGrid (const MPIComm &comm, int P)

	BLACSGrid (MPIComm &&comm, int P)

	~BLACSGrid ()

	BLACSGrid (const BLACSGrid &grid)

	BLACSGrid (BLACSGrid &&grid)

BLACSGrid &	operator= (const BLACSGrid &grid)

BLACSGrid &	operator= (BLACSGrid &&grid)

const MPIComm &	Comm () const

MPIComm &	Comm ()

int	ctxt () const

int	ctxt_all () const

int	nprows () const

int	npcols () const

int	prow () const

int	pcol () const

int	P () const

int	npactives () const

bool	active () const

BLACSGrid	transpose () const

const MPIComm &	Comm_active () const

Static Public Member Functions
static void	layout (int procs, int &proc_rows, int &proc_cols)

Friends
std::ostream &	operator<< (std::ostream &os, const BLACSGrid *g)

Detailed Description

This is a small wrapper class around a BLACS grid and a BLACS context.

The main purpose of this wrapper is to handle the (global) resource of BLACS grid, ie, creation and destruction, as well as to encapsulate the BLACS interface into a single small piece of code. This will hopefully make it much easier to at one point in the near future get rid of ScaLAPACK and BLACS, and replace it with SLATE.

This class also stores (and owns) an MPIComm object. The BLACS grid is a 2D grid, consisting of nprows x npcols processes, with ranks assigned COLUMN MAJOR!!. However, the total number of processes, P, can be larger than nprows x npcols, meaning some processes are included in the MPIComm, but not in the grid, ie, they are idle. Leaving some processes idle can give a grid with a better aspect ratio, compared to using all available ranks, leading to more scalable code.

The MPIComm object can also be MPI_COMM_NULL, while P can still be > 0. This is useful for getting info about a grid (the layout), in which this rank is not active, for instance to know to which rank to communicate.

Constructor & Destructor Documentation

◆ BLACSGrid() [1/7]

strumpack::BLACSGrid::BLACSGrid ( )

inline

Default constructor. Does not construct any BLACS grid or context.

◆ BLACSGrid() [2/7]

strumpack::BLACSGrid::BLACSGrid ( const MPIComm & comm )

inline

Construct a BLACSGrid from an MPIComm object. The MPIComm will be duplicated. The BLACS grid will be created. This operation is collective on comm.

Parameters

comm	MPIComm used to initialize the grid, this will be duplicated

◆ BLACSGrid() [3/7]

strumpack::BLACSGrid::BLACSGrid ( MPIComm && comm )

inline

Construct a BLACSGrid from an MPIComm object. The MPIComm will be moved from (reset to MPI_COMM_NULL). The BLACS grid will be created. This operation is collective on comm.

Parameters

comm	MPIComm used to initialize the grid, this will be moved (reset to MPI_COMM_NULL)

◆ BLACSGrid() [4/7]

strumpack::BLACSGrid::BLACSGrid	(	const MPIComm &	comm,
		int	P
	)

inline

Construct a BLACSGrid from an MPIComm object. The MPIComm will be duplicated. The BLACS grid will be created. This operation is collective on comm. comm can be a null communicator (MPI_COMM_NULL), but P can be used to specify the number of processes in the grid (including potentially idle ranks).

Parameters

comm	MPIComm used to initialize the grid, this will be duplicated
P	total number of ranks in the new grid (P == comm.size() or comm.is_null())

◆ BLACSGrid() [5/7]

strumpack::BLACSGrid::BLACSGrid	(	MPIComm &&	comm,
		int	P
	)

inline

Construct a BLACSGrid from an MPIComm object. The MPIComm will be moved. The BLACS grid will be created. This operation is collective on comm. comm can be a null communicator (MPI_COMM_NULL), but P can be used to specify the number of processes in the grid (including potentially idle ranks).

Parameters

comm	MPIComm used to initialize the grid, this will be moved from, and then reset to a null communicator (MPI_COMM_NULL)
P	total number of ranks in the new grid (P == comm.size() or comm.is_null())

◆ ~BLACSGrid()

strumpack::BLACSGrid::~BLACSGrid ( )

inline

Destructor, this will free all resources associated with this grid.

◆ BLACSGrid() [6/7]

strumpack::BLACSGrid::BLACSGrid ( const BLACSGrid & grid )

inline

Copy constructor. This might be expensive, the entire grid will be copied. The resulting grid will be a new grid, with a different BLACS context, and so cannot be used in the same ScaLAPACK call as the original grid. Also the MPIComm object will be copied. This operation is collective on all processes in the MPIComm.

Parameters

grid	BLACSGrid to copy from

◆ BLACSGrid() [7/7]

strumpack::BLACSGrid::BLACSGrid ( BLACSGrid && grid )

inline

Move constructor.

Member Function Documentation

◆ active()

bool strumpack::BLACSGrid::active ( ) const

inline

Checks whether this rank is active on this grid, ie, the grid has been initialized, and this rank is not one of the idle ranks.

◆ Comm() [1/2]

MPIComm & strumpack::BLACSGrid::Comm ( )

inline

Return a reference to the MPIComm object.

◆ Comm() [2/2]

const MPIComm & strumpack::BLACSGrid::Comm ( ) const

inline

Return a (const) reference to the MPIComm object.

◆ ctxt()

int strumpack::BLACSGrid::ctxt ( ) const

inline

Return the blacs context, a context to a blacs grid with nprows processor rows and npcols processor columns (ranks are assigned column major). Or -1 for a non-initialized grid or for ranks which are idle in this grid.

◆ ctxt_all()

int strumpack::BLACSGrid::ctxt_all ( ) const

inline

Return a BLACS context for a grid including all the ranks, not just those active on ctxt(), but all the ranks in Comm(). The grid is a single row with Comm().size() columns. Will return -1 for a non-initialized grid, or initialized with a null communicator.

This can be used as the last argument for a call to P*DGEMR2d.

◆ layout()

static void strumpack::BLACSGrid::layout	(	int	procs,
		int &	proc_rows,
		int &	proc_cols
	)

inlinestatic

For a given number of processes procs, find a 2D layout. This will try to find a 2D layout using P, or as close to P as possible, number of ranks, while making the grid as square as possible. The number of processes in the 2D layout can be less than procs, ie, some ranks will be idle, to improve parallelism. procs >= proc_rows * proc_cols.

Parameters

procs	maximum number of processes in the 2d layout
proc_rows	output, number of rows in the 2d layout
proc_cols	output, number of columns in the 2d layout

◆ npactives()

int strumpack::BLACSGrid::npactives ( ) const

inline

Number of processes active in the grid, simply nprows()*npcols().

◆ npcols()

int strumpack::BLACSGrid::npcols ( ) const

inline

Number of processor columns in the grid, -1 if this rank is idle in the grid, or if not initialized.

◆ nprows()

int strumpack::BLACSGrid::nprows ( ) const

inline

Number of processor rows in the grid, -1 if this rank is idle in the grid, or if not initialized.

◆ operator=() [1/2]

BLACSGrid & strumpack::BLACSGrid::operator= ( BLACSGrid && grid )

inline

Move assignment, does not make a copy.

Parameters

grid	grid to move from, this will be reset to a non-existing grid (a -1 context).

◆ operator=() [2/2]

BLACSGrid & strumpack::BLACSGrid::operator= ( const BLACSGrid & grid )

inline

Copy assignment. This might be expensive, the entire grid will be copied. The resulting grid will be a new grid, with a different BLACS context, and so cannot be used in the same ScaLAPACK call as the original grid. Also the MPIComm object will be copied. This operation is collective on all processes in the MPIComm.

Parameters

grid	BLACSGrid to copy from

Returns: the newly created grid

◆ P()

int strumpack::BLACSGrid::P ( ) const

inline

Total number of ranks in this grid, including any ranks that may be idle. P >= nprows()*npcols(). If Comm() is not a null communicator, then P == Comm.size().

◆ pcol()

int strumpack::BLACSGrid::pcol ( ) const

inline

This ranks processor column in the grid. -1 if this rank is not part of the grid.

◆ prow()

int strumpack::BLACSGrid::prow ( ) const

inline

This ranks processor row in the grid. -1 if this rank is not part of the grid.

◆ transpose()

BLACSGrid strumpack::BLACSGrid::transpose ( ) const

inline

Return a BLACSGrid which is the transpose of the current grid. Ie., has npcols processor rows and nprows processor columns. This can be used to perform operations on the transpose of a distributed matrix. Instead of transposing the matrix, which requires communication, one can transpose the BLACS grid and only transpose the local storage.

Friends And Related Function Documentation

◆ operator<<

std::ostream & operator<<	(	std::ostream &	os,
		const BLACSGrid *	g
	)

friend

Print some info about the BLACS grid to stream os. Just used for debugging.

The documentation for this class was generated from the following file:

/home/pieterg/LBL/STRUMPACK/src/dense/BLACSGrid.hpp

Public Member Functions

Static Public Member Functions

Friends

Detailed Description

Constructor & Destructor Documentation

◆ BLACSGrid() [1/7]

◆ BLACSGrid() [2/7]

◆ BLACSGrid() [3/7]

◆ BLACSGrid() [4/7]

◆ BLACSGrid() [5/7]

◆ ~BLACSGrid()

◆ BLACSGrid() [6/7]

◆ BLACSGrid() [7/7]

Member Function Documentation

◆ active()

◆ Comm() [1/2]

◆ Comm() [2/2]

◆ ctxt()

◆ ctxt_all()

◆ layout()

◆ npactives()

◆ npcols()

◆ nprows()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ P()

◆ pcol()

◆ prow()

◆ transpose()

Friends And Related Function Documentation

◆ operator<<