LAM/MPI logo

Everyday datatypes

  |   Home   |   Download   |   Documentation   |   FAQ   |   all just the tutorials
Heterogeneous computing requires that the data constituting a message be typed or described somehow so that its machine representation can be converted between computer architectures. MPI can thoroughly describe message datatypes, from the simple primitive machine types to complex structures, arrays and indices.

MPI messaging functions accept a datatype parameter, whose C typedef is MPI_Datatype:

    MPI_Send(void* buf, int count, MPI_Datatype datatype,
	    int dest, int tag, MPI_Comm comm);

Basic Datatypes

Everybody uses the primitive machine datatypes. Some C examples are listed below (with the corresponding C datatype in parentheses):

    MPI_CHAR (char)
    MPI_INT (int)
    MPI_FLOAT (float)
    MPI_DOUBLE (double)
The count parameter in MPI_Send( ) refers to the number of elements of the given datatype, not the total number of bytes.

For messages consisting of a homogeneous, contiguous array of basic datatypes, this is the end of the datatype discussion. For messages that contain more than one datatype or whose elements are not stored contiguously in memory, something more is needed.

Strided Vector

Consider a mesh application with patches of a 2D array assigned to different processes. The internal boundary rows and columns are transferred between north/south and east/west processes in the overall mesh. In C, the transfer of a row in a 2D array is simple - a contiguous vector of elements equal in number to the number of columns in the 2D array. Conversely, storage of the elements of a single column are dispersed in memory; each vector element separated from its next and previous indices by the size of one entire row.

An MPI derived datatype is a good solution for a non-contiguous data structure. A code fragment to derive an appropriate datatype matching this strided vector and then transmit the last column is listed below:


#include <mpi.h>

{
    float		mesh[10][20];
    int			dest, tag;
    MPI_Datatype	newtype;

/*
 * Do this once.
 */
    MPI_Type_vector(10,		/* # column elements */
	    1,			/* 1 column only */
	    20,			/* skip 20 elements */
	    MPI_FLOAT,		/* elements are float */
	    &newtype);		/* MPI derived datatype */

    MPI_Type_commit(&newtype);
/*
 * Do this for every new message.
 */
    MPI_Send(&mesh[0][19], 1, newtype,
	    dest, tag, MPI_COMM_WORLD);
}


MPI_Type_commit( ) separates the datatypes you really want to save and use from the intermediate ones that are scaffolded on the way to some very complex datatype.

A nice feature of MPI derived datatypes is that once created, they can be used repeatedly with no further set-up code. MPI has many other derived datatype constructors.

C Structure

Consider an imaging application that is transferring fixed length scan lines of eight bit color pixels. Coupled with the pixel array is the scan line number, an integer. The message might be described in C as a structure:

    struct {
	int		lineno;
	char		pixels[1024];
    } scanline;
In addition to a derived datatype, message packing is a useful method for sending non-contiguous and/or heterogeneous data. A code fragment to pack and send the above structure is listed below:


#include <mpi.h>

{
    unsigned int	membersize, maxsize;
    int			position;
    int			dest, tag;
    char		*buffer;
/*
 * Do this once.
 */
    MPI_Pack_size(1, 		/* one element */
	    MPI_INT,		/* datatype integer */
	    MPI_COMM_WORLD,	/* consistent comm. */
	    &membersize);	/* max packing space req'd */

    maxsize = membersize;
    MPI_Pack_size(1024, MPI_CHAR, MPI_COMM_WORLD, &membersize);
    maxsize += membersize;
    buffer = malloc(maxsize);
/*
 * Do this for every new message.
 */
    position = 0;

    MPI_Pack(&scanline.lineno,	/* pack this element */
	    1,			/* one element */
	    MPI_INT,		/* datatype int */
	    buffer,		/* packing buffer */
	    maxsize,		/* buffer size */
	    &position,		/* next free byte offset */
	    MPI_COMM_WORLD);	/* consistent comm. */

    MPI_Pack(scanline.pixels, 1024, MPI_CHAR,
	    buffer, maxsize, &position, MPI_COMM_WORLD);

    MPI_Send(buffer, position, MPI_PACKED,
	    dest, tag, MPI_COMM_WORLD);
}


A buffer is allocated once to contain the size of the packed structure. The size must be computed because of implementation dependent overhead in the message. Variable sized messages can be handled by allocating a buffer large enough for the largest possible message. The position parameter to MPI_Pack( ) always returns the current size of the packed buffer.

A code fragment to unpack the message, assuming a receive buffer has been allocated, is listed below:


{
    int             src;
    int             msgsize;
    MPI_Status      status;

    MPI_Recv(buffer, maxsize, MPI_PACKED,
	    src, tag, MPI_COMM_WORLD, &status);

    position = 0;
    MPI_Get_count(&status, MPI_PACKED, &msgsize);

    MPI_Unpack(buffer,		/* packing buffer */
	    msgsize,		/* buffer size */
	    &position,		/* next element byte offset */
	    &scanline.lineno,	/* unpack this element */
	    1,			/* one element */
	    MPI_INT,		/* datatype int */
	    MPI_COMM_WORLD);	/* consistent comm. */

    MPI_Unpack(buffer, msgsize, &position,
	    scanline.pixels, 1024, MPI_CHAR, MPI_COMM_WORLD);
}

You should be able to modify the above code fragments for any structure. It is completely possible to alter the number of elements to unpack based on application information unpacked previously in the same message.