### MPI_gather unequals array

243
views
0
6 months ago by
Hi,

I wanna determine volume of each cell in a mesh and then collect and put in an array. However, when I used mpi to run in 4 processes, the mesh was divided into 4 parts and the number of cells in each parts was different so the length of each array was different and I don't know how to gather unequal arrays. Could you please help me to sort it out? Thanks
Here is the script to find volume of cells and put in array.

from dolfin import *
import numpy as np

mesh = UnitCubeMesh (10,10,10
volCell_array = np.array([]))
for c in cells(mesh):
volCell = c.volume()
volCell_array = np.append(volCell, volCell_array)
Community: FEniCS Project

1
6 months ago by
Ok, this is more an MPI question, not really related to the FEniCS, but let me provide an answer.

Every process has its own piece of information about the part of the mesh it owns. You must utilize MPI to communicate this
information between them. I always find useful these type of images: http://www.mcs.anl.gov/research/projects/mpi/tutorial/gropp/node74.html

Now, it depends who (which process) would you like to have the information about all the volumes. Let's assume you want the root process numer 0 to have it. From the picture you can see, gather is the choice -- every process has its own piece of information and they are collected to the root.

The code looks like
from dolfin import *
import numpy as np

mesh = UnitCubeMesh(3, 3, 3)
volCell_array = np.array([])
for c in cells(mesh):
volCell = c.volume()
volCell_array = np.append(volCell, volCell_array)

# We need mpi4py communicator, not the one wrapped by dolfin
comm = mesh.mpi_comm().tompi4py()
volCell_gathered = comm.gather(volCell_array, root=0)

if comm.rank == 0:
print("On rank = {}. volCell_array = {}.".format(comm.rank, volCell_gathered))​

As you can see, the root process has variable volCell_gathered which is an array of collected volCell_arrays.
Hi Michal,

Thanks for your help. However, the result based on your code is volCell_array=[array([0.166]), array([0.166, 0.166]), array([0.166]), array([0.166,0.166])]. (numbers in array is just an example)
Do you have any way to put it together? For example, volCell_array=[0.166, 0.166, 0.166, 0.166, 0.166, 0.166].
Thank you so much.
written 6 months ago by Vi
np.concatenate(volCell_array)