Coordinates of Ghost DOFs

5 weeks ago by

Is there any way to get the coordinates of the unowned local dofs (ghost)?

The function tabulate_dof_coordinates() only tabulates owned dof coordinates when it works in parallel!
Thanks a lot!

Community: FEniCS Project

1 Answer

29 days ago by
There is very likely a better way, but below you can find code that I briefly tested for my own purposes and it seems to work.

This is not optimized in any way! It always returns x, y, and z coordinates, also in 2D because that was something I needed

def all_dof_coordinates(V):
    Return coordinates of all dofs, similar to tabulate_dof_coordinates,
    but also includes the ghosts dofs. This always returns 3-vectors for
    the coordinates, even if the mesh is 2D or 1D.

    This has all communication going via the root. Consider rewriting this
    by using DofMap.off_process_owner to avoid the root botleneck for
    large MPI runs
    mesh = V.mesh()

    # Get coordinates of regular dofs
    gdim = mesh.geometry().dim()
    dofs_x = V.tabulate_dof_coordinates().reshape((-1, gdim))
    global_dofs = V.dofmap().tabulate_local_to_global_dofs()
    Nlocal = len(dofs_x)

    # Make sure dof positions are 3-vectors
    if gdim != 3:
        tmp = numpy.zeros((dofs_x.shape[0], 3), float)
        tmp[:, :gdim] = dofs_x
        dofs_x = tmp

    # Do nothing in serial
    comm = mesh.mpi_comm()
    if comm.size == 1:
        return dofs_x, Nlocal

    # Missing and present information
    missing_dofs = global_dofs[Nlocal:]
    present_dofs = global_dofs[:Nlocal]

    # Send list of global dofs which we want to know the coordinates for
    # along with the global dofs where we know the coordinates
    alldata = comm.gather((missing_dofs, present_dofs, dofs_x))

    # Global dof coordinate map, may need to be changed for comm.size >> 1
    if comm.rank == 0:
        # Build global dof coordinate mapping using each ranks info
        global_dofs_x = {}
        for _rank_missing_dofs, rank_present_dofs, rank_dofs_x in alldata:
            for gdof, coord in zip(rank_present_dofs, rank_dofs_x):
                global_dofs_x[gdof] = coord

        # Find the missing local data in the global mapping for each rank
        ret_data = []
        for rank_missing_dofs, _rank_present_dofs, _rank_dofs_x in alldata:
            ret = []
            for gdof in rank_missing_dofs:
        ret_data = None

    # Get back the missing values
    ghost_coords = comm.scatter(ret_data)

    # Create dof coordinates with ghost positions
    dofs_x_all = numpy.zeros((len(global_dofs), 3), float)
    dofs_x_all[:Nlocal] = dofs_x
    for i, coord in enumerate(ghost_coords):
        dofs_x_all[Nlocal + i] = coord

    return dofs_x_all, Nlocal​
Please login to add an answer/comment or follow this question.

Similar posts:
Search »