MPI: segmentation fault with C++ demo Periodic


410
views
0
5 months ago by

Fenics experts:

------------------
UPDATE note:  I am still hoping to find a suggestion for this issue.  The .cpp code below is a MWE that can replace main.cpp in the demo path in Docker:
/home/fenics/demo/documented/periodic/cpp/main.cpp
The demo is then compiled normally (ffc...cmake...make) and if run with:
$ mpirun -n 2 ./demo_periodic
The code will crash (see below for detail).  Periodic boundary conditions are important to a problem that I use, and I am not able to use MPI with the code at present.   Any suggestions would be much appreciated! (or even if it is just not possible to do at present state of Fenics/MPI).
------------------

I am trying to integrate MPI into my C++ Fenics code and have trouble using a Periodic Boundary condition with the FunctionSpace declaration. 

The problem appears even with the C++ demo "Periodic" on making the change to "shared_facet" for the "ghost_mode" parameter, as detailed below.

I am using Docker on MacOS; DOLFIN version: 2017.1.0; 
I am running a modified version of /home/fenics/demo/documented/periodic/cpp/main.cpp
and it is run with: $ mpirun -n 2 ./demo_periodic

The code  exits with the following error message:

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 443 RUNNING AT 65e1b110a394
=   EXIT CODE: 11
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions


The above is the complete message.  Note there is no Dolfin error message--it simply exits with the segmentation fault.

Here is the modified main.cpp file.  I have narrowed the crash to the declaration of the FunctionSpace with the Periodic BC.  Without using Periodic BC, it works fine.  To get the code to work at all I have to set "ghost_mode" to "shared_facet" per a previous post:
https://www.allanswered.com/post/ealn/how-to-correctly-set-ghost-mode-for-parallel-assembly/

This appears to be part of the problem, since the BC does not pose a problem without this statement (but it is required, so I can't skip it).  There is only an issue if I run using 'mpirun' and with more than one MPI node, so this also narrows the problem to using the periodic BC with MPI (i.e., there was no problem with the same code before using MPI or by using: 'mpirun -n 1 ./demo_periodic').

#include <dolfin.h>
#include "Poisson.h"

using namespace dolfin;

int main()
{
  // Source term
  class Source : public Expression
  {
  public:

    void eval(Array<double>& values, const Array<double>& x) const
    {
      const double dx = x[0] - 0.5;
      const double dy = x[1] - 0.5;
      values[0] = x[0]*sin(5.0*DOLFIN_PI*x[1]) + 1.0*exp(-(dx*dx + dy*dy)/0.02);
    }

  };

  // Sub domain for Dirichlet boundary condition
  class DirichletBoundary : public SubDomain
  {
    bool inside(const Array<double>& x, bool on_boundary) const
    { return (x[1] < DOLFIN_EPS || x[1] > (1.0 - DOLFIN_EPS)) && on_boundary; }
  };

  // Sub domain for Periodic boundary condition
  class PeriodicBoundary : public SubDomain
  {
    // Left boundary is "target domain" G
    bool inside(const Array<double>& x, bool on_boundary) const
    { return (std::abs(x[0]) < DOLFIN_EPS); }

    // Map right boundary (H) to left boundary (G)
    void map(const Array<double>& x, Array<double>& y) const
    {
      y[0] = x[0] - 1.0;
      y[1] = x[1];
    }
  };

//MODIFICATIONS FOR USE WITH MPI:
std::string  old_ghost_mode = parameters["ghost_mode"];
//for MPI:
parameters["ghost_mode"] = "shared_facet";
std::cout <<"old ghost_mode: "<< old_ghost_mode <<std::endl;
//END MODIFICATIONS

  // Create mesh
  auto mesh = std::make_shared<UnitSquareMesh>(32, 32);

  // Create periodic boundary condition
  auto periodic_boundary = std::make_shared<PeriodicBoundary>();

  // Create functions
  auto f = std::make_shared<Source>();

  // Define PDE
//THE CODE CRASHES ON THIS STATEMENT, when using mpirun -n 2...:
  auto V = std::make_shared<Poisson::FunctionSpace>(mesh, periodic_boundary);
  Poisson::BilinearForm a(V, V);
  Poisson::LinearForm L(V);
  L.f = f;

  // Create Dirichlet boundary condition
  auto u0 = std::make_shared<Constant>(0.0);
  auto dirichlet_boundary = std::make_shared<DirichletBoundary>();
  DirichletBC bc0(V, u0, dirichlet_boundary);

  // Collect boundary conditions
  std::vector<const DirichletBC*> bcs = {&bc0};

  // Compute solution
  Function u(V);
  solve(a == L, u, bcs);


  return 0;
}



It is noted in the above quoted post that: "Ghosted mesh support is still experimental" so I'm not sure if there is any hope here. 

I have a rather opaque understanding of how MPI is implemented under the hood in Fenics, so it's not clear to me what would be the problem. 
Anyone have any help on this issue?

--James W.

Community: FEniCS Project
Please login to add an answer/comment or follow this question.

Similar posts:
Search »