Error: Duplication of MPI communicator failed


34
views
0
15 days ago by
Howdy,

So essentially I'm running a fenics pde solver in a double for loop, where every iteration I'm solving the pde with different parameters.

I'm trying to solve the pde 100 different times, with different parameter values, and it goes until about 91 and I get this issue,
Traceback (most recent call last):
  File "paper_runner.py", line 38, in <module>
    out = prog(mu_arr[j],10.0,k_arr[i],.75,domains[d],[direction_get(angle)],[1.0],32,20,True,False)
  File "/Users/celsloaner/Dropbox/MIPDECO/mu_test/paper/movies_and_plots_for_mu_y_planark=1 45/robust_paper.py", line 525, in prog
    solve(F == 0, uv, J=Jac)
  File "/Users/celsloaner/anaconda3/envs/fenicsproject/lib/python3.6/site-packages/fenics_adjoint/solving.py", line 32, in solve
    output = backend.solve(*args, **kwargs)
  File "/Users/celsloaner/anaconda3/envs/fenicsproject/lib/python3.6/site-packages/dolfin/fem/solving.py", line 300, in solve
    _solve_varproblem(*args, **kwargs)
  File "/Users/celsloaner/anaconda3/envs/fenicsproject/lib/python3.6/site-packages/dolfin/fem/solving.py", line 349, in _solve_varproblem
    solver.solve()
RuntimeError: *** Error: Duplication of MPI communicator failed (MPI_Comm_dup​

I think this is some sort of weird freeing issue related to memory. Am I running out of memory? How can I fix it so this issue does not occur?
Community: FEniCS Project
I had a similar problem...  I've not found a way to reproduce it with a minimal example.  If you can do that, it would help.  See my question here:

https://groups.google.com/forum/#!topic/fenics-support/VA28ABrss74
written 15 days ago by pf4d  
Yea, I found this before I posted here. I think its impossible to construct a minimal example, it appears to me that this error appears as a result of not freeing mpi communicators properly. For small problems the issue will not be apparently, but if you are calling the code a lot in the same program, I think this is guaranteed to happen eventually.
written 15 days ago by Ryan Vogt  

1 Answer


0
14 days ago by
There is definitely something strange going on. Memory is always increasing as I solve the same PDE, with different parameters. I would expect every run is "clean" in the sense that I'm not gaining additional memory usage should be pretty constant. Some freeing issue behind the scenes?
Please login to add an answer/comment or follow this question.

Similar posts:
Search »