### System can only be sovled by MUMPS - Why?

76

views

0

Hello,

I am using the PETScSNES Solver to solve my nonlinear problem (an electric system in the charge coservation formulation).

Curiously only MUMPS as linear solver finds a solution (usually in one iteration).

Other linear solver fail to find a solution (usually the exit after the zeroth iteration with diverged linear search as reason).

For small test problems MUMPS was fine, now I want to go to meshes with more elements and MUMPs gets quite slow.

Therefore I want to understand why only MUMPS works, but I have no idea how to approach this.

Can anyone point my in the right direction?

(I am aware thats not a direct Fenics questions, but maybe Fenics can give me some helpful statistics?)

Thank you,

Moritz

To clarify what problem I am solving:

\[

\frac{\partial q_1}{\partial t} - \nabla \sigma_1 \nabla \Phi_1 +f_1 = 0\, on\, \Omega_1\\

\frac{\partial q_2}{\partial t} - \nabla \sigma_2 \nabla \Phi_2 +f_2 = 0\, on\, \Omega_2 \\

- \frac{\partial q_1}{\partial t} - \frac{\partial q_2}{\partial t} - \nabla \sigma_3 \nabla \Phi_3 - f_1 -f_2 = 0\, on\, \Omega_3 \\

q_1= c_1 (\Phi_1 -\Phi_3)\, on\, \Omega_1 \cap \Omega_3 \\

q_2= c_2 (\Phi_2 -\Phi_3)\, on\, \Omega_2 \cap \Omega_3 \\

f_1=k_1 exp(a_1(\Phi_1 -\Phi_3))- k'_1 exp(-a'_1(\Phi_1 -\Phi_3)) \, on\, \Omega_1 \cap \Omega_3 \\

f_2=k_2 exp(a_2(\Phi_2 -\Phi_3))-k'_2 exp(-a'_2(\Phi_2 -\Phi_3))) \, on\, \Omega_2\cap \Omega_3 \\

\Phi_2 = 0\, on\, \Gamma_2 \\

\frac{\partial \Phi_1}{\partial n} =g \,on\, \Gamma_1

\]

where \(\sigma\) c,k and a are positive and spatially varing.

\( \sigma_1, \sigma_2 >> \sigma_3 \)

I am using the PETScSNES Solver to solve my nonlinear problem (an electric system in the charge coservation formulation).

Curiously only MUMPS as linear solver finds a solution (usually in one iteration).

Other linear solver fail to find a solution (usually the exit after the zeroth iteration with diverged linear search as reason).

For small test problems MUMPS was fine, now I want to go to meshes with more elements and MUMPs gets quite slow.

Therefore I want to understand why only MUMPS works, but I have no idea how to approach this.

Can anyone point my in the right direction?

(I am aware thats not a direct Fenics questions, but maybe Fenics can give me some helpful statistics?)

Thank you,

Moritz

To clarify what problem I am solving:

\[

\frac{\partial q_1}{\partial t} - \nabla \sigma_1 \nabla \Phi_1 +f_1 = 0\, on\, \Omega_1\\

\frac{\partial q_2}{\partial t} - \nabla \sigma_2 \nabla \Phi_2 +f_2 = 0\, on\, \Omega_2 \\

- \frac{\partial q_1}{\partial t} - \frac{\partial q_2}{\partial t} - \nabla \sigma_3 \nabla \Phi_3 - f_1 -f_2 = 0\, on\, \Omega_3 \\

q_1= c_1 (\Phi_1 -\Phi_3)\, on\, \Omega_1 \cap \Omega_3 \\

q_2= c_2 (\Phi_2 -\Phi_3)\, on\, \Omega_2 \cap \Omega_3 \\

f_1=k_1 exp(a_1(\Phi_1 -\Phi_3))- k'_1 exp(-a'_1(\Phi_1 -\Phi_3)) \, on\, \Omega_1 \cap \Omega_3 \\

f_2=k_2 exp(a_2(\Phi_2 -\Phi_3))-k'_2 exp(-a'_2(\Phi_2 -\Phi_3))) \, on\, \Omega_2\cap \Omega_3 \\

\Phi_2 = 0\, on\, \Gamma_2 \\

\frac{\partial \Phi_1}{\partial n} =g \,on\, \Gamma_1

\]

where \(\sigma\) c,k and a are positive and spatially varing.

\( \sigma_1, \sigma_2 >> \sigma_3 \)

Community: FEniCS Project

### 1 Answer

0

Hi Moritz,

I have not solved nonlinear problems so far and no experience with SNES. Anyway, I only use MUMPS to solve my systems of equations for the following reasons or experiences:

1. MUMPS is the only direct solver available which works in parallel for larger systems (i've not managed to install super_lu_dist for comparison and "umfpack fails with memory errors if systems become larger").

2. MUMPS is significantly faster than the "default" or "umfpack" solver if the system matrix is symmetric.

3. MUMPS still works if the matrix is very bad conditioned - if it is almost singular you can still set specific options to get a solution. I've not managed to get sufficiently accurate or even converging solutions with iterative solvers for my ill-conditioned EM problems.

4. MUMPS is not slow in my opinion if used in parallel but of course, you need sufficient CPU and RAM - how many are more elements/dofs in your case?

- Example: I can solve second-order Nedelec systems with 5M dofs with 32 procs and ~250 GB RAM in 3-7 minutes, which require significantly more effort due to reduced sparsity compared to first order polynomial systems,

In my experience, the matrix condition resulting from element size and physical contrasts and also the amount of non-zero elements on the right-hand side have a huge influence on computation times.

I hope my experience helps to understand your observations a bit better,

RR
Hi,

thank you for your insight.

I am going from 100k to around 2M (and later perhaps more) dofs.

Since it is a transient Problem I have to solve it many times.

I dont know if the other methods are faster, since I cannot test them at the moment.

I have not solved nonlinear problems so far and no experience with SNES. Anyway, I only use MUMPS to solve my systems of equations for the following reasons or experiences:

1. MUMPS is the only direct solver available which works in parallel for larger systems (i've not managed to install super_lu_dist for comparison and "umfpack fails with memory errors if systems become larger").

2. MUMPS is significantly faster than the "default" or "umfpack" solver if the system matrix is symmetric.

3. MUMPS still works if the matrix is very bad conditioned - if it is almost singular you can still set specific options to get a solution. I've not managed to get sufficiently accurate or even converging solutions with iterative solvers for my ill-conditioned EM problems.

4. MUMPS is not slow in my opinion if used in parallel but of course, you need sufficient CPU and RAM - how many are more elements/dofs in your case?

- Example: I can solve second-order Nedelec systems with 5M dofs with 32 procs and ~250 GB RAM in 3-7 minutes, which require significantly more effort due to reduced sparsity compared to first order polynomial systems,

In my experience, the matrix condition resulting from element size and physical contrasts and also the amount of non-zero elements on the right-hand side have a huge influence on computation times.

I hope my experience helps to understand your observations a bit better,

RR

thank you for your insight.

I am going from 100k to around 2M (and later perhaps more) dofs.

Since it is a transient Problem I have to solve it many times.

I dont know if the other methods are faster, since I cannot test them at the moment.

written
5 days ago by
Moritz

Please login to add an answer/comment or follow this question.

How would I attach a nullspace to the problem ? (sorry I am not a trained mathematician or computer scientist)