[Wien] mBJ + U ？

Discussion:

[Wien] mBJ + U ？

wanxiang feng

2010-07-21 08:09:29 UTC

Dear Tran and Blaha,

On theoretical background and practical program code, is it possible
to do scf and bandstructure calculation using mBJ+U(just like the
LDA+U) for some strong correlation system?
If it is feasible, how to determine the U value, is it the same as the
U in LDA+U scheme? Could you give me some literatures about this?

Thank you,

Feng,

F. Tran

2010-07-21 08:46:04 UTC

Permalink

In principle, it is possible to run a MBJ+U calculation with WIEN2k,
but the MBJ+U method is probably not a good theory for the following
reasons:

1) In LDA+U (or GGA+U), the exchange part of the double-counting term
is an (rough) approximation to the LDA (or eventually GGA) term.
Therefore, it is maybe not compatible with the MBJ exchange potential.

2) We showed that the band gap and magnetic moment of MnO, NiO, and FeO
are already improved by MBJ, which means that the value of U for the
MBJ+U method should be very small or even zero if the MBJ results are
already ok.

There are several ways to calculate U. Very often, U is adjusted such
that the theoretical results agree with experiment.

Post by wanxiang feng
Dear Tran and Blaha,
On theoretical background and practical program code, is it possible
to do scf and bandstructure calculation using mBJ+U(just like the
LDA+U) for some strong correlation system?
If it is feasible, how to determine the U value, is it the same as the
U in LDA+U scheme? Could you give me some literatures about this?
Thank you,
Feng,
_______________________________________________
Wien mailing list
Wien at zeus.theochem.tuwien.ac.at
http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien

wanxiang feng

2010-07-21 09:02:02 UTC

Permalink

Ok, I known, thanks for your reply.

Post by F. Tran
In principle, it is possible to run a MBJ+U calculation with WIEN2k,
but the MBJ+U method is probably not a good theory for the following
1) In LDA+U (or GGA+U), the exchange part of the double-counting term
is an (rough) approximation to the LDA (or eventually GGA) term.
Therefore, it is maybe not compatible with the MBJ exchange potential.
2) We showed that the band gap and magnetic moment of MnO, NiO, and FeO
are already improved by MBJ, which means that the value of U for the
MBJ+U method should be very small or even zero if the MBJ results are
already ok.
There are several ways to calculate U. Very often, U is adjusted such
that the theoretical results agree with experiment.

_______________________________________________
Wien mailing list
Wien at zeus.theochem.tuwien.ac.at
http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien

bothina hamad

2010-07-21 08:53:44 UTC

Permalink

Dear Wien users,

When running optimisation jobs under torque queuing system for anything but
very small systems:

Job runs for many cycles using lapw0, lapw1, lapw2 (parallel) successfully but eventually the 'mom-superior' node (that launches ) mpirun becomes non-communicating with the other nodes involved with the job.

At the console of this node there is correct load (4 for quad processor) and memory free... but can no longer access any nfs mounts, can no longer ping other nodes in cluster... am eventually forced to reboot node and kill job from cluster queuing system (job enters 'E' state and stays there... need to stop pbs_server and manually remove jobfiles from /var/spool/torque/server_priv/jobs... then restart pbs_server)

A similar problem is encountered on larger cluster (same install procedure) but with added problem that the .dayfile reports that for lapw2 only the 'mom-superior' node is reporting doing work (even though logging into other job nodes top reports correct load and 100%cpu use).

DOS calculation seems to work properly on both clusters...

We have used a modified x_lapw that you provided earlier.
We have been inserting 'ulimit -s unlimited' into job-scripts

We are using...
Centos5.3 x86_64
Intel compiler suite with mkl v11.1/072
openmpi-1.4.2, compiled with intel compilers
fftw-2.1.5, compiled with intel compilers and openmpi above
Wien2k v10.1

Optimisation jobs for small systems complete OK on both clusters.

The working directories for this job are large (>2GB).

Please let us know what
files we could send you from these that may be helpful for diagnosis...

Best regards
Bothina

Laurence Marks

2010-07-21 11:47:46 UTC

Permalink

Hard to know for certain, but this looks like a OS problem rather than
a WIen2k issue. Things to check:

1) Use ompi_info and check, carefully, that the compilation
options/libraries used for openmpi are the same as what you are using
to compile Wien2k.

2) For 10.1, ulimit -s is not needed for mpi (and in any case does
nothing with openmpi) as this is done in software in Wien2kutils.c.
Make sure that you are exporting environmental parameters in your
mpirun call, for instance use in parallel_options
setenv WIEN_MPIRUN "mpirun -x LD_LIBRARY_PATH -x PATH -np _NP_
-machinefile _HOSTS_ _EXEC_"

3) Check the size of the job you are running, e.g. via top, by looking
in case.output1_X. by using "lapw1 -p -nmat_only", using ganglia or
nmon, cat /proc/meminfo (or anything else you have available).
Particularly with openmpi but with some other flavors as well, if you
are asking for too much memory and/or have too many processes running,
problems occur. A race condition can also occur in openmpi which makes
this problem worse (maybe patched in latest version, I am not sure).

4) Check, carefully (twice) for format errors in the input files. It
turns out that ifort has it's own signal traps so a child can exit
without correctly calling mpi_abort. A race condition can occur with
openmpi when the parent is trying to find a child, the child does not
exist, the parent waits then keeps going....

5) Check the OS logs in /var/log (beyond my competence). You may have
too high a nfs load, bad infiniband/myrinet (recent OFED?) etc. Use
-assu buff in compilation options to reduce nfs load.

Post by bothina hamad
Dear Wien users,
When running optimisation jobs under torque queuing system for anything but
Job runs for many cycles using lapw0, lapw1, lapw2 (parallel) successfully but eventually the 'mom-superior' node (that launches ) mpirun becomes non-communicating with the other nodes involved with the job.
At the console of this node there is correct load (4 for quad processor) and memory free... but can no longer access any nfs mounts, can no longer ping other nodes in cluster... am eventually forced to reboot node and kill job from cluster queuing system (job enters 'E' state and stays there... need to stop pbs_server and manually remove jobfiles from /var/spool/torque/server_priv/jobs... then restart pbs_server)
A similar problem is encountered on larger cluster (same install procedure) but with added problem that the .dayfile reports that for lapw2 only the 'mom-superior' node is reporting doing work (even though logging into other job nodes top reports correct load and 100%cpu use).
DOS calculation seems to work properly on both clusters...
We have used a modified x_lapw that you provided earlier.
We have been inserting 'ulimit -s unlimited' into job-scripts
We are using...
Centos5.3 x86_64
Intel compiler suite with mkl v11.1/072
openmpi-1.4.2, compiled with intel compilers
fftw-2.1.5, compiled with intel compilers and openmpi above
Wien2k v10.1
Optimisation jobs for small systems complete OK on both clusters.
The working directories for this job are large (>2GB).
?Please let us know what
files we could send you from these that may be helpful for diagnosis...
Best regards
Bothina
_______________________________________________
Wien mailing list
Wien at zeus.theochem.tuwien.ac.at
http://zeus.theochem.tuwien.ac.at/mailman/listinfo/wien

--
Laurence Marks
Department of Materials Science and Engineering
MSE Rm 2036 Cook Hall
2220 N Campus Drive
Northwestern University
Evanston, IL 60208, USA
Tel: (847) 491-3996 Fax: (847) 491-7820
email: L-marks at northwestern dot edu
Web: www.numis.northwestern.edu
Chair, Commission on Electron Crystallography of IUCR
www.numis.northwestern.edu/
Electron crystallography is the branch of science that uses electron
scattering and imaging to study the structure of matter.

Laurence Marks

2010-07-21 12:03:47 UTC

Permalink

Also:

5) Use ldd of $WIENROOT/lapw1_mpi on different nodes to check that
these are correct, and also "which mpirun".

On Wed, Jul 21, 2010 at 6:47 AM, Laurence Marks

Post by Laurence Marks
Hard to know for certain, but this looks like a OS problem rather than
1) Use ompi_info and check, carefully, that the compilation
options/libraries used for openmpi are the same as what you are using
to compile Wien2k.
2) For 10.1, ulimit -s is not needed for mpi (and in any case does
nothing with openmpi) as this is done in software in Wien2kutils.c.
Make sure that you are exporting environmental parameters in your
mpirun call, for instance use in parallel_options
setenv WIEN_MPIRUN "mpirun -x LD_LIBRARY_PATH -x PATH -np _NP_
-machinefile _HOSTS_ _EXEC_"
3) Check the size of the job you are running, e.g. via top, by looking
in case.output1_X. by using "lapw1 -p -nmat_only", using ganglia or
nmon, cat /proc/meminfo (or anything else you have available).
Particularly with openmpi but with some other flavors as well, if you
are asking for too much memory and/or have too many processes running,
problems occur. A race condition can also occur in openmpi which makes
this problem worse (maybe patched in latest version, I am not sure).
4) Check, carefully (twice) for format errors in the input files. It
turns out that ifort has it's own signal traps so a child can exit
without correctly calling mpi_abort. A race condition can occur with
openmpi when the parent is trying to find a child, the child does not
exist, the parent waits then keeps going....
5) Check the OS logs in /var/log (beyond my competence). You may have
too high a nfs load, bad infiniband/myrinet (recent OFED?) etc. Use
-assu buff in compilation options to reduce nfs load.

bothina hamad

2010-07-27 10:41:34 UTC

Permalink

Dear Laurence,
Thank you for assisting us in our problems, we are still facing problems, I know it is not related to the code, but we still need the help from experts in parallel compilation.

The code works just fine on a cluster where we have compiled it using the extra '-assu buff' flag to alleviate NFS problems.
However, on another cluster that has a GPFS parallel file system and does not use NFS the only problem that we encounter is the following compile issue:

Compile time errors (if any) were:
SRC_vecpratt/compile.msg: signal ( SIGBUS, w2ksignal_bus ); /* Bus Error */
SRC_vecpratt/compile.msg: signal ( SIGBUS, w2ksignal_bus ); /* Bus Error */

Check file compile.msg in the corresponding SRC_* directory for the
compilation log and more info on any compilation problem.

I'm not sure what the vecpratt part of the code actually does.

When I look in the compile.msg file there are just warnings, not errors and looking in the SRC_vecpratt directory I see that the executables are actually built.

where is vecpratt used by the program?

Thanks in advance
Bothina

From: Laurence Marks <L-marks at northwestern.edu>
Subject: Re: [Wien] Problems in parallel jobs
To: "A Mailing list for WIEN2k users" <wien at zeus.theochem.tuwien.ac.at>
Date: Wednesday, July 21, 2010, 3:03 PM
5) Use ldd of $WIENROOT/lapw1_mpi on different nodes to
check that
these are correct, and also "which mpirun".
On Wed, Jul 21, 2010 at 6:47 AM, Laurence Marks
<L-marks at northwestern.edu>

Post by Laurence Marks
Hard to know for certain, but this looks like a OS

problem rather than

Post by Laurence Marks
1) Use ompi_info and check, carefully, that the

compilation

Post by Laurence Marks
options/libraries used for openmpi are the same as

what you are using

Post by Laurence Marks
to compile Wien2k.
2) For 10.1, ulimit -s is not needed for mpi (and in

any case does

Post by Laurence Marks
nothing with openmpi) as this is done in software in

Wien2kutils.c.

Post by Laurence Marks
Make sure that you are exporting environmental

parameters in your

Post by Laurence Marks
mpirun call, for instance use in parallel_options
setenv WIEN_MPIRUN "mpirun -x LD_LIBRARY_PATH -x PATH

-np _NP_

Post by Laurence Marks
-machinefile _HOSTS_ _EXEC_"
3) Check the size of the job you are running, e.g. via

top, by looking

Post by Laurence Marks
in case.output1_X. by using "lapw1 -p -nmat_only",

using ganglia or

Post by Laurence Marks
nmon, cat /proc/meminfo (or anything else you have

available).

Post by Laurence Marks
Particularly with openmpi but with some other flavors

as well, if you

Post by Laurence Marks
are asking for too much memory and/or have too many

processes running,

Post by Laurence Marks
problems occur. A race condition can also occur in

openmpi which makes

Post by Laurence Marks
this problem worse (maybe patched in latest version, I

am not sure).

Post by Laurence Marks
4) Check, carefully (twice) for format errors in the

input files. It

Post by Laurence Marks
turns out that ifort has it's own signal traps so a

child can exit

Post by Laurence Marks
without correctly calling mpi_abort. A race condition

can occur with

Post by Laurence Marks
openmpi when the parent is trying to find a child, the

child does not

Post by Laurence Marks
exist, the parent waits then keeps going....
5) Check the OS logs in /var/log (beyond my

competence). You may have

Post by Laurence Marks
too high a nfs load, bad infiniband/myrinet (recent

OFED?) etc. Use

Post by Laurence Marks
-assu buff in compilation options to reduce nfs load.
On Wed, Jul 21, 2010 at 3:53 AM, bothina hamad <both_hamad at yahoo.com>

Post by bothina hamad
Dear Wien users,
When running optimisation jobs under torque

queuing system for anything but