SCF Convergence Issues

Converging the SCF is a somewhat awkward aspect of computational chemistry. Closed-shell organic molecules tend to be easy to converge with modern SCF algorithms (requiring only occasional SCF tuning) while transition metal compounds and particularly open-shell transition metal compounds are troublemakers. Sometimes one is forced to spend some time coming up with SCF settings that give reliably convergence for the class of compounds one is working with, especially if one is dealing with open-shell systems and/or transition metals or other heavy elements.

The ORCA manual contains a lot of detail on SCF convergence aspects than found here while this page mainly highlights some recommended things to try when faced with SCF convergence problems.

ORCA behaviour when SCF does not converge

Since ORCA 4.0, the behaviour after SCF non-convergence was changed to make things more consistent and prevent users from accidentally using results from non-converged calculations. ORCA distinguishes between 3 cases of SCF convergence a) complete SCF convergence, b) near SCF convergence and c) no SCF convergence

Whether a calculation converges, depends of course on tolerance settings (e.g. TightSCF) as well as the set maximum number of SCF iterations (default is 125).

Near SCF convergence is defined as being not completely converged but: deltaE < 3e-3; MaxP < 1e-2 and RMSP < 1e-3; otherwise no SCF convergence is signalled.

Default behaviour for a single-point calculation when no SCF convergence or near SCF convergence is for ORCA to stop right after the SCF finishes (at MaxIter). ORCA will NOT go on to do a post-HF calculation, calculate molecular properties or excitations (e.g. TDDFT). This behaviour prevents one from accidentally using results from a calculation where the SCF is not completely converged and hence, not reliable.

Default behaviour for a geometry optimization (or gradient job) when near SCF convergence occurs for a particular Optimization Cycle, however, is for ORCA to continue with the geometry optimization. This behaviour is intended to prevent ORCA from stopping a long optimization job due to minor SCF convergence problems that usually happen in the beginning of the optimization and will usually be resolved in later optimization cycles (as the geometry is improved and ORCA reuses previous orbitals as guess for each SCF). When no SCF convergence occurs in an Optimization Cycle, ORCA stops, however, and the user is encouraged to either increase modify SCF settings and check whether the molecular geometry is reasonable.

When near SCF convergence occurs (in either single-point job or geometry optimization), the FINAL SINGLE POINT ENERGY line will make this clear:

FINAL SINGLE POINT ENERGY -137.654063943692 (SCF not fully converged!)

It is possible to modify the default behaviour if one wishes:

- Simple input keyword SCFConvergenceForced lets the user insist on a fully converged SCF for a geometry optimization. This would make a geometry optimization stop for both no SCF convergence and near SCF convergence where 'forced convergence' is not active by default. Can also be set using the SCF block : %scf ConvForced true end

Since 'forced convergence' is the default for post-HF and excited state calculations one can overrule this if this is desired:

%scf ConvForced false end

This would allow a post-HF calculation (MP2, CCSD etc.) or excited state calculation (TDDFT, CIS, ROCIS) to be carried out on a sloppily converged SCF. Molecular properties or vibrational frequencies always require a fully converged SCF, however, so that behaviour can not be overruled.

General recommendations for SCF convergence problems (RB)

• Note that since ORCA 5.0, the Trust Radius Augmented Hessian (TRAH) approach was implemented which is a robust second-order converger (but slower and more expensive) that will automatically be activated if the regular DIIS-based SCF converger in ORCA struggles to converge. With TRAH available, some of the information on this page are a bit out of date (though much of it is still applicable) and the default SCF-settings in ORCA (combination of DIIS and SOSCF with TRAH activating when problems) are usually fairly reliable for a lot of systems.

• If the SCF was almost converged (monitor DeltaE and orbital gradient) but failed to converge before the maximum number of iterations, simply increase the maximum number of iterations: e.g. %scf MaxIter 500 end Then Restart calculation using the almost converged orbitals. This is pointless, however, if the calculation showed no signs of converging.

• If the SCF is oscillating wildly in the first iterations or just converging slowly, the grid (DFT or COSX) can sometimes be the cause of the problem. This is much rarer reason in ORCA 5.0, however. Increase the grid (see Numerical accuracy). May also be a sign that damping is required (e.g. by using the Slowconv keyword). Levelshifting can also work.

• If TRAH was activated but struggles to converge or is taking a really long time it might be worth it to play around with the AutoTRAH settings first and perhaps delay before TRAH kicks in. See manual for more details:

%scf
AutoTRAH true
AutoTRAHTOl 1.125 #Threshold to determine when TRAH should be activated. Default value 1.125
AutoTRAHIter 20 #Number of iterations before interpolation is used
AutoTRAHNInter 10 #Number of iterations used in interpolation. Default 20.
end

• If TRAH is slowing things down for you, you can disable TRAH: ! NoTrah

• If the calculation seems to be close to convergence but the convergence is "trailing", (happens sometimes with DIIS) it might be a good idea to turn SOSCF on (if turned off for some reason, which is default for UHF/UKS) or try a second order method like NRSCF or AHSCF. Levelshifting can also work. SOSCF does not always work well for open-shell cases, however.

• If the SOSCF algorithm runs into trouble ("HUGE, UNRELIABLE STEP WAS ABOUT TO BE TAKEN"), try disabling SOSCF: !NOSOSCF or delay the startup of SOSCF (by choosing a lower value for the orbital gradient before SOSCF starts):

%scf

SOSCFStart 0.00033 # Default value of orbital gradient is 0.0033. Here reduced by a factor of 10.

end

• Check the geometry, is it reasonable ? If part of a geometry optimization anyway, nudge the starting geometry a bit towards a more reasonable structure.

• Try converging a simpler calculation (e.g. BP86/def2-SVP or HF/def2-SVP) and read in the orbitals from that calculation as a guess:

! MORead

%moinp "bp-orbitals.gbw"

• Try changing the Guess. PAtom, Hueckel and HCore are possible alternatives to the default PModel guess.

• Try converging a 1- or 2-electron oxidized state (ideally a closed-shell state), read in the orbitals from that solution and try again.

Converging TM complexes and other difficult molecular systems

Transition metal complexes can be difficult to converge, particularly open-shell species. ORCA has some built-in keywords that select appropriate SCF algorithm settings for such difficult systems. Using these keywords will slow down the SCF procedure and should thus only be used when necessary. For closed-shell organic molecules, the default SCF procedure or KDIIS+SOSCF (below) usually leads to much faster SCF convergence.

Keywords below mostly modify the damping parameters that aids convergence, particularly when there are large fluctuations in the first SCF iterations.

! SlowConv

! VerySlowConv

Convergence can sometimes be too slow with Slowconv. SOSCF can be used to speed up convergence, once a certain threshold is reached. Note that for open-shell systems, SOSCF is automatically turned off. Use SOSCF keyword to turn it back on if desired (it's not always suitable for open-shell systems though), possibly with a delayed start (as discussed above). Modifying the levelshift (below) can also work to speed up convergence:

! SlowConv

%scf

Shift Shift 0.1 ErrOff 0.1 end

end

KDIIS+SOSCF

Using the KDIIS algorithm with or without SOSCF as well, sometimes enables faster convergence than other SCF procedures.

! KDIIS SOSCF

If the SOSCF algorithm runs into trouble ("HUGE, UNRELIABLE STEP WAS ABOUT TO BE TAKEN"), one can remove SOSCF or it is also possible to delay the start of the SOSCF algorithm (usually necessary for TM complexes):

%scf

SOSCFStart 0.00033 # Default value is 0.0033. Here reduced by a factor of 10.

end

Converging pathological cases

For truly pathological systems, e.g. metal clusters, the following SCF settings usually results in a converged SCF while slowing down the calculation enormously (i.e. requiring both more expensive SCF iterations and usually more of them). If nothing else seems to work then these settings are worth a try.

! Slowconv

%scf

MaxIter 1500 # Here setting MaxIter to a very high number. Intended for systems that require sometimes 1000 iterations before converging (very rare).

DIISMaxEq 15 # Default value is 5. A value of 15-40 necessary for difficult systems.

directresetfreq 1 # Default value is 15. A value of 1 (very expensive) is sometimes required. A value between 1 and 15 may be more cost-effective.

end

DIISMaxEq is the value for how many Fock matrices to remember for the DIIS extrapolation. Default is 5. Values of 15-40 are more appropriate for difficult cases when DIIS is struggling.

directresetfreq determines how often the full Fock matrix should be calculated. A value of 1 means a rebuild in each iteration which is very expensive but gets rid of numerical noise that may be hindering convergence. 15 is the default. Setting directresetfreq to values inbetween 1 and 15 is probably worth exploring for finding an SCF procedure that does not take too long for your system of interest.

The Slowconv keyword is not always necessary, but useful if there are large fluctuations at the start of the SCF (damping required). The VerySlowconv keyword is useful if even larger damping is required.

Using the above settings are often the only way to reliably converge large iron-sulfur clusters in my experience (RB).

Converging conjugated radical anions with diffuse functions (BDR)

When calculating conjugated radical anions with diffuse functions (e.g. ma-def2-SVP) it has been found that a full rebuild of the Fock matrix can aid convergence as well as starting SOSCF early.

%scf

soscfmaxit 12

directresetfreq 1

end

Dealing with linear dependencies

For large basis sets or diffuse basis sets (e.g. aug-cc-pVTZ) it sometimes happens in calculations that there are a few redundant (linear dependent) functions for the system. This may give rise to numerical instabilities such as SCF convergence problems. This linear dependence can often be easily identified by inspecting the smallest eigenvalue of the overlap matrix. See manual for more details.

In the output you will find the smallest eigenvalue printed:

Diagonalization of the overlap matrix:

Smallest eigenvalue ... 4.956e-12 # A value smaller than 1e-8 or so may be an indicator of linear dependence.

The output above is a clear indicator of linear dependence in the basis set as the smallest eigenvalue is considerably smaller than 1e-08. If the basis set is non-standard it might be worth checking whether it was correctly put together (duplicate basis functions for example). Else, dealing with linear dependence is done automatically in ORCA for values smaller than 1e-08, where these eigenvalues are set to zero to remove numerical instability problems. For borderline cases such as the range 1e^-5 to 1e^-8 it may be worth exploring to increase the threshold manually:

!

%scf

sthresh 1e-7

end

However, I would also recommend asking yourself whether the basis set really is reasonable. Are all those diffuse functions on plain carbons really necessary for example?

Note that if you are running a calculation where eigenvalues are left out and then restart this calculation or read in the converged orbitals in a separate calculation, the SCF may explode due to reorthogonalization issues. The Rescue keyword prevents this.