DeepMD training problem

Open
  • Is something wrong?
#5,950 12 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Active
Tech stack
python, tensorflow
Domain
machine-learning

Research direction

Start with the training and validation system lists in the supplied input configuration and inspect lcurve.out alongside the attached DFT-versus-prediction plots. Compare the stated data split, bootstrapped datasets, loss weights, and DeePMD-kit 3.1.1 environment to determine why the predictions deviate. Done means identifying a reproducible configuration or data issue and documenting the resulting correction or explanation.

Written by the indexing model from the issue text.

Description

wontfix
Summary

I am using DeepMD right now.
Now in licurve.out file errors are reduced. I am giving last few steps of lcurve.out

step rmse_val rmse_trn rmse_e_val rmse_e_trn rmse_f_val rmse_f_trn lr
If there is no available reference data, rmse_*_{val,trn} will print nan
......
1492000 6.66e-01 9.37e-01 7.04e-02 5.71e-02 6.04e-01 8.78e-01 1.1e-08
1493000 4.68e-01 7.73e-01 9.73e-02 5.93e-02 3.60e-01 6.81e-01 1.1e-08
1494000 8.44e-01 4.53e-01 3.80e-02 7.81e-02 7.82e-01 2.31e-01 1.1e-08
1495000 6.62e-01 6.34e-01 1.09e-01 1.13e-01 5.57e-01 5.20e-01 1.1e-08
1496000 9.56e-01 1.39e+00 1.16e-01 2.26e-01 8.53e-01 8.05e-01 1.1e-08
1497000 4.40e-01 8.34e-01 2.61e-02 4.64e-02 4.00e-01 7.63e-01 1.1e-08
1498000 1.23e+00 8.01e-01 2.10e-01 1.23e-01 1.03e+00 5.00e-01 1.1e-08
1499000 5.88e-01 4.31e-01 6.72e-02 7.78e-02 4.64e-01 1.92e-01 1.1e-08
1500000 1.06e+00 8.67e-01 7.30e-02 1.59e-01 9.88e-01 7.07e-01 1.0e-08

i am using 14 data set for each of which 80 percent is training and 20 percent for validation. datasets are increased also by bootstrapping. Now problem is that when I plot dft data and pred value plots are like

Image Image

my code is also given below
{
"_comment1": " model parameters",
"model": {
"type_map": [
"Nb",
"Co"
],
"descriptor": {
"type": "se_e2_a",
"sel":"auto",
"rcut_smth": 0.5,
"rcut": 8.00,
"neuron": [
100,
200,
400
],
"resnet_dt": true,
"axis_neuron": 20,
"type_one_side": false,
"precision": "float64",
"seed": 1,
"_comment2": " that's all"
},
"fitting_net": {
"neuron": [
100,
100,
100
],
"resnet_dt": true,
"precision": "float64",
"seed": 1,
"_comment3": " that's all"
},
"_comment4": " that's all"
},

"learning_rate": {
"type": "exp",
"decay_steps": 10000,
"start_lr":1e-4,
"stop_lr": 1e-8,
"_comment5": "that's all"
},
"loss": {
"type": "ener",
"start_pref_e": 0.02,
"limit_pref_e": 1,
"start_pref_f": 1000,
"limit_pref_f": 1,
"start_pref_v": 0,
"limit_pref_v": 0,
"_comment6": " that's all"
},
"training": {
"training_data": {
"systems": [
"Data_8new/data_0",
"Data_8new/data_1",
"Data_8new/data_2",
"Data_8new/data_3",
"Data_9new/data_0",
"Data_9new/data_1",
"Data_9new/data_2",
"Data_9new/data_3",
"Data_10new/data_0",
"Data_10new/data_1",
"Data_10new/data_2",
"Data_10new/data_3",
"Data_11new/data_0",
"Data_11new/data_2",
"Data_11new/data_3",
"Data_11new/data_4",
"Data_12new/data_0",
"Data_12new/data_2",
"Data_12new/data_3",
"Data_12new/data_4",
"Data_13new/data_0",
"Data_13new/data_2",
"Data_13new/data_3",
"Data_13new/data_4",
"Data_14new/data_0",
"Data_14new/data_1",
"Data_14new/data_3",
"Data_14new/data_4",
"Data_15new/data_0",
"Data_15new/data_1",
"Data_15new/data_3",
"Data_15new/data_4",
"Data_16new/data_0",
"Data_16new/data_1",
"Data_16new/data_3",
"Data_16new/data_4",
"Data_17new/data_0",
"Data_17new/data_1",
"Data_17new/data_2",
"Data_17new/data_4",
"Data_18new/data_0",
"Data_18new/data_1",
"Data_18new/data_2",
"Data_18new/data_4",
"Data_19new/data_0",
"Data_19new/data_1",
"Data_19new/data_2",
"Data_19new/data_4",
"Data_20new/data_0",
"Data_20new/data_1",
"Data_20new/data_2",
"Data_20new/data_4",
"Data_21new/data_0",
"Data_21new/data_1",
"Data_21new/data_2",
"Data_21new/data_4"
],
"batch_size": "auto",
"_comment7": "that's all"
},
"validation_data": {
"systems": [
"Data_8new/data_4",
"Data_9new/data_4",
"Data_10new/data_4",
"Data_11new/data_1",
"Data_12new/data_1",
"Data_13new/data_1",
"Data_14new/data_2",
"Data_15new/data_2",
"Data_16new/data_2",
"Data_17new/data_3",
"Data_18new/data_3",
"Data_19new/data_3",
"Data_20new/data_3",
"Data_21new/data_3"
],
"batch_size": "auto",
"numb_btch": 1,
"_comment8": "that's all"
},
"numb_steps": 1500000,
"seed": 10,
"disp_file": "lcurve.out",
"disp_freq": 1000,
"save_freq": 10000,
"_comment9": "that's all"
},
"_comment10": "that's all"
}

can you pls help where I am going wrong??

DeePMD-kit Version

3.1.1

Backend and its version

TensorFlow 2.19.1

Python Version, CUDA Version, GCC Version, LAMMPS Version, etc

No response

Details

Training problem occur, error decreases but deviation between DFT and potential predicted value. details are mentioned above

Dominant language
Python
Stars
2.1k
Forks
663
Avg merge
3d 11h
Merged PRs (30d)
25

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from deepmodeling/deepmd-kit

All issues in deepmodeling/deepmd-kit

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.