dpgen数据收集分类续算

dpgen 推荐安装升级方法：删除之前工作目录的dpgen文件夹

• Install from source code: git clone https://github.com/deepmodeling/dpgen && pip install ./dpgen

想收集TL的数据集且按化学成分分类：/work/qan/kluo/NaSPO/run/TransL

dpgen collect ./ ./collect -p simplify.json -m 不起作用,只能用param.json格式

所以单独新建文件夹，构造虚拟的param.json其中初始数据集地址，直接复制最后一个train的input.json.再用dpgen collect ./ ./collect -p param.json -m就可以收集初始的数据了，但是这时候没有分类，尽管dpgen collect -m有这个功能但没有作用。

所以，构建虚拟的simplify任务，记住是精简过程，所以

     "labeled": true,

     "init_pick_number":0,

     "iter_pick_number":0,

.....

     "training_init_model": false,

运行之后，第一步就得到了 所有的分类数据集。

Q: /work/qan/kluo/NaSPO/run 最近几轮的准确率上不去，猜测是数据集太大232334 batch，而每一代的学习步长step只有50万，以及不够了。所以出现了：一些模型的准确率一直上不去，就算修改上下限也没有改观。或者一些模型学好了，另一些模型就被遗忘了，很明显是学习步长不够了。

A: 解决办法如下：将现有数据收集分类，分拣出确实学习率低的重写单独重点学习。其次，以后用dpgen自动采样产生数据集的时候，时刻关注准确率的变化，一旦上不去了赶紧分家。

D:

1.       收集数据

a.       $ dpgen collect ./ ./collect -p param.json
from https://wiki.cheng-group.net/wiki/%E8%BD%AF%E4%BB%B6%E4%BD%BF%E7%94%A8/DP-GEN

b.       或者用笨方法$ find . -name "data.0*" -type d -print -exec cp --parents -rf {} ../collect2/ \;
收集所有的data数据集https://blog.csdn.net/recher_He1107/article/details/123175582

c.       $ mkdir collect92

d.       $ dpgen collect ./ ./collect92 -p param.json
init.*是初始训练集与param的"init_data_sys"对应，sys.*是后来DP-GEN生成的训练集，与param的"sys_configs"对应。

2.       确定学习率确实低的模型

a.       "init.006","init.011","init.015","init.018","init.019","init.020","init.021","init.022","sys.009","sys.011","sys.015","sys.016","sys.020","sys.021","sys.022","sys.023","sys.026","sys.027","sys.037","sys.039","sys.043","sys.044","sys.048","sys.049","sys.050","sys.051","sys.054","sys.055"

b.       确定需要学习的模型初始结构编号: 37,39,43,44,48,49,50,51,54,55

3.       新建工作目录/work/qan/kluo/NaSPO/run01

a.       collect92  dp.sh  INCAR_scf  indata  lmp  lmp36.sh  machine.json  param.json  POTCAR_Na  POTCAR_O  POTCAR_P  POTCAR_S

4.       修改param.json

a.            "init_data_prefix": "./collect92/",
     "init_data_sys": ["init.006","init.011","init.015","init.018","init.019","init.020","init.021","init.022","sys.009","sys.011","sys.015","sys.016","sys.020","sys.021","sys.022","sys.023","sys.026","sys.027","sys.037","sys.039","sys.043","sys.044","sys.048","sys.049","sys.050","sys.051","sys.054","sys.055"],

b.            "model_devi_jobs": [{"sys_idx":[37,39,43,44,48,49,50,51,54,55],"trj_freq": 10, "template":{"lmp":"./lmp/nvt.lammps"}, "rev_mat":{"lmp":{"lmpfq": [10],"nsteps": [20000], "temps1": [10],"temps2": [2000]}}, "_idx": "93-1"}

5.       (base) [kluo@nova run01]$ nohup dpgen run param.json machine.json 1>log 2>err &

6.       一切正常，出现警告，修改重提Run
DEEPMD INFO    training data with min nbor dist: 1.083750224728927

DEEPMD INFO    training data with max nbor size: [9, 8, 5, 17]

DEEPMD WARNING sel of type 3 is not enough! The expected value is not less than 17, but you set it to 13. The accuracy of your model may get worse.