What's Changed

Breaking change

• breaking: use shlex.quote to quote shell commands by @njzjz in #1396
  Single quotes (') can be automatically escaped. One does not need to escape single quotes in command manually anymore. To those who have escaped single quotes in command manually, this is a breaking change.

Enhancement

• exclude docs and tests from sdist by @njzjz in #1385
• tests: add UT for run_model_devi by @njzjz in #1386
• throw error if training_iter0_model_path is not found by @njzjz in #1391
• pwscf: write floaing point number for atomic masses by @Yi-FanLi in #1387
• model devi: download LAMMPS log files for pimd by @Yi-FanLi in #1406
• only log the current iteration when restarting by @njzjz in #1420
• support collecting MultiSystems by @njzjz in #1422
• keep original model_devi.out files for pimd by @Yi-FanLi in #1472

Documentation

• docs: add links to external documentation by @njzjz in #1392
• docs: add the link for how to setup the conda environment by @njzjz in #1484
• fp: add document and example for Quantum Espresso by @Yi-FanLi in #1405
• dpgen simplify: add document for fp style pwscf by @Yi-FanLi in #1413

CI/CD

• CI: Setup Dependabot for GitHub Actions by @njzjz-bot in #1492
• CI: Bump codecov/codecov-action from v3 to v4 by @njzjz-bot in #1491

Bugfix

• Fix UTs destroyed by pymatgen's update by @Yi-FanLi in #1471
• Bugfix for pimd: sorting model_devi files by @Yi-FanLi in #1470
• issue1475: a small bug of iterdict (generator/lib/cp2k.py) by @yohsama in #1476

New Contributors

• @yohsama made their first contribution in #1476
• @njzjz-bot made their first contribution in #1492

Full Changelog: v0.12.0...v0.12.1

Breaking Changes

• drop Python 3.8 support by @njzjz in #1264
• breaking: remove default values for training reuse arguments by @njzjz in #1330

New features

• add dp_train_skip_neighbor_stat argument by @njzjz in #1179
• support mixed:N batch size by @njzjz in #1183
• support one_h5 for simplify model_devi step by @njzjz in #1185
• simplify: allow initial models trained by initial data by @njzjz in #1201
• support automatic training_reuse_old_ratio by @njzjz in #1209
• split make_fp and share make_fp_calculation between run and simplify by @njzjz in #1235
• add fp_style custom by @njzjz in #1244
• simplify: support model deviation of energy per atom by @njzjz in #1312
• simplify: support using true error as error indicator by @njzjz in #1321
• dprc: add restart_from_iter option by @njzjz in #1322
• support electronic tempeture data used in simplify; refactor post_fp_vasp by @njzjz in #1329
• support training with -f or -t argument by @njzjz in #1340
• Add dpgui entry point and dpgen gui CLI by @njzjz in #1349
• model_devi: add support for pimd by @Yi-FanLi in #1366

Enhancement

• refactor(abacus): modify poscar2stru in auto_test.lib.abacus by @pxlxingliang in #1193
• simplify: improve performance of selecting frames by @njzjz in #1202
• Use get_global_number_of_atoms() instead by @njzjz in #1256
• load files according to filename suffix by @njzjz in #1255
• check arguments with 2D lists by @njzjz in #1258
• throw RuntimeError if no system is found by expand_sys_str by @njzjz in #1292
• rm old files when performing dp model-devi by @njzjz in #1332
• argcheck: restrict the type of elements in a list by @njzjz in #1364
• Update {Elastic,run,surf,Vasp,vasp}.py: To be compatible with Pymatgen by @ZhouXY-PKU in #1302

Bugfixes

• fix(abacus): fix deepks bug in autotest by @pxlxingliang in #1163
• fix(abacus): get ION step by searching "ION=" in relax job by @pxlxingliang in #1169
• fix a bug in convert_training_data_to_hdf5 by @njzjz in #1167
• fix batch size for simplify by @njzjz in #1184
• fix(abacus): Compatibility with kspacing set to 3 values by @pxlxingliang in #1197
• simplify: do not symlink if the file does not exist by @njzjz in #1191
• Fix the numerical mismatch for 02.fp/task/cp2k_output after update dpdata by @robinzyb in #1219
• fix(abacus): put pp/orb/dpks files to folder pporb in 02fp. by @pxlxingliang in #1214
• fix bug in make_supercell_abacus by @Liu-RX in #1212 and #1229
• len(all_sys) should > 0 for post_fp_cp2k by @robinzyb in #1245
• fix a typo in new_to_old_ratio message by @njzjz in #1270
• fix a bug in the number of frames for auto ratio by @njzjz in #1276
• Using dict.get methods to get machine.local_root by @dingzhaohan in #1287
• fix KeyError in init_reaction by @njzjz in #1352
• fix Python 3.12 compatibility by @njzjz in #1367
• abacus: fix bug of finding the final relax STRU by @pxlxingliang in #1344

Documentation

• fix the links of DP-GUI by @njzjz in #1187
• change size and apply dark mode for the logo by @njzjz in #1190
• extend the doc of cp2k external_input_path in detail by @robinzyb in #1225
• fix: description in README.md by @cherushui in #1241
• docs: move ratio_failed out of fp variant by @njzjz in #1243
• docs: fix docs for fp_task_min by @njzjz in #1257
• update deepmd input to v2; format JSON files by @njzjz in #1315
• update issue templates by @njzjz in #1368
• docs: update installation documentation by @njzjz in #1372

CI/CD

• bump pymatgen to a stable version by @njzjz in #1186
• bypass the upstream pymatgen error for CI by @njzjz in #1233
• bump pymatgen to 2023.5.31 by @njzjz in #1250
• use OIDC to publish packages by @njzjz in #1298
• test: pin pymatgen-analysis-defects to <2023.08.22 by @njzjz in #1313
• skip Sphinx 7.2.5 by @njzjz in #1324

Lint

• apply ruff rules and fix lots of bugs by @njzjz in #1240
• move to ruff formatter by @njzjz in #1375

New Contributors

• @pre-commit-ci made their first contribution in #1171
• @ZhouXY-PKU made their first contribution in #1302
• @Yi-FanLi made their first contribution in #1366

Full Changelog: v0.11.1...v0.12.0

Breaking changes:

• breaking: remove old dispatcher by @njzjz in #1097

Enhancements:

• Add parameter 'srtab_file_path' so that one can use DP-ZBL potential successfully by @Chengqian-Zhang in #918
• Refactor: remove association between ABACUS/STRU and the element number in param.json by @pxlxingliang in #1076
• choose numb_steps/stop_batch automatically by @njzjz in #1086
• replace distutils.version.LooseVersion by packaging.version.Version by @njzjz in #1095
• Support forward files for simplify by @HuangJiameng in #1092
• Replace "python3" in command by sys.executable by @HuangJiameng in #1105
• add an option to merge data to one H5 file by @njzjz in #1119
• add and run pre-commit to format codes by @njzjz in #1124
• refactor(abacus): link the pp/orb files defined in STRU not jdata by @pxlxingliang in #1127
• Raise warnings if there is no system in the path listed in sys_configs by @HuangJiameng in #1130
• Updated the revise_lmp_input_dump function. by @panxiang126 in #1137
• add isort to sort imports by @njzjz in #1133
• always pass codecov status by @njzjz in #1157

Documentation, argument parsing and examples

• add placeholders for variants
  by @njzjz in #1068
• Doc(abacus): modify the description of parameters by @pxlxingliang in #1080
• fix doc: "structures" should be a list by @HuangJiameng in #1091
• docs: add logo to docs and README by @njzjz in #1128
• docs: add links to DP-GUI
  by @njzjz in #1131

Bug fixings:

• Fix: remove the reading of ntype when treating ABACUS related info by @pxlxingliang in #1075
• fix pass action by @njzjz in #1082
• fix compatibility with NumPy and pymatgen by @njzjz in #1085
• Fix(abacus): parse orbital information after parsing of INPUT by @pxlxingliang in #1079
• fix_bug simplify.py by @Vibsteamer in #1113
• fix_bug simplify.py by @Vibsteamer in #1114
• fix compatibility with the latest pymatgen by @njzjz in #1126
• fix restart trajectory for merge_traj by @wankiwi in #1111
• fix a bug in make_abacus_relax by @maki49 in #1150
• fix(abacus): do not write to deepmd raw/npy, if no abacus data in post_fp_abacus_scf() by @pxlxingliang in #1153
• bug_fix reproduce.py by @Vibsteamer in #1156
• fix numpydoc styles by @njzjz in #1134

New Contributors

• @wankiwi made their first contribution in #1111

Full Changelog: v0.11.0...v0.11.1

Breaking changes:

• bump requires-python to >=3.8 (#1014)

New features:

• gamma line calculation for bcc and fcc systems (#801)
• support several CALYPSO run in different pressure in one iteration (#814)
• autotest: add the interface of ABACUS in autotest (#834)
• support merge_traj for lammps (#838 #961 )
• support HDF5 files as initial data system (#973)
• simplify: support list for pick_data (#994)

Enhancements:

• refactor: delete relax_kspacing and md_kspacing in abacus init step (#848)
• check run_tasks before submission in run_md_model_devi (#854 )
• UT: add UT for create_disturb_abacus_dev (#852)
• Automatically set the relative atomic mass according to "type_map" (#867 #893)
• add tests to check machine files (#874)
• UT: update test_disturb_abacus.py (#879)
• add an error remind in dpgen_fp when NSW is not set to 1 (#903)
• move some imports into runtime functions (#894)
• support forward files for init_surf (#909)
• fp: check failed tasks (#882)
• remove requirements.txt (#948)
• migrate from setup.py to pyproject.toml (#995)
• _crd2frag: use OB native PBC implementation (#1006)
• skip mirror gitee when organization is not deepmodeling (#1048)
• add pass step to GitHub Actions (#1058)

Documentation, argument parsing and examples

• args: ensembles -> ensemble (#805)
• args: model_devi_v_trust_lo and model_devi_v_trust_hi are optional (#804)
• removed model_devi_e_trust_lo and model_devi_e_trust_hi, which have been removed in #79 (#807)
• args: add shuffle_poscar to run jdata (#806)
• fix readme and qm7.json (#817)
• args: trust level for model_devi_job should be optional (#818)
• args: model_devi_nopbc should be optional (#819)
• args: fp_aniso_kspacing and cvasp should be optional (#820)
• args: add entire training parameters (#829)
• add entire arguments of gaussian style (#830)
• fix deprecated bare pair_coeff (#831)
• UT checks examples (#821 #870 )
• change 'dpgen test' to 'dpgen autotest' (#875)
• args: remove duplicated argument (#839)
• update abacus input parameters (#836)
• update the type of fp_aniso_kspacing as List of float (#843)
• document: add the information of ABACUS in arginfo.fp_style_variant_type_args (#845)
• add DP-GEN Structure.md to doc (#844)
• doc: add FAQs (#856)
• remove documentation for old dispatcher (#862)
• add dargs to install_requires (#863)
• example: add abacus examples (#853)
• update run/dp2.x-lammps-cp2k example (#864)
• add simplify example of MAPbI3-scan based on lebesgue (#840)
• docs: add authors section for credits (#869)
• docs: add init surf (#890 #904)
• docs: ddd a Troubleshooting.md file (#888)
• docs: user guide (#876)
• docs: add numpydoc; set proper info (#895)
• example_dp2.x_lmp_gaussian (#887)
• docs: update common-errors.md (#897)
• fix typos and optimize autotest part in README (#899)
• add DPRc (model_devi_engine amber and fp_style amber/diff) arguments (#911)
• update simplify doucment (#912)
• add init-bulk in dpgen documents (#907)
• docs: clarify the usage of skip_relax in init_surf (#929)
• docs: optimize autotest document based on #891 (#923)
• manage TOC of DPGEN's document (#932)
• add publication link to docs (#939)
• move ch4 folder to examples directory (#949)
• fix the description of fp_task (#950)
• loose check dpgen run jdata (#952)
• optimize document and error report based on FAQs (#967)
• remove personal info in dp2.x-lammps-gaussian example (#972)
• args: add init_bulk_jdata_arginfo and init_surf_jdata_arginfo (#936)
• strict check simplify jdata (#951)
• fix : autotest relaxation TOC (#982)
• merge run-process.rst to index.rst (#966)
• docs: add links to keys (#981)
• docs: init_surf forward_files (#1025)
• refactor readme and remove some duplicated contents (#946)
• add arguments for lammps revise_template (#1015)
• docs: add information about Python requirements (#1060)

Bug fixings:

• fix post_fp_vasp and make_fp_configs for simplify (#803 )
• fix UT test_make_md (#812)
• fix bug of matrix multiply in create_disturbs_abacus_dev (#815)
• set type_map for MultiSystems in model_devi step (#825)
• test: using assertEqual to compare two value in test_vasp.py (#828)
• autotest: delete the redundant print in test_vasp.py (#835)
• fix bug of no funding fp_params (#842)
• fix a bug in run_abacus_md and delete redundant line about gamma_only. (#850)
• fix: incorrect using of range in auto_test/lib/abacus.py (#884)
• fix: auto_test: equi_stress should have the same unit as stress (#910)
• fix: skip cell_type if from_poscar is True in init_bulk (#930)
• fix: fix forward_files for merge_traj in run_md_model_devi (#947)
• remove _link_fp_vasp_pp from make_fp_gaussian (#971)
• fix incompatibility of autotest module with latest pymatgen>2022.7.19 (#963)
• fix: modify the reading of parameter 'fp_dpks_descriptor' in run.py (#989)
• CVE-2007-4559 Patch (#990)
• fix ImportError (#1001)
• python3.10 support: collection.Mapping -> collection.abc.Mapping (#1020)
• check data instead of tasks (#1031)
• assign type_map for all systems (#1033)
• fix codecov (#1034)
• ignore dpdata 0.2.11 (#1062)

Full Changelog: v0.10.6...v0.11.0

New features:

• add DPRc implementation by @njzjz in #729

Enhancements:

• add link to DP-GUI by @njzjz in #724
• add many docs and examples by @njzjz in #736 , #758 , #751 , #759 , #762 , #747 , #788 , #787
• refactor simplify by @njzjz in #730
• supports non-list mdata by @njzjz in #748
• Create Performing the Run process.md by @LiangWenshuo1118 in #714
• upgrade all tasks to dpdispatcher by @njzjz in #749 , #755
• update: add a new option of absolute volume in /dpgen/dpgen/auto_test/EOS.py by @HuangJiameng in #741
• remove train_param from examples, which was deprecated in #68 by @njzjz in #789
• Add netcdf4 dependency to conda recipe by @AnguseZhang in #792

Bug fixings:

• fix a bug in make_abacus_scf_input by @Liu-RX in #754
• correct the wrong spelling of 'failure' by @HuangJiameng in #764
• fix ‘post_fp_cp2k’, add param rfailed by @HuangJiameng in #765
• Fix simplify by @HuangJiameng in #767 , #768
• fix ZeroDivisionError raised by run.py by @HuangJiameng in #769
• fix bug for sys_idx dependent trust level and add option for model_devi_job by @Cloudac7 in #786

New Contributors

• @HuangJiameng made their first contribution in #741

Full Changelog: v0.10.5...v0.10.6

Breaking changes:

• ABACUS versions < 2.2 are not supported anymore.

New features:

• support Calypso structure search in the exploration stage (#663)
• add bcc interstitial calculation in autotest (#696)
• Add ABACUS/lcao deepks interface to dpgen init and run, upgrade to support ABACUS >= 2.2. (#705)

Enhancements:

• change some default parameters in lmp when doing autotest (#696)
• batch replace 'except:' with 'except Exception:' (#703)
• doc: dpgen run machine parameters (#710)
• Delete information printed by dpgen -h to enable pip install dpgen (#718)

Bug fixings:

• Update param_CH4_deepmd-kit-2.0.1.json (#694)
• Fix the usage of pair_coeff in auto_test (#696 #698)
• Fix QE pwscf AssertionError (#685)

New features:

• Added ABACUS pw interface for dpgen init_bulk (#590)

Improvements:

• Add minimize step with fix 1 all box/relax tri 0.0 to autotest/relax step (#631)
• Update simplify.py for api > 1.0, and fix multi-suffixes bug . (#670)
• Make Autotest compatible with the new version of dpdispatcher. (#667)
• Add deepmodeling banner to doc (#688)
• Set the seed of type_embedding network (#689)

Bug fixings:

• Bug introduced by pr#611 (#626)
• Add convert_mdata definition to fix simplify bug. (#634)
• Fix cp2k post fp numerical mismatch (#644)
• pwscf gamma sampling; fix the conflict between loc_frame descriptor and automatic sel label generation (#650)

New features:

• support dp compress (#607)
• Adapt different trust level for different sys_configs (#609)

Enhancement:

• Several enhancement in gromacs engine (#553)
• autotest: run relaxation tasks in parallel (#573)
• update abacus-scf generator (#575)
• add lebesgue ch4 example (#571)
• model_devi_clean_traj support int type (#583)
• add a citation badge (#589)
• bump dpdata version to 0.2.4 (#595)
• change compress level to 6 from default 9 to accelerate compress (#593)

Bug fixings:

• Update gen.py | compatibility with VASP (#529)
• bug_fix | init_bulk | stages 3 | lost the last pert-dir in stages 2 (#530)
• ssh: archieve files the symlink points to instead of symlink iteself (#552)
• Change mv to cp when downloading softlinks. (#562)
• fix the usage of pair_coeff (#567)
• Fix examples in INCAR_md and POTCAR (#570)
• fix dispatcher bugs in autotest run (#568)
• autotest: fix early return and result checking (#574)
• autotest: rename task directory to avoid false result in a rerun (#572)
• Fix bug in make_submission of Auto-test (#577)
• Add ase, custodian, GromacsWrapper in conda packages. (#578)
• remove jr.json to allow rerun in autotest make (#579)
• will convert local_root's './' to abspath to avoid hash conflict in new dpdispatcher (#582)
• fix bug issue: #597 (#598)
• init_bulk: skip relax if skip_relax is true (#594)
• bug_fix | to support multi-element cases (#608)
• bug_fix | typo in setting default KSPACING (#611)
• model_devi | sort models in default&support user-defined order (#610)

Bug fixing:

• fix typo in github actions release to conda #524

New features:

• Allow user defined forward_common_files and backward_files (#482)
• Adaptive trust levels (#495)
• Support relative force model deviation by normalizing the RMS force magnitude (#496)

Enhancement:

• Update gromacs model devi engine #493
• add delete_apg interface in dpgen/dispatcher/ALI.py #519

Bug fixings:

• fix bug with 1-dim model_devi_activation_func (#484)
• exclude no converged task for cp2k fp (#486)
• rename n_neuron to neuron (#492)
• delete dump.0.xyz in back_forward file #498
• cant get group_size in older ALI-Dispatcher #520