Fixed

• Added tzdata dependency and missing logging import by @minhajuddin2510 in #99

Full Changelog: 0.13.1...0.13.2

Changed

• Reordered imports for clarity #97
• Minor Download improvements to make streaming download more robust as we're doing streaming downloads when using preview rows by doing request in a with clause. #97
• added vscode setting to use black formatter

Fixed

• added missing dependencies for pytz and python-dateutil. These new dependencies are required because of the fix in 0.13.0 that checked if a resource's metadata has been modified (c8d1a36), allowing a DP+ job even if the file hash has not changed (e.g. when the Data Dictionary data types are changed and the user wants the resource file re-pushed to use the new data types).

Full Changelog: 0.13.0...0.13.1

Added

• Add unsafe headers configuration settings. Resolves #93. This allows DP+ to use an alternate unsafe prefix when sanitizing column names. #94
• Add SNIFF_DELIMITER setting. Implements #70. This allows DP+ to automatically infer the delimiter used by a CSV file if its not a comma. #96 and 59c36d4
• The inferred Data Dictionary now also has a "Unit" column. Note that you'll still need to modify your CKAN theme to expose the Unit field in the Data Dictionary tab.

Changed

• set minimum qsv version to 0.107.0 d863b14 and 29886e8

Fixed

• Allow url parameters. Resolves #77. This allows DP+ to process links with URL parameters. Just be sure to specify the resource format to one of the supported DP+ formats so it will be processed. #95
• Properly handle when there is no timezone info when checking if a resource is updated c8d1a36

Full Changelog: 0.12.0...0.13.0

Changed

• Use single source of configuration by @bluepython508 in #82
• Containerfile dependencies by @bluepython508 in #81

Fixed

• Don't crash when not given content-length header by @bluepython508 in #85
• Use --prefer-dmy with qsv instead of --prefer_dmy by @bluepython508 in #83
• Don't crash on missing original column name by @bluepython508 in #84
• Allow reupload of file if resource metadata has changed (Fixes #88) 0e39e15.
• Reset resource.preview_rows to False if existing resource falls below preview_rows threshold 3e60765

New Contributors

• @bluepython508 made their first contribution in #85

Full Changelog: 0.11.0...0.12.0

Added

• Added-link-to-datapusher-plus-docker by @minhajuddin2510 in #67
• Added uninstallation procedure
• Added more comments in the main jobs.py process where all the main work is done
• Added details about what qsv analysis enables

Changed

• Revamped documentation to streamline installation
• set config.py to more conservative defaults
• set minimum QSV version to 0.99.0

Fixed

• Container packaging fixes by @EricSoroos in #71
• Fix error handling in validate by @Zharktas in #74
• pinned ckanserviceprovider to 1.1.0 and APScheduler to 3.9.1.post1 to fix #64

New Contributors

• @EricSoroos made their first contribution in #71
• @Zharktas made their first contribution in #74

Full Changelog: 0.10.1...0.11.0

What's Changed

• add separate AUTO_UNIQUE_INDEX setting #62
• improved Development Installation procedure
• improved Datapusher+ Configuration section, with heavily commented dot-env.template
• bumped qsv from 0.87.0 to 0.87.1, with improved safenames sanitizing...
• added qsv version checks

Full Changelog: 0.9.0...0.10.1

What's Changed

• Updated the readme to include locale installation by @minhajuddin2510 in #51
• Initial implementation of PII screening by @jqnatividad in #56

New Contributors

• @minhajuddin2510 made their first contribution in #51

Full Changelog: 0.8.0...0.9.0

More detailed release notes forthcoming...

Full Changelog: 0.7.0...0.8.0

More detailed release notes forthcoming...

What's Changed

• fix import of MutableMapping from collections.abc by @ctrepka in #49

New Contributors

• @ctrepka made their first contribution in #49

Full Changelog: 0.6.0...0.7.0

• validate excel file exported CSVs as well, as they can potentially be invalid CSVs (e.g. differing column counts per row)
• support negative values for PREVIEW_ROWS to start previewing from the end of a file (e.g. -1000 = last 1000 rows)
• if an Excel file is invalid or password-protected, show additional file metadata by using the file command
• remove obsolete CHUNK_INSERT_ROWS setting as we now do Postgres COPY
• add PREFER_DMY setting for parsing dates and doing column date inferencing (otherwise, the default is YMD)
• add logic to DROP VIEWS if ALIAS_UNIQUE is false, and show warning on datastore log
• implement smart auto-indexing which is controlled by AUTO_INDEX_THRESHOLD (default: 3) and AUTO_INDEX_DATES (default: true)
• improved log messages (comma-separated formatting for numbers, context-sensitive normalizing/transcoding messages, etc.)
• applied Black formatter to jobs.py

Full Changelog: 0.5.1...0.6.0