Import feature: show the differences of an element when it is updated by an import (and refactor) #810

MyPyDavid · 2023-11-01T15:52:10Z

Description

This PR implements a text difference viewer for the import of RDMO content elements (catalogs, optionsets, etc..).
It uses the react-diff-viewer-continued to display the changes during an import. In the back-end a sort of "track changes" function tracks the current and new state of each element which is returned in the json response.
The code architecture of the import is refactored into one central import function under management with helpers for each type of element.

Related issue: #468

Includes bug fix for: #998

Motivation and Context

How has this been tested?

Screenshots (if appropriate)

Currently:

previous version

![management-import-optionsets-1-changes](https://github.com/rdmorganiser/rdmo/assets/13996213/e5fcb01c-bc8e-4c86-9b89-4318fd015c47)

older previous version

![management-import-optionsets-1-changes](https://github.com/rdmorganiser/rdmo/assets/13996213/afec9fae-839b-496f-a817-5329f4167a45)

Types of Changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Code style update (formatting, renaming)
- some type hints have been added to the new code
Refactoring (no functional changes, no api changes)
Build related changes
Documentation content changes
Other (please describe):

Checklist

I have read the contributor guide.
My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.
- also added more front end e2e tests
All new and existing tests passed.

Still todo

Front-end

add filter on top of import page
add info messages summary on top of import page
button to show only the changed elements

rdmo/management/assets/js/components/main/Import.js

MyPyDavid · 2024-04-26T16:02:13Z

fix the actions in the sidebar

package-lock.json

rdmo/conditions/imports.py

rdmo/core/constants.py

rdmo/core/import_helpers.py

rdmo/core/xml.py

rdmo/management/imports.py

jochenklar · 2024-05-16T14:00:13Z

I am sorry, there is more work to do on the css. Paddings and margins are off all over the place.

jochenklar · 2024-05-16T15:17:28Z

The padding needs to align with the rest of the panel.

I think we should only use the <code> for URIs or Element/Model names. Just Normal font should suffice. Also the "Changed:" label in the front is redundant.

Signed-off-by: David Wallace <[email protected]>

…1 fixture Signed-off-by: David Wallace <[email protected]>

Signed-off-by: David Wallace <[email protected]>

MyPyDavid · 2024-07-03T09:52:06Z

rebased to dev-2.2.0 and force pushed

jochenklar · 2024-07-03T09:54:11Z

I think new is great since we use new/neu also in the interface.

Signed-off-by: David Wallace <[email protected]>

MyPyDavid · 2024-07-03T10:43:49Z

ok, changed the text to "new"
and added (new from dev-220) the short_title field to the import helpers

jochenklar · 2024-07-04T13:39:44Z

Sorry I found more bugs:

if you upload and import a faulty file the message is missleading:

with this xml

<?xml version="1.0" encoding="UTF-8"?>
<rdmo xmlns:dc="http:https://purl.org/dc/elements/1.1/" created="2024-07-01T10:53:07.185903+02:00" version="2.2.0.dev1">
	<question dc:uri="http:https://example.com/terms/questions/catalog/blocks/set/block/b">
		<uri_prefix>http:https://example.com/terms</uri_prefix>
	</question>
</rdmo>

(GitHub does not allow XML uploads)

For the same file: I got this

after I click Question or click show. I think this is also because of the broken xml file, so it is not super important.

I would remove the onClick on the created/updated badge or add cursor: pointer; using css, upt to you.

MyPyDavid force-pushed the feat-import-diff branch from 46365bd to 052c154 Compare November 2, 2023 07:43

jochenklar force-pushed the dev-2.1.0 branch from d8dbae1 to 6cc0dbd Compare November 16, 2023 13:01

jochenklar assigned MyPyDavid Nov 24, 2023

jochenklar added this to the 2.2.0 milestone Nov 24, 2023

jochenklar changed the base branch from dev-2.1.0 to dev-2.2.0 December 12, 2023 13:46

MyPyDavid added status:hold status:work-in-progress labels Dec 14, 2023

jochenklar force-pushed the dev-2.2.0 branch from ca32db7 to d10fa87 Compare December 21, 2023 16:22

MyPyDavid force-pushed the dev-2.2.0 branch from d10fa87 to 4e64e34 Compare January 10, 2024 16:24

MyPyDavid force-pushed the feat-import-diff branch from 98b7154 to 04d332a Compare January 10, 2024 17:21

MyPyDavid force-pushed the dev-2.2.0 branch from 4e64e34 to 6debf66 Compare January 17, 2024 08:22

MyPyDavid force-pushed the feat-import-diff branch from f797f10 to c35469e Compare January 29, 2024 17:19

MyPyDavid force-pushed the feat-import-diff branch from 55379af to 8995ac2 Compare February 8, 2024 12:37

MyPyDavid force-pushed the dev-2.2.0 branch from 6debf66 to 8581b6d Compare February 16, 2024 07:36

MyPyDavid changed the title ~~Show the differences of an element when it is updated by an import~~ Show the differences of an element when it is updated by an import (and refactor import) Apr 4, 2024

MyPyDavid force-pushed the feat-import-diff branch from 267f5b1 to 21722cc Compare April 5, 2024 07:30

MyPyDavid added status:acceptance and removed status:hold status:work-in-progress labels Apr 5, 2024

MyPyDavid marked this pull request as ready for review April 11, 2024 09:52

MyPyDavid requested review from jochenklar and CalamityC April 11, 2024 09:52

CalamityC requested changes Apr 25, 2024

View reviewed changes

rdmo/management/assets/js/components/main/Import.js Outdated Show resolved Hide resolved

rdmo/management/assets/js/components/main/Import.js Show resolved Hide resolved

CalamityC approved these changes Apr 29, 2024

View reviewed changes

MyPyDavid changed the title ~~Show the differences of an element when it is updated by an import (and refactor import)~~ Import feature: show the differences of an element when it is updated by an import (and refactor) May 15, 2024

MyPyDavid added the type:feature-request label May 15, 2024

jochenklar requested changes May 16, 2024

View reviewed changes

MyPyDavid and others added 22 commits July 3, 2024 11:34

tests(import): update and fix import tests

9a357b8

Signed-off-by: David Wallace <[email protected]>

js(import): update and refactor import

41c5a3b

Signed-off-by: David Wallace <[email protected]>

tests(multisite): update imports for multisite tests

6210082

Signed-off-by: David Wallace <[email protected]>

Adjust CSS in new import interface

da73954

Adjust CSS for diff feature in new import interface

5712e2c

refactor(js, import): remove extra ul from warnings

9b62bda

Signed-off-by: David Wallace <[email protected]>

refactor(import): update import command error comment

8533368

Signed-off-by: David Wallace <[email protected]>

feat(import): update style and fix display bugs for import

5a26e8d

Signed-off-by: David Wallace <[email protected]>

style(import): remove row and col- from divs in Warnings and Errors

0b84b05

Signed-off-by: David Wallace <[email protected]>

style,refactor(import): display errors and warnings in list-groups

dae5872

Signed-off-by: David Wallace <[email protected]>

tests(import): add one condition that raises a warning to optionsets-…

29609cd

…1 fixture Signed-off-by: David Wallace <[email protected]>

feat(import): use label-info for changed

167cf1c

Signed-off-by: David Wallace <[email protected]>

Adjust warning panel in import

2b626f1

Fix ImportSuccessElement component

f82d589

Fix whitespaces

bb57561

Adjust error panel im import

f20109f

Change ShowLink to chevron

b13b443

refactor(js,import): add components for import labels

a384f8d

Signed-off-by: David Wallace <[email protected]>

feat(js,import): fix updating key for attributes at import

2766b0b

Signed-off-by: David Wallace <[email protected]>

fix(domain,import): use value from helper when needed

f241878

Signed-off-by: David Wallace <[email protected]>

feat(js, import): change label texts during import

b5a0353

Signed-off-by: David Wallace <[email protected]>

refactor(js, import): rename to buildPathForAttribute and remove extras

550aeb5

Signed-off-by: David Wallace <[email protected]>

MyPyDavid force-pushed the feat-import-diff branch from 038ee3f to 550aeb5 Compare July 3, 2024 09:49

MyPyDavid added 2 commits July 3, 2024 12:41

feat(js, import): change create to new

fc431e5

Signed-off-by: David Wallace <[email protected]>

feat(import): add short_title to section and page

7649ef8

Signed-off-by: David Wallace <[email protected]>

MyPyDavid removed the status:work-in-progress label Jul 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import feature: show the differences of an element when it is updated by an import (and refactor) #810

Import feature: show the differences of an element when it is updated by an import (and refactor) #810

MyPyDavid commented Nov 1, 2023 •

edited

Loading

MyPyDavid commented Apr 26, 2024 •

edited

Loading

jochenklar commented May 16, 2024

jochenklar commented May 16, 2024 •

edited

Loading

MyPyDavid commented Jul 3, 2024

jochenklar commented Jul 3, 2024

MyPyDavid commented Jul 3, 2024

jochenklar commented Jul 4, 2024

Import feature: show the differences of an element when it is updated by an import (and refactor) #810

Are you sure you want to change the base?

Import feature: show the differences of an element when it is updated by an import (and refactor) #810

Conversation

MyPyDavid commented Nov 1, 2023 • edited Loading

Description

Motivation and Context

How has this been tested?

Screenshots (if appropriate)

Types of Changes

Checklist

Still todo

MyPyDavid commented Apr 26, 2024 • edited Loading

jochenklar commented May 16, 2024

jochenklar commented May 16, 2024 • edited Loading

MyPyDavid commented Jul 3, 2024

jochenklar commented Jul 3, 2024

MyPyDavid commented Jul 3, 2024

jochenklar commented Jul 4, 2024

MyPyDavid commented Nov 1, 2023 •

edited

Loading

MyPyDavid commented Apr 26, 2024 •

edited

Loading

jochenklar commented May 16, 2024 •

edited

Loading