Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes fmu properties update #234

Merged
merged 10 commits into from
Jul 13, 2021
Merged

Fixes fmu properties update #234

merged 10 commits into from
Jul 13, 2021

Conversation

develaper
Copy link
Contributor

@develaper develaper commented Jun 1, 2021

UPDATE!! 09/07/2021:
Forget about those 2 branches that I was talking about yesterday!
I have discarded removing duplicated "properties". It was not going to work for the front end. Then properties is updated, then geojson[" properties"] is merged with properties updated.
I have changed the query in update_geometry to work only over the current fmu. I don't see any reason to go over all the fmus neither.

UPDATE END

UPDATE!! 08/07/2021:
I decide to split the code that was here into two different branches.
First one, this one:

  • will update the Fmu.properties column to match the relationship reflected in the fmu_operator table so properties will have company_na and operator_id.

spoiler: we don't need to sync the info at geojson.properties neither at geometry because we are not going to use it anymore
Removes properties from def update_geometry

why? First of all because it was messing around mismatching properties.
As far as I understand, we are adding to the geometries information that is already present in the geojson column.
This info is duplicated in properties and some of the information also is reflected at its own column or directly accessible:
'id', 'fmu_name', 'iso3_fmu', 'company_na', 'operator_id', 'certification_fsc', 'certification_pefc', 'certification_olb', 'certification_pafc', 'certification_fsc_cw', 'certification_tlv', 'certification_ls', 'observations', 'fmu_type_label'

  • SO, the second PR's purpose is to get rid of that annoying duplication of information deleting duplicated data from geometries, geojson and properties but retrieving the same info with the same structure to avoid any unexpected error somewhere somehow.

UPDATE END

Here I am trying to fixes the lack of sync between the information stored in the properties of some Fmus(forest management units).
fmu.rb
has_one :fmu_operator, ->{ where(current: true) }
has_one :operator, through: :fmu_operator

and its properties should look like this:
properties['company_na'] = operator&.name
properties['operator_id'] = operator&.id

I have added this task to spot "wrong" fmus and save them:
lib/tasks/fmus.rake
Saving the object triggers some clalbacks:
This one always updates geojson['properties'] so we will always call update_geometry:
before_validation :update_geojson
I have modified this one to ensure that the info in properties match the info in fmu_operator:
before_save :update_properties
And this one is a little motherfucker:
after_save :update_geometry, if: :geojson_changed?
Why?
I have discovered that after running rails fmu:current FOR_REAL=true the wrong fmus were fixed but 2 fmus that were OK get broken.
After a session of debugging I am 99% sure that nothing unexpected happens until, somehow, running update_geometry for the broken fmus brokes the properties for "some" fmus that are OK.

def update_geometry
    query =
      <<~SQL
        WITH g as (
        SELECT *, x.properties as prop, ST_GeomFromGeoJSON(x.geometry) as the_geom
        FROM fmus CROSS JOIN LATERAL
        jsonb_to_record(geojson) AS x("type" TEXT, geometry jsonb, properties jsonb )
        )
        update fmus
        set geometry = g.the_geom , properties = g.prop
        from g
        where fmus.id = g.id;
      SQL
    ActiveRecord::Base.connection.execute query
  end

If this callback is removed or simply commenting the content of the method makes all the fixing work as expected and no other fmu is harmed in the process.
SO, my biggest doubt here is why do we use this query instead of a simple update_attributes approach? And how can this query affect to a different fmu?

Copy link
Collaborator

@tsubik tsubik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I wonder why All geometeries are updated when geojson changed and not just the changed one.

app/models/fmu.rb Outdated Show resolved Hide resolved
@tsubik
Copy link
Collaborator

tsubik commented Jun 1, 2021

Maybe executing that query on ActiveRecord::Base in after_save running in not the same transaction. Maybe moving update_geometry to after_commit will help, but then geojson_changed? won't be available, there is previous_changes hash for this.

@tsubik
Copy link
Collaborator

tsubik commented Jun 1, 2021

Also when using after_commit and previous_changes the hash could not provide correct changes in if statement, when used with lambda like this -> { previous_changes['foo'].present? } just saing that this Rails version is buggy, and it worked for me when using proc instead, or checking the change in the callback message.

@develaper develaper marked this pull request as ready for review June 21, 2021 13:21
app/models/fmu.rb Outdated Show resolved Hide resolved
app/models/fmu.rb Outdated Show resolved Hide resolved
Copy link
Collaborator

@tsubik tsubik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I saw usage of properties also in calculate_current fmu_operator model. That needs to be removed as well.

app/models/fmu.rb Outdated Show resolved Hide resolved
@develaper develaper changed the title WIP Fixes fmu properties update Fixes fmu properties update Jul 8, 2021
@develaper develaper requested review from agnessa and tsubik July 8, 2021 13:41
app/models/fmu.rb Outdated Show resolved Hide resolved
app/models/fmu.rb Outdated Show resolved Hide resolved
app/models/fmu.rb Outdated Show resolved Hide resolved
@develaper develaper requested a review from tsubik July 9, 2021 15:31
@tsubik
Copy link
Collaborator

tsubik commented Jul 12, 2021

One test is failing, otherwise looks good 👍

@develaper develaper merged commit 224b59a into develop Jul 13, 2021
@tsubik tsubik deleted the fix/update_fmu_properties branch June 3, 2022 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants