Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IO changes #134

Merged
merged 6 commits into from
Apr 23, 2022
Merged

IO changes #134

merged 6 commits into from
Apr 23, 2022

Conversation

mtb-za
Copy link
Contributor

@mtb-za mtb-za commented Apr 19, 2021

The first of a few projected PRs on this branch. Others should look at things like from_dict, from_json and so on.

This one makes changes to from_csv to use np.genfromtxt in the background.
This allows for more flexibility, and a more robust set of customisation when reading in data.

We might still want to add some new args, for things like top, base columns. In theory, the use of the names arg should fix this.

There are a fair few that will be deprecated, going forwards.

This will address #128, and part of #46

Simplifying and extending `from_csv`.

Initial test uses `np.genfromtext` internally. Explicitly expects to be told which columns the `top`, `base` and/or `thickness` are in. Other columns will be added to a dictionary from which the intervals will be created.

Addresses, but does not entirely close #128
Expanded docs. The will need to be restyled to match the rest of the library, but this will work for now.
Added a series of FutureWarnings for args that we want to remove going forward.
Added some new args.
Updated docs and args to be the same.
Changed version for deprecation to 0.9.1.
Changed tests to use new `from_csv` method.

The tests were passing in strings. These no need to be explicitly converted to `io.StringIO` objects, along with the `names=True`.
Removed extra args. These are not currently being used, but we might want them later. No need for them for now.
@coveralls
Copy link

coveralls commented Apr 19, 2021

Coverage Status

Coverage decreased (-0.5%) to 74.932% when pulling 140a86c on IO-changes into 0c68f63 on master.

Removed old `from_csv` function - it is no longer being used anyway.
@mtb-za
Copy link
Contributor Author

mtb-za commented Apr 19, 2021

@kwinkunks How do you feel about names being a mandatory argument (even if the default is just True)? If we do not have that, then we need to explicitly ask for which column is which (for important columns like top), and then we can only make a generic name for the ones that we do not actually ask for, which is not very helpful.

Having either names=True, or names=['top', 'base', 'description'] would sidestep the above potential headache. names=True will take the first row (after any skip_header) as the column names, while the other passes them in explicitly. Making it mandatory should make the code cleaner and easier to understand.

As it is, the code is using top and base if names == None, but these are not being passed, because I took them out in a commit earlier. I can put them back if we decide not to go this route easily enough.

@mtb-za mtb-za changed the base branch from master to develop July 20, 2021 13:16
@kwinkunks
Copy link
Member

Damn, I can't remember where we left this after the hackathon. Looking at it again, but ping me @mtb-za if you have anything to add!

@mtb-za
Copy link
Contributor Author

mtb-za commented Feb 15, 2022

I recall it being ready, with no major changes to the user side, but will take another, proper look.

Copy link
Member

@kwinkunks kwinkunks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to try integrating this today. I think we might have to deprecate more arguments, and probably write more tests.

@kwinkunks kwinkunks merged commit 58ca021 into develop Apr 23, 2022
@kwinkunks kwinkunks deleted the IO-changes branch April 23, 2022 11:58
@mtb-za
Copy link
Contributor Author

mtb-za commented Apr 23, 2022

Yeah, it definitely will need additional tests, or changes to existing ones.

@kwinkunks
Copy link
Member

Bottom line: it's not working, and it's going to take quite some refactoring. Knee-deep in it now.

@kwinkunks
Copy link
Member

I think I am going to have to back it out --- I can't get it to work. It needs looking at again.

kwinkunks added a commit that referenced this pull request Apr 23, 2022
This reverts commit 58ca021.
@kwinkunks kwinkunks mentioned this pull request Apr 23, 2022
kwinkunks added a commit that referenced this pull request Apr 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants