Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add set function to PrimitiveArray #97

Closed
alamb opened this issue Apr 26, 2021 · 2 comments
Closed

add set function to PrimitiveArray #97

alamb opened this issue Apr 26, 2021 · 2 comments
Labels
arrow Changes to the arrow crate

Comments

@alamb
Copy link
Contributor

alamb commented Apr 26, 2021

Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-9826

For in-place value replacement in Array, a set() function (maybe unsafe?) would be required.

@alamb alamb added the arrow Changes to the arrow crate label Apr 26, 2021
@alamb
Copy link
Contributor Author

alamb commented Apr 26, 2021

Comment from Neville Dipale(nevi_me) @ 2020-08-25T03:59:35.414+0000:

Once arrays are built, they're meant to be immutable. Wouldn't this better belong in ArrayBuilder?

Comment from Francesco Gadaleta(frag) @ 2020-08-26T13:05:41.441+0000:

Yes, it makes sense to me.

Generally speaking if arrays are immutable, there are some operations that should directly modify elements at specific index.

How are you going about that?

 

Comment from Paddy Horan(paddyhoran) @ 2020-09-07T02:50:04.369+0000:

Essentially, we are not addressing that.  I don't believe any of the other implementations support this.  It's kind of against the general principles of Arrow.

 

However, there is nothing stopping you adding this for yourself in your application.  You have access to underlying buffers.  We are trailing behind the C++ impl in terms of what we support in Rust so we will likely focus on "catching up" for now.

Comment from Jorge Leitão(jorgecarleitao) @ 2020-09-08T04:16:41.818+0000:

> Generally speaking if arrays are immutable, there are some operations that should directly modify elements at specific index.

if arrays as immutable, there should be _no_ operations that modify its elements, right?

I also understood that arrays are immutable. Some general reasons is that this allows to pass slices of data by reference, both within a thread and across threads, without the need to worry about data races or threads waiting around. It is a whole computational model.

In rust, it allow us to use {{Arc}} to safely share arrays across threads, as otherwise we would need a Mutex or other mechanism.

To "modify" elements, we create a new array with the modified elements (see e.g. {{src/compute}}).

 

 

Comment from Francesco Gadaleta(frag) @ 2020-09-08T07:13:00.133+0000:

But that can be extremely inefficient. If one needs to change a dozen values in a column of millions of elements, that can become prohibitive.
In-place value changes are quite a common operation in data science.

@tustvold
Copy link
Contributor

tustvold commented Nov 1, 2022

Closing in favour of #1981

@tustvold tustvold closed this as completed Nov 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

No branches or pull requests

2 participants