Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add takewhile, dropwhile to iterators #33437

Merged
merged 23 commits into from
Oct 10, 2019
Merged

Add takewhile, dropwhile to iterators #33437

merged 23 commits into from
Oct 10, 2019

Conversation

o314
Copy link
Contributor

@o314 o314 commented Oct 1, 2019

Constating that python, ruby, rust, elixir all offer takewhile, dropwhile

language take while drop while
python takewhile dropwhile
ruby take_while drop_while
rust take_while skip_while
elixir take_while drop_while

And also dotnet, java, scala !

I think Julia should have them too.

IterTools.jl, and rust have been consulted.

base/iterators.jl Outdated Show resolved Hide resolved
base/iterators.jl Outdated Show resolved Hide resolved
base/iterators.jl Outdated Show resolved Hide resolved
Copy link
Member

@ararslan ararslan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of stylistic fixes needed but I agree that the functionality would be nice to have.

base/iterators.jl Outdated Show resolved Hide resolved
base/iterators.jl Outdated Show resolved Hide resolved
test/iterators.jl Outdated Show resolved Hide resolved
test/iterators.jl Outdated Show resolved Hide resolved
test/iterators.jl Outdated Show resolved Hide resolved
test/iterators.jl Outdated Show resolved Hide resolved
@ararslan ararslan added needs compat annotation Add !!! compat "Julia x.y" to the docstring needs docs Documentation for this change is required needs news A NEWS entry is required for this change labels Oct 1, 2019
base/iterators.jl Outdated Show resolved Hide resolved
base/iterators.jl Outdated Show resolved Hide resolved
base/iterators.jl Outdated Show resolved Hide resolved
@ararslan
Copy link
Member

ararslan commented Oct 1, 2019

Should rebase on master to get the whitespace fix. Also takewhile and dropwhile should be added to the Iterators submodule exports along with the other functions there. They need docstrings with a compat annotation and should be added to the manual with the other iterators.

base/iterators.jl Outdated Show resolved Hide resolved
Relax unionall annotations

Co-Authored-By: Jeff Bezanson <[email protected]>
base/iterators.jl Outdated Show resolved Hide resolved
@o314
Copy link
Contributor Author

o314 commented Oct 1, 2019

Thanks for the feedback and openness !
As stated, i have some work ahead for now, i will fix remaining parts asap,
tomorrow the better.

- [x] fix order of iterate? / ctor args / struct fields
- [x] review (leaning of) type annotation
- [x] replace `x->x<k` by `<(k)` @ test
- [x] replace `x->some_const` by `_->some_const` @ test
- [x] remove excedentary collect @ test
- [x] fix order of ctor args / struct fields
- [y] review (leaning of) type annotation 
    - [x] redo explicitation @ IteratorSize(::Type{DropWhile}).  
    otherwise MethodError: no method matching length was raised
- [x] replace `x->x<k` by `<(k)` @ test
- [x] replace `x->some_const` by `_->some_const` @ test
- [x] remove excedentary collect @ test
- [/] remove pipeline @ test
Choice has been made to reimplement filter with a recoded while pred loop.
State partition is determined only by the emptiness of iterator

One could use Iterators.filter with Iterators.Stateful but filter erases last elem state and stateful did not revert propertly due to nextvalstate/taken machinery. Stateful needs some cleanup.
base/iterators.jl Outdated Show resolved Hide resolved
@o314
Copy link
Contributor Author

o314 commented Oct 2, 2019

Possible Enhancement

  • optimize nested TakeWhile(TakeWhile()), DropWhile(DropWhile())
    DropWhile(TakeWhile()), TakeWhile(DropWhile())

@JeffBezanson
Copy link
Sponsor Member

Code looks good now, thanks. Just needs news and docs.

- [x] set docs of takewhile, dropwhile
    - [x] set compat
- [x] fix takewhile, dropwhile test comment
@o314
Copy link
Contributor Author

o314 commented Oct 6, 2019

news update

@ new library functions

@o314
Copy link
Contributor Author

o314 commented Oct 6, 2019

compat update

compat has been set to Julia 1.4

base/iterators.jl Outdated Show resolved Hide resolved
base/iterators.jl Outdated Show resolved Hide resolved
@ararslan
Copy link
Member

ararslan commented Oct 7, 2019

Julia 1.3 is feature frozen, so the compatibility notice here should be set to 1.4. As for a NEWS item, I'd just say

* `takewhile` and `dropwhile` have been added to the Iterators submodule ([#33437]).

@o314
Copy link
Contributor Author

o314 commented Oct 7, 2019

I've updated the news and update sections according to your remarks.

@ararslan ararslan removed needs compat annotation Add !!! compat "Julia x.y" to the docstring needs docs Documentation for this change is required labels Oct 7, 2019
@ararslan
Copy link
Member

ararslan commented Oct 7, 2019

Looks like the NEWS file hasn't been updated.

@ararslan ararslan removed the needs news A NEWS entry is required for this change label Oct 8, 2019
NEWS.md Outdated Show resolved Hide resolved
Co-Authored-By: Alex Arslan <[email protected]>
@ararslan ararslan dismissed their stale review October 8, 2019 00:47

Requested changes implemented

@o314 o314 requested a review from JeffBezanson October 8, 2019 02:20
@o314
Copy link
Contributor Author

o314 commented Oct 9, 2019

A filter(bitvector,itr) has been evocated (think compress @ python).

A oneliner can do the thing.
(May be two oneliners)

# zip variant. clone of python compress
@inline Base.Iterators.filter(markers::Union{Vector{Bool},BitVector}, s) =
    (i for (m,i) in zip(markers,s) if m)

# enumerate variant. julia thing
@inline Base.Iterators.filter(markers::Union{Vector{Bool},BitVector}, s) =
    (i for (n,i) in enumerate(s) if markers[n])

Benches

Bench initialization

using Random
bs = bitrand(10_000);
is = rand(1:100,10_000)

Bench of filter (zip variant)

julia> @benchmark Iterators.filter(bs, is)
BenchmarkTools.Trial:
  memory estimate:  80 bytes
  allocs estimate:  4
  --------------
  minimum time:     35.092 ns (0.00% GC)
  median time:      39.066 ns (0.00% GC)
  mean time:        54.293 ns (23.81% GC)
  maximum time:     41.544 μs (99.83% GC)
  --------------
  samples:          10000
  evals/sample:     1000

Bench of filter (enumerate variant)

julia> @benchmark Iterators.filter(bs, is)
BenchmarkTools.Trial:
  memory estimate:  80 bytes
  allocs estimate:  4
  --------------
  minimum time:     31.782 ns (0.00% GC)
  median time:      35.423 ns (0.00% GC)
  mean time:        50.731 ns (25.69% GC)
  maximum time:     42.052 μs (99.82% GC)
  --------------
  samples:          10000
  evals/sample:     1000

The enumerate variant seems 10-15% faster but does not checkbounds!
I don't know if we should have something in Base for this.

@JeffBezanson JeffBezanson merged commit 5f013d8 into JuliaLang:master Oct 10, 2019
StefanKarpinski pushed a commit that referenced this pull request Feb 21, 2020
#33437 has added the two functions, but it didn't appear in the documentation.
birm pushed a commit to birm/julia that referenced this pull request Feb 22, 2020
JuliaLang#33437 has added the two functions, but it didn't appear in the documentation.
KristofferC pushed a commit that referenced this pull request Apr 11, 2020
#33437 has added the two functions, but it didn't appear in the documentation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants