-
Notifications
You must be signed in to change notification settings - Fork 78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fold / reduce missing #33
Comments
That's an interesting question. |
Hmm, not sure. I initially thought as an end pipe, but that makes a messier API as you need to pass the operation and the destination. Maybe a better approach would be to pass the intermediate results on and store them in some variable assigning end pipe (named
|
I see. Don't you find it natural that the accumulated value and the operation are passed together? auto const input = std::vector<std::string>{"1", "2", "3"};
int result = 0;
input >>= pipes::transform(std::stoi)
>>= pipes::reduce(result, std::plus); |
TL;DR that would be fine In my initial sketch (not given) it looked messier becaue I had used an in situ lambda, but looking at it now in a cleaner version the end pipe does seem more appealing. I'm a little more partial to reduce than to fold, but I am ok with either choice as long as it's not accumulate which has too much association with summation IMHO. I don't know whether there is any use for it, but the separated version would have the additional flexibility to pass the stream on
|
I suggest to return by value : auto const input = std::vector<std::string>{"1", "2", "3"};
auto const result = (
input
>>= pipes::transform(std::stoi)
>>= pipes::reduce(std::plus)
);
Also, you could construct a vector of results directly : auto const input = std::vector<std::string>{"1", "2", "3"};
auto const result = (
input
>>= pipes::transform(std::stoi)
>>= pipes::to_vector
); Here, you "reduce" the pipe to a vector. |
Common reduction operators should probably have a name like in python : |
Returning by value is the most appealing syntax, but each time I've thought about implementing it, I'm facing the issue that a pipe can have multiple outputs. For example, if the pipeline has demux and only one branch ends with reduce, what should the overall expression return? |
That's a good point. The first thing I can think of is to return a std::tie of results, so that structured bindings and std::ignore can be used. The use of auto const [B, C, D, E]=(
A
>>= pipes::transform(f)
>>= pipes::filter(p)
>>= pipes::unzip(
pipes::to_vector,
pipes::fork(
pipes::to_vector,
pipes::filter(q) >>= pipes::to_vector,
pipes::filter(r) >>= pipes::to_vector
)
)
); But there are still pipes with no return value such as auto const [B, C, std::ignore, E]=(
A
>>= pipes::transform(f)
>>= pipes::filter(p)
>>= pipes::unzip(
pipes::to_vector,
pipes::fork(
pipes::to_vector,
pipes::filter(q) >>= pipes::for_each(print),
pipes::filter(r) >>= pipes::to_vector
)
)
); Another way would be to return a auto const [B, C, E]=(
A
>>= pipes::transform(f)
>>= pipes::filter(p)
>>= pipes::unzip(
pipes::to_vector,
pipes::fork(
pipes::to_vector,
pipes::filter(q) >>= pipes::for_each(print),
pipes::filter(r) >>= pipes::to_vector
)
)
); |
This looks like an interesting solution. pipes::reduce would then return the reduced value, potentially in a tuple too? And what if there is only one way out, in a linear pipeline, should we return a value and not a tuple? |
Regarding From the user point of view, I think that single pipes are expected to return a value, not a tuple. Is it too weird to write |
Ya I was just thinking of a mechanism for pipe expressions to return a value (in #58). Fold/Reduce and all its derivatives seems like an obvious use case for it. Until the semantics for non-SISO pipes is decided we could always implement returning a value for SISO pipes and leave fork and similar pipes to return void. As for the semantics for fork() I think returning a tuple by value with some special void type to represent terminal pipes that don't have a return value (for_each/push_back). I don't think its worth flattening nested tuples, realistically nested forks are probably going to be relatively rare and therefore making the semantics more complicated to accommodate them doesn't make sense. Although if there are some common use cases it might be worth putting more thought. Returning a tie seems like it may lead to dangling reference issues. |
After thinking about it, I think that the single output case is by far the most common, and should return a simple value, not a tuple. The behaviour for fork could probably be decided later. |
Actually, the type of the |
As you can see, I have made a first implementation of A common forked pipeline with returning tails and non returning tails is a tee, with one returning path and one debugging or logging path. auto const [dummy,y]=(
x
>>= pipes::transform(f)
>>= pipes::fork(
pipes::to_out_stream(debug_stream),
pipes::transform(g) >>= pipes::to_<std::vector>
)
) In this example, it might seem better to return a single value to avoid the dummy. However, always mirroring the pipeline structure in the return type makes it easier to replace parts of the pipeline, even dynamically. auto get_debug_pipeline=[&debug_enabled](){
// dynamically choose this pipeline
if(!debug_enabled) return pipes::dev_null;
// or this one
return pipes::transform(get_debug_info) >>= pipes::to_<std::vector<debug_info_t>>;
// you can replace previous line with this one without breaking client code
// return pipes::transform(get_debug_info) >>= pipes::to_out_stream(debug_stream);
};
auto cont [debug,y]=(
x
>>= pipes::transform(f)
>>= pipes::fork(get_debug_pipeline(),
>>= pipes::transform(g)
>>= pipes::to_<std::vector<int>>
)
)
handle_debug_info(debug); //overload this function to handle different types of debug info To conclude, I believe that it is the correct choice to return something even for non-returning pipelines (returning a custom type instead of std::ignore would probably be better though). |
About nested tuples, it is actually acceptable to return them even without nested structured bindings, because one can do the following. auto const [A,BC]=(
x
>>= pipes::transform(f)
>>=pipes::fork(
pipes::to_<std::vector<int>>,
pipes::fork(
pipes::transform(g) >>= pipes::to_<std::vector<int>>,
pipes::transform(h) >>= pipes::to_<std::vector<int>>,
)
)
);
auto const [B,C]=BC; So, the API of my current implementation seems satisfactory to me. |
I'm curious. Have you thought about if it is possible to write
rather than
and have the type system figure out what the element type should be? |
Of course, it would be more convenient to omit the type of the element, which is redundant. However, it would be more complicated to implement because the pipe object has to be constructed before any input goes through it. Besides, So, Short answer: yes, it can be done. |
Maybe I missed some way to combine things, but to me it looks as if something like fold/reduce is missing.
The text was updated successfully, but these errors were encountered: