gregwebs 2 days ago

Thanks for sharing what is working for you in production. I made a somewhat similar library that also does batching (docs are sparse, but I am updating docs on my libraries this week) [1].

I would call this parallelism rather than concurrency.

The main issue I have with this library's implementation is how errors are handled. Errors are retrieved rather than assigned- but assignment is preferable because it gets verified by tools. In my library I used a channel for errors- that gives ultimate flexiblitiy- it can be converted to wait and collect them to a slice or to perform a cancellation on first error.

[1] https://github.com/gregwebs/go-parallel

1
destel 2 days ago

Thank you for the feedback. My design decision is of course a tradeoff. When multiple channels are exposed to the users (not encapsulated inside the lib), this forces them to use "select". And this is very error prone in my experience

gregwebs 2 days ago

I never needed to use select on an error channel for my use cases because at the point I operate on the error channel I want to block for completion. And I provide helpers for the desired behavior for the channel so I don't even directly receive from it. I see that some of Rill is designed to operate on continuous streams, and in that light the design decision makes sense. For my use cases though the stream always had an end.