Pipelines
Whenever you have several stages of operations performed on an input, you can construct a pipeline. Goroutines and channels can be used to construct high-throughput processing pipelines with different structures.
Simple pipeline without fan-out/fan-in
A simple pipeline can be constructed by connecting each stage running in its own goroutine using channels. The structure of the pipeline looks like Figure 10.1.
Figure 10.1: Simple asynchronous pipeline
How to do it...
This pipeline uses a separate error channel to report processing errors. We use a custom error type to capture diagnostic information:
type PipelineError struct {
// The stage in which error happened
Stage int
// The payload
Payload any
// The actual error
Err error
} Every stage...
