Chapter 7

1. The Go interface{} type conveys no useful information about the underlying type. If we use it for representing an argument to a function or a method, we effectively bypass the compiler's ability to statically check the function/method arguments at compile-time and instead have to manually check whether the input can be safely cast into a supported known type.
2. Instead of running the compute-intensive stages locally, we can migrate them to a remote machine with enough computing resources. The respective local stages can then be replaced with a proxy that transmits the local payload data to the remote machine via a remote procedure call (RPC), waits for the results, and pushes them to the next local stage. The following diagram outlines the proposed solution:

3. Each processor function must satisfy the Processor interface, whose definition is as follows:

type Processor interface {
    Process(context.Context, Payload) (Payload, error)
}

In addition, we also defined the ProcessorFunc type, which acts as an adaptor for converting a function with a compatible signature into a type that implements the Processor interface.

For this particular use case, we can define a function that receives a Processor and a logger (for example, from the logrus package) instance and returns a new Processor that decorates the call to the Process method with additional logic that emits a log entry if an error occurs. The makeErrorLoggingProcessor function shows one of the possible ways of implementing this pattern:

func makeErrorLoggingProcessor(proc Processor, logger *logrus.Logger) Processor {
    return ProcessorFunc(func(ctx context.Context, p Payload) (Payload, error) {
        out, err := proc.Process(ctx, p)
        if err != nil {
            logger.Error(err)
        }
        return out, err
    })
}

4. A synchronous pipeline processes one payload at a time in first-in, first-out (FIFO) order and waits for it to exit the pipeline before processing the next available payload. As a result, if a single payload takes a long time to be processed, it effectively delays processing the payloads that are queued behind it. In an asynchronous pipeline, each stage operates asynchronously and can immediately begin processing the next payload as soon as the current payload has been sent to the next stage.
5. A dead-letter queue is a mechanism for deferring error handling for pipeline payloads to a later time. When the pipeline encounters an error while processing a payload, it appends the payload to the dead-letter queue, along with the error that occurred. The application can then introspect the contents of the dead-letter queue and decide how it wants to handle each error according to its business logic (for example, retry the failed payload, log or ignore the error, and so on).

6. A fixed-size worker pool contains a predetermined number of workers that are created at the same time as the pool and remain active (even when they are idle) until the pool is destroyed. A dynamic pool is configured with lower and upper worker limits and can automatically grow or shrink on demand to accommodate changes in the rate of incoming payloads.
7. To measure the total time that each payload spent in the pipeline, we will modify the pipeline.Payload struct and add a new private field of the time.Time type called processStartedAt. This new field will be used to record the timestamp when the payload entered the pipeline. Next, we will modify the linkSource implementation to populate processStartedAt when it emits a new Payload. Finally, we will update the (currently empty) Consume method of nopSink to calculate the elapsed time via a call to time.Since.