Using a Pipeline Pattern in Golang

While working on a project, I opted to use Golang. I’ve been incorporating Golang into my projects for some time now, yet I haven’t completely moved away from PHP. Golang is an incredibly powerful language, and I’m eager to further hone my skills in its ecosystem.

For a specific requirement of the project, I needed a Pipeline Pattern. This was necessary because I had to pass the data through multiple filters, processing it in a sequential manner. Let me explain:

Let’s imagine that we have a lengthy string. My initial step would be to parse it based on specific criteria. Following this, I aim to make some adjustments to the parsed data. Subsequently, I intend to save this parsed data into the database. As you can see, there are multiple tasks at hand, and each is inherently linked to the other.

Basic Pipelines Flow

In software development, this scenario can be managed using the pipeline design pattern. This pattern is tailor-made to handle sequential modifications to an object. Visualize an assembly line: each station is a piece of “pipe”, and after an object passed through the whole pipeline, it has been transformed. Essentially, pipelines facilitate the sequential passage of a value through a series of callable “pipes”, be they middlewares, filters, or processors. Each piece of pipe has the potential to alter the value before relaying it to the subsequent pipe in the sequence. The pattern is especially useful in scenarios such as request handling, data processing, or transformations, offering a clean, maintainable, and testable approach.

As a solution, I prepared a simple Golang package, which allows you to use the pipeline pattern in your processes and it’s built upon the chain of responsibility (CoR) design pattern. Let’s install it on the project and use it:

go get github.com/izniburak/pipeline-go

After the package installation, we can make a simple demo. Firstly, we need a few new structs, which have Handle method that is implemented from PipeInterface. Because the pipeline package needs more than one pipeline, we’re using the Handle method to run each one:

package main

import (
 "strings"
 "github.com/izniburak/pipeline-go"
)

type UpperCasePipe struct{}

func (u *UpperCasePipe) Handle(value pipeline.PipeValue, next pipeline.PipeNext) pipeline.PipeValue {
  // get value
  text := value.(string)
  capitalized := strings.ToUpper(text)
  return next(capitalized)
}

type TrimSpacePipe struct{}

func (t *TrimSpacePipe) Handle(value pipeline.PipeValue, next pipeline.PipeNext) pipeline.PipeValue {
  // get value
  text := value.(string)
  trimmed := strings.Trim(text, " ")
  return next(trimmed)
}

Then, we can start using the pipeline:

package main

import (
 "fmt"
 "strings"
 "github.com/izniburak/pipeline-go"
)

type UpperCasePipe struct{}

func (u *UpperCasePipe) Handle(value pipeline.PipeValue, next pipeline.PipeNext) pipeline.PipeValue {
  // get value
  text := value.(string)
  capitalized := strings.ToUpper(text)
  return next(capitalized)
}

type TrimSpacePipe struct{}

func (t *TrimSpacePipe) Handle(value pipeline.PipeValue, next pipeline.PipeNext) pipeline.PipeValue {
  // get value
  text := value.(string)
  trimmed := strings.Trim(text, " ")
  return next(trimmed)
}

func main() {
  text := "   buki.dev   "

  pipes := []pipeline.PipeInterface{
    new(UpperCasePipe),
    new(TrimSpacePipe),
  }
  result := pipeline.Send(text).Through(pipes).ThenReturn()

  fmt.Println(result) // BUKI.DEV
}

As you see above, we used two different pipes, which are UpperCasePipe and TrimSpacePipe. Our input is buki.dev with spaces on both sides, and the output is BUKI.DEV.

That’s it. Pipelines are very useful, aren’t they?

You can check the package on GitHub here.

GitHub - izniburak/pipeline-go: Pipeline pattern for your jobs, tasks, etc… (Golang)

This is the first version of the package. I’ll try to keep it up-to-date and continue to improve it. In the upcoming release, I plan to expand the integration capabilities to include not just the PipeInterface and its Handle method, but also multiple other interfaces.

Please feel free to post any bugs you find or suggest any improvements. As I said, I’m trying to improve my skills in Golang and I’m aware that I may make mistakes, even if they are basic points. I would be happy to hear your feedback :)

See you for my next posts about Golang!

If you want contact to me, you can check my X profile or my personal page.

Thanks.

Sources

Pipeline (software) - Wikipedia

Chain-of-responsibility pattern - Wikipedia