Skip to content

About

Temporal-style durable workflows without running Temporal. Define workflows as plain Go structs; every state transition persists to Postgres, MySQL or SQLite. Engine, REST API and web console are a library you import — one static binary, no cluster to operate.

Topics

Resources

Contributing

Stars

14 stars

Watchers

1 watching

Forks

Repository files navigation

GopherFlow

CI Lint codecov Go Reference Go Version Docker Image License

Temporal-style durable workflows without running Temporal.

GopherFlow A workflow advancing through its states, with the executed path highlighted on the generated diagram

The diagram is generated from the workflow definition. Blue marks the path this run actually took.

Write workflows as plain Go structs. Every state transition is persisted to a database you already run — Postgres, MySQL or SQLite — so workflows survive restarts, crashes and deploys, and resume from where they stopped. The engine, the REST API and the web console are a library you import into your own binary: no control-plane cluster, no broker, no sidecar, no separate worker fleet to operate.

go get github.com/RealZimboGuy/[email protected]

Why GopherFlow

  • One static binary, one database. gopherflow.Setup(registry) in your main() gives you the engine, the API and the console. Schema migrations run themselves on start. Pure Go, no cgo — the container runs under the default seccomp profile and cross-compiles without a C toolchain.
  • No determinism rules to learn. States are ordinary Go methods that return the next state. Keep them idempotent and the engine handles persistence, retry and backoff — there is no replay model and no workflow-versioning trap.
  • Operable on day one. Dashboard, search, per-workflow action history, live executor list, and a flow diagram generated from the definition with the executed path overlaid.
  • Scale by starting another copy. Executors register in the database, heartbeat, and repair each other's stuck workflows. Adding capacity means running your binary again.

How it compares

GopherFlow Temporal (self-hosted) Dagu Hand-rolled cron + DB
What you operate your binary + a DB frontend / history / matching / worker services, a DB, often Elasticsearch, plus your own workers one binary + files on disk your binary + a DB
How workflows are defined Go structs and methods Go / Java / TS / Python SDK, replay-deterministic code YAML DAGs of commands however you write them
Durable state, retry, resume built in built in per-step retry and run history you build it
Web console built in built in (separate UI service) built in you build it
Parent / child, parallel fan-out yes yes, plus signals, queries, timers, sagas DAG deps and nested DAGs you build it
What you have to learn one Go interface determinism, versioning, task queues the YAML schema nothing, until it grows
Scale ceiling ~19k workflows/min on one laptop + SQLite (measured) very high, multi-cluster single-node scheduler whatever you build

Choose Temporal if you need signals and queries, workers in several languages, or scale that outgrows a single database. Choose Dagu if your steps are shell commands on a schedule rather than Go code. Choose GopherFlow if you want durable, retryable, observable business workflows written in Go — without operating another distributed system to get them.

Highlights

  • Define workflows in Go using a state-machine approach
  • each function is idempotent and can be retried
  • Persistent storage (Postgres, SQLite, Mysql supported) with action history
  • Concurrent execution with executor registration, heartbeats, and stuck-workflow repair
  • Web console (dashboard, search, definitions with diagrams, executors, details)
  • Mermaid-like flow visualization generated from workflow definitions
  • Container-friendly, single-binary deployment
  • Parent / Child Workflows
    • GopherFlow supports parent workflows spawning child workflows, allowing for parallel execution and coordination.
    • Spawn Children: A parent workflow can create multiple child workflow requests.
    • Wait & Wake: The parent can wait for children to complete. Children can explicitly wake their parent when they reach a certain state or finish.
    • Parallel Execution: Child workflows run independently and in parallel.

Quick start

Prerequisites:

  • Go 1.26+ (or Docker if you prefer containers)

Demo Application

This starts the demo application with a SQLite database, there are two workflows

  • DemoWorkflow - does some ficticious steps and adds some variables

  • GetIpWorkflow - gets the current public IP address from ifconfig.io and puts it into a state variable

      docker run -p 8080:8080 \
      -e GFLOW_DATABASE_TYPE=SQLLITE \
      -e GFLOW_DATABASE_SQLLITE_FILE_NAME=/data/gflow.db \
      -v gflow-data:/data \
      juliangpurse/gopherflow:1.9.1
    

Access the web console at http://localhost:8080/

Username : admin
Password : admin

The database lives in the named volume gflow-data, which survives docker rm and can be removed with docker volume rm gflow-data.

To keep the database file in the current directory instead, bind mount it and run as yourself. The image runs as a non-root user, so a bind mount owned by your account is not writable by the container unless you say who to run as:

    docker run -p 8080:8080 \
    -e GFLOW_DATABASE_TYPE=SQLLITE \
    -e GFLOW_DATABASE_SQLLITE_FILE_NAME=/data/gflow.db \
    -v "$(pwd):/data" \
    --user $(id -u):$(id -g) \
    juliangpurse/gopherflow:1.9.1

Postgres quickstart (docker compose)

SQLite serialises writes, so Postgres takes you further on concurrent executions. The docker-compose.yml at the repo root brings up Postgres and the demo image wired together:

    docker compose up

That gives you a working console at http://localhost:8080/ (admin / admin) with a real database behind it. Migrations run automatically on start — there is no init step. The compose file spells out the GFLOW_DATABASE_* environment variables rather than hiding them in an .env, so it doubles as documentation of how to configure a database connection. Postgres data lives in the named volume gopherflow-postgres.

Web Console

Dashboard: workflow counts by type and status A single workflow with its action history A workflow definition rendered as a flow diagram

REST API

GopherFlow provides a REST API for programmatic interaction with workflows. A Postman collection is available in the postman directory to help you get started:

  • Collection File: in the postman/ directory
  • API Key Authentication: All endpoints use an X-API-Key header for authentication, check users tab in the web ui for the api key

Available Endpoints:

  1. Get Workflow Definitions - GET /api/definitions
  2. Create Workflow - POST /api/workflows
  3. Get Workflow Details - GET /api/workflows/{id}
  4. Get Workflow by External ID - GET /api/workflowByExternalId/{externalId}
  5. Search Workflows - POST /api/workflows/search
  6. Create and Wait - POST /api/createAndWait - Create a workflow and wait for it to reach specific states
  7. Update State and Wait - POST /api/workflows/{externalId}/stateAndWait - Update a workflow's state and wait for it to reach specific states

To use the Postman collection:

  1. Import the collection into Postman
  2. Configure your environment variables (if needed)
  3. Use the pre-configured requests to interact with your GopherFlow instance

Health check

GET /healthz needs no session or API key. It returns 200 {"status":"ok","database":"ok"} when the server is up and the database answers a ping, and 503 otherwise.

Use it as a readiness probe, not a liveness probe. It reports on the database, and a liveness probe that fails during a database outage makes Kubernetes restart every GopherFlow pod, which cannot fix the database.

readinessProbe:
  httpGet:
    path: /healthz
    port: 8080

GopherFlow registers its routes on Go's global http.DefaultServeMux. If your own binary also registers GET /healthz there, it will panic at startup with a duplicate-pattern error; rename yours or serve it from your own ServeMux.

Performance

Measured on an 8-core Apple M3 (16 GB, macOS 26.6, Go 1.26) against SQLite in WAL mode, pushing 20,000 three-state workflows through a single process with 32 executors and a batch size of 200:

Workflows completed 20,000
State transitions 140,000
Wall time 63.3 s
Throughput ~19,000 workflows/min (316/sec)
~2,200 state transitions/sec

Every state transition is a state function call plus a workflow row update plus an action-history insert, so transitions/sec is the honest unit of work — workflows/min just happens to be the number people ask for. Smaller batches run faster per workflow (~37,000 workflows/min at 5,000 workflows) because the tables are still small; the figure above is the sustained one.

Reproduce it yourself:

GFLOW_BENCH=1 GFLOW_BENCH_WORKFLOWS=20000 go test ./test/benchmark/ -run TestThroughput -v -timeout 20m

The harness is test/benchmark/throughput_test.go. It is skipped unless GFLOW_BENCH is set, and it measures execution only — the seed inserts happen before the clock starts.

Tune with GFLOW_ENGINE_EXECUTOR_SIZE, GFLOW_ENGINE_BATCH_SIZE and GFLOW_ENGINE_CHECK_DB_INTERVAL. Postgres will take you further than SQLite on concurrent writes; SQLite is the easy default, not the ceiling.

A caveat worth stating plainly: if you need far more throughput than this, you probably do not want a workflow engine at all — you want hand-written code and a queue. GopherFlow is for workflows whose rate is set by the APIs and humans they wait on, where what you actually want is persistence, retry and a console to see what happened.

Building your own Workflow and running it

refer to the example application: https://github.com/RealZimboGuy/gopherflow-examples

Specific details

go get github.com/RealZimboGuy/[email protected]

a struct that extends the base

type GetIpWorkflow struct {
    core.BaseWorkflow
}

the workflow interface must be fully implemented

type Workflow interface {
    StateTransitions() map[string][]string // map of state name -> list of next state names
    InitialState() string // where to start
    Description() string
    Setup(wf *domain.Workflow)
    GetWorkflowData() *domain.Workflow
    GetStateVariables() map[string]string
    GetAllStates() []models.WorkflowState 
    GetRetryConfig() models.RetryConfig
}

Here is the example for the GetIpWorkflow

This lives in your own module — say workflows/getip_workflow.go. Workflows are ordinary Go types in your code; GopherFlow never needs them to live anywhere particular.

package workflows

import (
	"context"
	"io"
	"log/slog"
	"net/http"
	"time"

	"github.com/RealZimboGuy/gopherflow/pkg/gopherflow/core"
	"github.com/RealZimboGuy/gopherflow/pkg/gopherflow/domain"
	"github.com/RealZimboGuy/gopherflow/pkg/gopherflow/models"
)

// State names are yours to choose. They only need to agree between
// StateTransitions and GetAllStates.
var (
	StateStart     = "Start"
	StateGetIpData = "StateGetIpData"
	StateFinish    = "Finish"
)

const VAR_IP = "ip"

type GetIpWorkflow struct {
	core.BaseWorkflow
}

func (m *GetIpWorkflow) Setup(wf *domain.Workflow) {
	m.BaseWorkflow.Setup(wf)
}

func (m *GetIpWorkflow) GetWorkflowData() *domain.Workflow {
	return m.WorkflowState
}

func (m *GetIpWorkflow) GetStateVariables() map[string]string {
	return m.StateVariables
}

func (m *GetIpWorkflow) InitialState() string {
	return StateStart
}

func (m *GetIpWorkflow) Description() string {
	return "Fetches the public IP address and stores it in a state variable"
}

func (m *GetIpWorkflow) GetRetryConfig() models.RetryConfig {
	return models.RetryConfig{
		MaxRetryCount:    10,
		RetryIntervalMin: time.Second * 10,
		RetryIntervalMax: time.Minute * 60,
	}
}

func (m *GetIpWorkflow) StateTransitions() map[string][]string {
	return map[string][]string{
		StateStart:     {StateGetIpData},
		StateGetIpData: {StateFinish},
	}
}

func (m *GetIpWorkflow) GetAllStates() []models.WorkflowState {
	return []models.WorkflowState{
		{Name: StateStart, StateType: models.StateStart},
		{Name: StateGetIpData, StateType: models.StateNormal},
		{Name: StateFinish, StateType: models.StateEnd},
	}
}

// Each state is a method named after the state, returning the next state.
func (m *GetIpWorkflow) Start(ctx context.Context) (*models.NextState, error) {
	// Use the InfoContext form: the engine puts worker and workflow ids into
	// the context and the logger writes them out with each line.
	slog.InfoContext(ctx, "Starting workflow")

	return &models.NextState{
		Name:      StateGetIpData,
		ActionLog: "using ifconfig.io to return the public IP address",
	}, nil
}

func (m *GetIpWorkflow) StateGetIpData(ctx context.Context) (*models.NextState, error) {
	resp, err := http.Get("http://ifconfig.io")
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()

	ipBytes, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}
	m.StateVariables[VAR_IP] = string(ipBytes)

	return &models.NextState{
		Name: StateFinish,
	}, nil
}

Main function

Register each workflow type by name and start the engine. Replace example.com/myapp with your own module path.

package main

import (
	"context"
	"log/slog"

	"github.com/RealZimboGuy/gopherflow/pkg/gopherflow"
	"github.com/RealZimboGuy/gopherflow/pkg/gopherflow/core"

	"example.com/myapp/workflows"
)

func main() {
	ctx := context.Background()

	// Use your own logger setup, or this default one built on slog.
	gopherflow.SetupLogger(slog.LevelInfo)

	// Register every workflow type by name.
	workflowRegistry := map[string]func() core.Workflow{
		"GetIpWorkflow": func() core.Workflow {
			// Inject your own dependencies here.
			return &workflows.GetIpWorkflow{}
		},
	}

	app := gopherflow.Setup(workflowRegistry)

	if err := app.Run(ctx); err != nil {
		slog.Error("Engine exited with error", "error", err)
	}
}

Example: Spawning Children

In your parent workflow state transition:

func (w *MyParentWorkflow) SpawnChildren(ctx context.Context) (*models.NextState, error) {
    // Create child workflow requests.
    // Signature: CreateChildWorkflowRequest(workflowType, businessKey, stateVars).
    // The child's initial state is taken from the child workflow's own InitialState();
    // the engine assigns an externalId automatically.
    childRequests := []models.ChildWorkflowRequest{
        gopherflow.CreateChildWorkflowRequest(
            "MyChildWorkflow",
            fmt.Sprintf("child-%d", w.WorkflowState.ID),
            map[string]string{"input": "value"},
        ),
    }

    return &models.NextState{
        Name:           "WaitForChildren",
        ChildWorkflows: childRequests,
    }, nil
}

Example: Waiting for Children

func (w *MyParentWorkflow) WaitForChildren(ctx context.Context) (*models.NextState, error) {
    children, err := w.GetChildWorkflows(ctx)
    if err != nil {
        return nil, err
    }

    allComplete := true
    for _, child := range children {
        if child.Status != models.WorkflowStatusFinished {
            allComplete = false
            break
        }
    }

    if !allComplete {
        // Wait and check again later
        return &models.NextState{
            Name:                "WaitForChildren",
            NextExecutionOffset: "1 minute",
        }, nil
    }

    return &models.NextState{Name: "Finish"}, nil
}

About

Temporal-style durable workflows without running Temporal. Define workflows as plain Go structs; every state transition persists to Postgres, MySQL or SQLite. Engine, REST API and web console are a library you import — one static binary, no cluster to operate.

Topics

Resources

Contributing

Stars

14 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages