-.- --. .-. --..

Go Reflect for structural checking

I recently used Go’s Reflect package and I’m pretty chuffed about it; hence this blog post.

My hobby project this year has been a to build simple log processor1, as a chance to get used to Go and its paradigms. It takes incoming (over HTTP) blobs of new-line-delimited HTTP log lines, parses them, converts and dumps the metrics into Prometheus. The details of the processing itself is not that important for this post, and it’s a very run-of-the-mill setup. The main bits are two structs that group the necessary metrics: a base struct with metric vectors, and another a label-curried2 struct. The labels are created out of the parsed log lines. The overall structure (ahem) looks something like this:

type metricsVec struct {
	reqCount *prometheus.CounterVec
	resSize  *prometheus.CounterVec
}

func (mv *metricsVec) With(labels promtheus.Labels) *metrics {
	newMetric := &metrics{
		reqCount: mv.With(labels),
		resSize:  mv.With(labels),
	}
}

type metrics struct {
	reqCount prometheus.Counter
	resSize  prometheus.Counter
}

// somewhere in the code
vec := metricsVec{}
labels := prometheus.Labels{
	"method": method,
	"status": statusCode,
}

m := vec.With(labels)
m.reqCount.Add(1)
m.resSize.Observe(processedLog.size)

The (*metricsVec).With() method creates the label-mapped metrics struct that’s then used for logging the actual metrics later on in the process. Often I forget to add a metric within the metrics struct which then causes runtime exceptions. I always have to keep these two structs in sync. To do this, one option is to auto-generate the metrics struct via some “builder” style process, but I ended up using the Reflect functionality instead. I also used this opportunity to sneak in some generics as well. Below are a couple of functions I created to help with this. The first, a method that returns a sequence of the fields within the struct, using the Reflect APIs:

func fieldsForType[M metricsVec | metrics]() iter.Seq[string] {
	return func(yield func(string) bool) {
		t := reflect.TypeFor[M]()
		fields := reflect.VisibleFields(t)

		for _, field := range fields {
			if !yield(field.Name) {
				return
			}
		}
	}
}

for key := range fieldsForType[metrics]() {
	fmt.Println(key)
}

// Output:
//   reqCount
//   resSize

Next is a function that, on the program init, validates the fields within the two structs, and exits immediately if they don’t match, as a early signal during development:

func validateMetricsStructs() {
	mKeys := slices.Sorted(fieldsForType[metrics]())
	mVecKeys := slices.Sorted(fieldsForType[metricsVec]())

	if !slices.Equal(mKeys, mVecKeys) {
		log.Fatalf("metrics and metricsVec have diverged")
	}
}

// And I call this function in the startup. Good enough for the use-case
// I have.
func init() {
	validateMetricsStructs()
}

Simple. I didn’t want to go overboard while using Reflect, and wanted to keep the meta-programming to a minimum to nonexistent. It’s not a lot of cost to do this.

  1. At this point I’m not sure if I should proclaim this or not, but for this project I didn’t want to use any LLM or AI help. This is for me to learn Go, and not to just “ship” the project. When I say “simple”, I mean it, but I like to put in some constraints for myself; for this project I decided to process logs as fast as possible while using as little memory as possible. ↩

  2. I’m not using the right terminology here, but hopefully the example shoes the use-case. The reason I have these two metrics structs, which may not be clear based on the examples in this post, is to pool the objects based on label set as as a key. ↩

← Rewrites outside location blocks in Nginx are bad!