I recently used Go’s Reflect package and I’m pretty chuffed about it; hence this blog post.
My hobby project this year has been a to build simple log processor1, as a chance to get used to Go and its paradigms. It takes incoming (over HTTP) blobs of new-line-delimited HTTP log lines, parses them, converts and dumps the metrics into Prometheus. The details of the processing itself is not that important for this post, and it’s a very run-of-the-mill setup. The main bits are two structs that group the necessary metrics: a base struct with metric vectors, and another a label-curried2 struct. The labels are created out of the parsed log lines. The overall structure (ahem) looks something like this:
type metricsVec struct {
reqCount *prometheus.CounterVec
resSize *prometheus.CounterVec
}
func (mv *metricsVec) With(labels promtheus.Labels) *metrics {
newMetric := &metrics{
reqCount: mv.With(labels),
resSize: mv.With(labels),
}
}
type metrics struct {
reqCount prometheus.Counter
resSize prometheus.Counter
}
// somewhere in the code
vec := metricsVec{}
labels := prometheus.Labels{
"method": method,
"status": statusCode,
}
m := vec.With(labels)
m.reqCount.Add(1)
m.resSize.Observe(processedLog.size)
The (*metricsVec).With() method creates the label-mapped metrics struct that’s then used for logging the actual metrics later on in the process. Often I forget to add a metric within the metrics struct which then causes runtime exceptions. I always have to keep these two structs in sync. To do this, one option is to auto-generate the metrics struct via some “builder” style process, but I ended up using the Reflect functionality instead. I also used this opportunity to sneak in some generics as well. Below are a couple of functions I created to help with this. The first, a method that returns a sequence of the fields within the struct, using the Reflect APIs:
func fieldsForType[M metricsVec | metrics]() iter.Seq[string] {
return func(yield func(string) bool) {
t := reflect.TypeFor[M]()
fields := reflect.VisibleFields(t)
for _, field := range fields {
if !yield(field.Name) {
return
}
}
}
}
for key := range fieldsForType[metrics]() {
fmt.Println(key)
}
// Output:
// reqCount
// resSize
Next is a function that, on the program init, validates the fields within the two structs, and exits immediately if they don’t match, as a early signal during development:
func validateMetricsStructs() {
mKeys := slices.Sorted(fieldsForType[metrics]())
mVecKeys := slices.Sorted(fieldsForType[metricsVec]())
if !slices.Equal(mKeys, mVecKeys) {
log.Fatalf("metrics and metricsVec have diverged")
}
}
// And I call this function in the startup. Good enough for the use-case
// I have.
func init() {
validateMetricsStructs()
}
Simple. I didn’t want to go overboard while using Reflect, and wanted to keep the meta-programming to a minimum to nonexistent. It’s not a lot of cost to do this.
-
At this point I’m not sure if I should proclaim this or not, but for this project I didn’t want to use any LLM or AI help. This is for me to learn Go, and not to just “ship” the project. When I say “simple”, I mean it, but I like to put in some constraints for myself; for this project I decided to process logs as fast as possible while using as little memory as possible. ↩
-
I’m not using the right terminology here, but hopefully the example shoes the use-case. The reason I have these two metrics structs, which may not be clear based on the examples in this post, is to pool the objects based on label set as as a key. ↩