The data structure for a token that makes most intuitive sense to me is a tagged union.
So I defined an „const iota“-style enum. Stuck it into a struct that has the appropriate fields to cover all the cases and it was fine.
Having some syntax sugar for tagged unions would be nice. Having exhaustiveness checks if you switch over them, could be useful in some cases.
But that’s not where my mental energy went at all.
Reading the bytes (runes) efficiently and correctly into the data structure however is the part that needs focus. Once the data is in shape, I‘m not „worried“ at all anymore. Sure a bit of extra support is nice, but also kind of superficial.
Also going further, dispatching on them is again never the tricky part. It’s handling the cases correctly and efficiently that has my focus.
In Clojure, a common thing is to write multimethods that dispatch on (namespaced) keywords. Similar in spirit and structure, but each method might now reside in a different namespace or not even be written by you. But I have never worried about exhaustive matching or similar. What’s in the method bodies is the important part.
The data structure for a token that makes most intuitive sense to me is a tagged union.
So I defined an „const iota“-style enum. Stuck it into a struct that has the appropriate fields to cover all the cases and it was fine.
Having some syntax sugar for tagged unions would be nice. Having exhaustiveness checks if you switch over them, could be useful in some cases.
But that’s not where my mental energy went at all.
Reading the bytes (runes) efficiently and correctly into the data structure however is the part that needs focus. Once the data is in shape, I‘m not „worried“ at all anymore. Sure a bit of extra support is nice, but also kind of superficial.
Also going further, dispatching on them is again never the tricky part. It’s handling the cases correctly and efficiently that has my focus.
In Clojure, a common thing is to write multimethods that dispatch on (namespaced) keywords. Similar in spirit and structure, but each method might now reside in a different namespace or not even be written by you. But I have never worried about exhaustive matching or similar. What’s in the method bodies is the important part.