Generating Envoy Config with Cue - Transforming Inputs
More in this series:
- Schema Definition
- Transforming Inputs
- Testing
- Language Refinements (coming soon)
- Docker Packaging (coming soon)
This post is part two in a series exploring config generation for Envoy using Cue.
In part one we defined a schema for our custom reverse proxy input format. In this post we'll use Cue to transform that input format into a valid envoy.yaml file.
The Function Pattern
We'll encapsulate our transformation logic using the Function Pattern. Before we dive into the transformation in detail, let's see a simple example of the function pattern in action.
We start by defining a schema for the function inputs:
package example
#Input: {
firstName: string
lastName: string
}Here we're defining our input to be two string fields: firstName and lastName. Next we define the transform itself:
#Transform: {
_input: #Input
result: "Hello \(_input.firstName) \(_input.lastName)"
}We can think of this #Transform type as a function that takes as input a struct of type #Input on key _input. The underscore at the start of the _input field means that it won't be included in any output. The result field of the transform uses string interpolation to construct a message using the fields from the #Input.
To use the transform, we unify the #Transform schema with an instance of the #Input schema:
#Transform & {_input: {
firstName: "Rob"
lastName: "Harrop"
}}Running this through cue export gives us this output:
❯ cue export
{
"result": "Hello Rob Harrop"
}In general, we probably don't want to see the result field in the output, rather we want to see just its contents. We can achieve this with the -e Cue flag:
❯ cue export -e result
"Hello Rob Harrop"Basic Project Structure
In part one we defined the input schema in schema.cue and we had a test input in input.cue. We'll now add transform.cue with all our transformation logic:
❯ tree
.
├── input.cue
├── schema.cue
└── transform.cueAs a reminder, our input schema is:
package envoy
input: #InputSchema
#InputSchema: {
hosts: [#VHostName]: #VHost
targets: #Targets
}
// Virtual Hosts
#VHostName: string
#VHost: {
routes: [#Route, ...#Route]
}
// Targets
#Targets: [#Target, ...#Target]
#Target: {name: #TargetName, port: >0 & <=65_535}
#TargetName: string
// Routes
#Route: #PathRoute | #PrefixRoute | #RegexRoute
#PathRoute: {path: #Path, target: #ValidTargetName}
#PrefixRoute: {prefix: #Prefix, target: #ValidTargetName}
#RegexRoute: {regex: #Regex, target: #ValidTargetName}
#Prefix: =~"\\^?/[/A-Za-z\\-]*"
#Path: =~"/[/A-Za-z\\-]*"
#Regex: string
#ValidTargetName: or([ for t in input.targets {t.name}])We'll generate the output using cue export --out yaml -e result > envoy.yaml. This will select the result expression from the unifed Cue content and then write it as YAML to envoy.yaml
Top-Level Transform
The envoy.yaml output has a single top-level key static_resources. Under that key we have listeners and clusters:
result: {
static_resources: {
listeners: [#_ListenerTransform & {
_hosts: input.hosts
}]
clusters: [ for t in input.targets {
#TargetTransform & {_target: t}
}]
}
}We generate a single entry in listeners using the content of our input.hosts field. For every entry in input.targets we generate an entry in clusters.
Generating Clusters
Let's start by looking at the #TargetTransform used to generate clusters since it's the simpler of the two top-level transforms.
Using the function pattern, #TargetTransform accepts an input parameter _target. We take the total output of the unification rather than using a dedicated result field:
#TargetTransform: {
_target: #Target
name: _target.name
connect_timeout: "15s"
type: "strict_dns"
load_assignment: {
cluster_name: _target.name
endpoints: [{
lb_endpoints: [{
endpoint: {
address: {
socket_address: {
address: _target.name
port_value: _target.port
}
}
}
}]
}]
}
}Notice that we have three places where the name field for the #Target is used. This duplication reduction combined with how much boilerplate we're able to encapsulate here shows a little of what we can achieve using Cue.
Generating Listeners
The virtual host configuration for our reverse proxy configuration ends up deeply-nested inside an Envoy listener accompanied by quite a lot of boilerplate configuration:
#ListenerTransform: {
_hosts: [#VHostName]: #VHost
#ListenerBoilerplate
filter_chains: [{
filters: [{
typed_config: {
route_config: {
virtual_hosts: [ for n, h in _hosts {#VHostTransform & {_host: h, _name: n}}]
}
}
},
]
},
]
}The #ListenerTransform has an input field called _hosts that we'll link to our input.hosts field. We use this _hosts field alongside the #VHostTransform type to generate virtual_hosts embedded deep in filter_chains.
Listener Boilerplate
Before we look at that transform function, let's take a look #ListenerBoilerplate. This is a type that extracts the all the boilerplate config that doesn't depend on the input data:
#ListenerBoilerplate: {
name: "http"
address: {
socket_address: {
address: "0.0.0.0"
port_value: 80
}
}
filter_chains: [{
filters: [{
name: "envoy.filters.network.http_connection_manager"
typed_config: {
"@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
stat_prefix: "ingress_http"
access_log: [{
name: "envoy.access_loggers.stdout"
typed_config: {
"@type": "type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog"
}
}]
http_filters: [
{name: "envoy.filters.http.router"},
]
route_config: {
name: "local_route"
}
}
},
]
},
]
}The exact details of this config aren't particularly interesting. What is interesting is that the filter_chains definition in #ListenerBoilerplate overlaps with the filter_chains definition in #ListenerTransform. Cue will unify these definitions and, provided they don't clash, we'll see the result of that unification.
Virtual Host Transform
Our Envoy virtual hosts need a name, a list of domains for the host and the routes for that host. By convention, we use the name as the only domain:
#VHostTransform: {
_host: #VHost
_name: #VHostName
name: _name
domains: [_name]
routes: [ for r in _host.routes {#RouteTransform & {_route: r}}]
}Generation of routes follows a pattern we have seen before: a list comprehension delegating to another transform function.
Route Generation
Route generation is where things start to get really interesting. Recall from part one that we have three types of route: path, prefix and regex.
To handle this conditionality, we can check the existence of the path, prefix and regex fields inside each route by comparing against the bottom value _|_:
#RouteTransform: {
_route: #Route
match: {
if _route["prefix"] != _|_ {
prefix: _route.prefix
}
if _route["regex"] != _|_ {
safe_regex: {
google_re2: {}
regex: _route.regex
}
}
if _route["path"] != _|_ {
path: _route.path
}
}
route: {
cluster: _route.target
}
}Since our input schema constrains #Route to have exactly one of path, prefix or regex, we know we'll get only one type of route generated inside the match block.
What's Next?
Testing. We've got a working input format and a transformer to output envoy.yaml, but how can we be sure that the transformer is doing the right thing? We need some tests. In part three, we'll see how to write these tests in a way that slots naturally into our Cue project.