Generating Envoy Config with Cue - Testing

For any reasonably complex Cue project, you'll want to write some tests. I'm not aware of any official testing patterns for Cue, but one pattern I've adopted is using Cue to test transformations written in Cue.

The basic idea is to construct a schema defining the test expectation and then to ask Cue to unify this schema with the output of your transformation.

A Basic Example

Imagine we have output like this:

output:
    name: "Rob"
    country: "UK"

To write a test to validate that our transform generates the correct name for the output, we can build a schema like this:

#ExpectCorrectName: {
    name: "Rob"
    ...
}
output: #ExpectCorrectName

We define a schema #ExpectCorrectName setting our expectation for the name value. The ... allows the schema to include any other fields - this test is just about the name field.

Then we say that output should unify with our test schema type.

With the output saved in output.yaml and the tests in test.cue we can run cue vet to check that the output is as expected.

cue vet output.yaml tests.cue && echo VALID
VALID

We can add more tests to tests.cue:

#ExpectCorrectName: {
    name: "Rob"
    ...
}
output: #ExpectCorrectName

#ExpectCorrectCountry: {
    country: "GB"
    ...
}
output: #ExpectCorrectCountry

When we vet now, we get an error:

❯ cue vet output.yaml tests.cue
output.country: conflicting values "GB" and "UK":

A More Complex Example

Consider this subset of the the YAML output from part two:

static_resources:
  clusters:
    - name: user-service
      connect_timeout: 15s
      type: strict_dns
      load_assignment:
        cluster_name: user-service
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: user-service
                      port_value: 8080
    - name: api-service
      connect_timeout: 15s
      type: strict_dns
      load_assignment:
        cluster_name: api-service
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: api-service
                      port_value: 8080
    - name: frontend-users
      connect_timeout: 15s
      type: strict_dns
      load_assignment:
        cluster_name: frontend-users
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: frontend-users
                      port_value: 8080
    - name: monolith
      connect_timeout: 15s
      type: strict_dns
      load_assignment:
        cluster_name: monolith
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: monolith
                      port_value: 8080

There are so many things we could test here, let's just consider three that I think give a good overview of what is possible.

Testing we always use strict_dns

To validate that all clusters have type set to strict_dns we can define a basic schema requiring this:

#AlwaysStrictDns: {
    clusters: [
        ...{
            type: "strict_dns"
            ...
        }
    ]
    ...
}

static_resources: #AlwaysStrictDns

This schema says that clusters is a list of objects, where each object must have a type field with the value strict_dns.

Testing we always expose the correct address

We take the same idea of loosely defining the structure of part of the output and apply it to something nested a bit more deeply in the output.

#AlwaysCorrectAddress: {
    clusters: [...{
        name: string
        let _n = name
        load_assignment: {
            cluster_name: _n
            endpoints: [
                {
                    lb_endpoints: [
                        {
                            endpoint: address: socket_address: {
                                address: _n
                                port_value: 8080
                            }
                        }
                    ]
                }
            ]
        }
        ...
    }]
    ...
}
static_resources: #AlwaysCorrectAddress

In this case, we're checking that the deeply nested socket_address objects have the correct address and port. Of particular interest here, we use a let binding to capture the name and then use that to validate that the socket address matches the cluster name.

Testing all cluster entries are present

To check that all expected clusters are present we need a little more trickery.

import "list"

#AllClustersArePresent: {
    let _names = ["user-service", "frontend-users", "api-service", "monolith"]
    
    clusters: [for n in _names {name: or(_names), ...}]

    _clusterNames: list.SortStrings([for c in clusters {c.name}])
    _clusterNames: list.SortStrings(_names)
    
    ...
}
static_resources: #AllClustersArePresent

First we make a generic statement about clusters: it's a list of objects each with a name field whose value is one of the four expected cluster names. The or creates a disjunction type from an input list.

This constraint validates the number of clusters and their names, but it doesn't validate that we have exactly one cluster for each of the names.

To do that we add in a dummy definition _clusterNames and specify it twice. The first specification says it's equal to the sorted list of cluster names as specified in the output we are verifying. The second specification says that it's equal to sorted list of expected names. Provided the output has the exact same cluster names as the expectation, this test will pass.

Closing Thoughts

The line between testing and schema specification in Cue is very blurry. I'm not yet really sure whether line is real or whether it's just a matter of perspective. I think that schemas are statements that are always true, about either the input or the output. Whereas tests, are statements about a particular set of outputs. As such, tests can be far more specific - down to exact values - than can schemas.

Either way, the ability to define small schemas to capture expectations about the output of transformations is incredibly valuable.