pkg/ottl: add section about design principles

[axw]

Description:

Drawing inspiration from https://github.com/bazelbuild/starlark#design-principles and https://github.com/google/cel-spec/blob/master/doc/langdef.md#overview, add a brief section about design principles.

The aim of this is to ensure OTTL is and remains safe for execution of untrusted programs in multi-tenant systems, where tenants can provide their own OTTL programs.

[axw]

@evan-bradley @TylerHelmuth apologies for the delay, we discussed this at KubeCon. I just came across #29289 and it jogged my memory.

[TylerHelmuth]

I really like the idea of documenting design principals, but I am not sure I want to commit to these yet.

OTTL programs operating in separate contexts cannot influence one another -- an OTTL program may have side-effects only within its own execution context.

While the current contexts have restricted cross-cutting between signals, there isn't anything stopping someone from using OTTL with a custom context that does allow this. OTTL is designed to not worry about how the getting and setting is done or what data is being manipulated. I'd prefer to not restrict ourselves with this principal because some day it may be necessary to support more cross-cutting scenarios.

OTTL programs cannot loop forever, except through the use of non built-in functions.

Can you provide more what you mean by this statement? I disagree with the term program as OTTL is not a programming language.

A design principles for OTTL that I do feel confident about documenting are:

1. Context cannot provide paths that reach "lower" in the OTLP heirarchy. A function in the metric context, for example, can provide access to the DataPointSlice, but not individual datapoints.
2. OTTL Functions should trend towards erroring when dealing with unexpected results or situations.

[axw]

@TylerHelmuth thanks for your thoughts!

While the current contexts have restricted cross-cutting between signals, there isn't anything stopping someone from using OTTL with a custom context that does allow this. OTTL is designed to not worry about how the getting and setting is done or what data is being manipulated. I'd prefer to not restrict ourselves with this principal because some day it may be necessary to support more cross-cutting scenarios.

Sorry, I worded this poorly. What I intended was to (roughly) capture this Starlark design principle:

• Hermetic execution. Execution cannot access the file system, network, system clock. It is safe to execute untrusted code.

Execution should definitely not be able to access the system or network. I don't see any reason to exclude the system clock, and doing so would would exclude the Now() converter.

In addition to that, what I meant that execution of one OTTL program (sorry, need another word) must not be able to leak into the execution of another.

OTTL programs cannot loop forever, except through the use of non built-in functions.

Can you provide more what you mean by this statement?

I was trying to capture what CEL says:

• terminating: CEL programs cannot loop forever;

The point being to guarantee that CEL can be evaluated without taking down the system evaluating it. It doesn't matter so much for single-tenant systems, since you can just say "don't do that." But for multi-tenant systems it will affect others.

It's not quite enough to say that about the grammar -- as long as there are built-in functions, they would be an escape hatch if they're not also covered by this principle. We can't really enforce what a non built-in function does, hence the comment "except through the use of non built-in functions".

I disagree with the term program as OTTL is not a programming language.

Right, I wasn't sure what to call it, and it felt a little wrong even if "program" doesn't necessarily mean "general purpose program". FWIW, I went looking for inspiration in cuelang, and found a few references to "CUE program" at:

• https://cuelang.org/docs/references/
• https://cuelang.org/docs/references/spec/
• https://cuelang.org/docs/integrations/go/

Having said that, it doesn't appear to be defined anywhere formally 🤷‍♂️

Is there an established word to describe a series of OTTL statements?

[TylerHelmuth]

Hermetic execution. Execution cannot access the file system, network, system clock. It is safe to execute untrusted code.

I'm not sure we want to restrict functions in this way. Although we wouldn't include it in the transform processor, OTTL as a framework allows users to create functions that interact with the network or filesystem. Since function inclusion is per-component and a compile-time decision there is no untrusted code - OTTL does not support dynamic function loading or remote code execution.

In addition to that, what I meant that execution of one OTTL program (sorry, need another word) must not be able to leak into the execution of another.

This I do agree with. OTTL statements should be completely independent.

It's not quite enough to say that about the grammar -- as long as there are built-in functions, they would be an escape hatch if they're not also covered by this principle. We can't really enforce what a non built-in function does, hence the comment "except through the use of non built-in functions".

Ok I see what you're saying. Yes I think it is safe to say we wouldn't create any functions in ottlfuncs that would loop forever.

Is there an established word to describe a series of OTTL statements?

Not officially. I refer to them a statements.

[evan-bradley]

Could we move these design goals to be in the ottlfuncs readme? I think they make a lot of sense for the OTTL standard library, which I would expect to be limited in scope and to have good security guarantees. For the language itself, since OTTL statements run directly in the Go VM we can't make any guarantees for what a user-authored function could do in its implementation.

[axw]

Could we move these design goals to be in the ottlfuncs readme? I think they make a lot of sense for the OTTL standard library, which I would expect to be limited in scope and to have good security guarantees.

Sounds good, I'll have a go at that.

For the language itself, since OTTL statements run directly in the Go VM we can't make any guarantees for what a user-authored function could do in its implementation.

I agree that we can't make any guarantees about user-defined functions, but I still think it's important to address the language to cover any future changes. For example #29289 has a few options, one of which is to add looping to the language. That's fine because the proposed loop syntax would guarantee termination. (Obviously you know all that, just giving an example to explain my rationale.)

[axw]

Since function inclusion is per-component and a compile-time decision there is no untrusted code - OTTL does not support dynamic function loading or remote code execution.

Understood. Should we capture that here?

[axw]

Added a mention of no dynamic loading/evaluation, and moved some bits to ottlfuncs.

[evan-bradley]

Thanks for clarifying these points in the docs.