Use component method calls for multi deployer logging. #630

ghemawat · 2023-09-20T16:43:32Z

We plan to make it easier to create deployers by providing pluggable system facilities (logging for example). We will achieve this by using component method calls to communicate between a weavelet and its envelope.

This change starts us on this path by adding a logging component and changing the multi deployer to collect log entries via method calls made by the weavelet:

The weavelet contains a Logger component that logs to stderr.
The multi deployer has a different component multiLogger that has the same set of methods as the Logger component, but writes log entries to the multi deployer's database.
This multiLogger component is made available over a unix domain socket.
The multi deployer passes configuration information to the weavelet asking it to redirect calls made to the Logger component to the multiLogger component.
The weavelet calls a Logger method to write log entries (though for the benefit of other deployers, it falls back to using the envelope pipe protocol if a Logger redirect is not available).

Other related improvements

The multi deplopyer could previously lose log messages since the process could exit before the log messages written to the logs database could be read and printed to stderr. We now write log messages to stderr as soon as they arrive instead of routing them through the logs database.

Other Details

Fix generator so it can handle components defined in the weaver package.
multi deployer creates a temporary directory and a unix domain socket inside it.
The Logger component method accepts a batch of log entries, which should help efficiency when the rate of logging is high.
Fixed comment with outdated example of logging output.
Added a list of redirects to EnvelopeInfo proto.

Added helpers

call.NewHandlerMap() provides a single place to register ready method.
FixedListener provides a call.Listener that serves a fixed set of components that are not subject to any communication graph constraints.
reflection.ComponentName[T] returns the name of component T.
codegen.Find returns registration by name.
dirs.NewTempDir makes a provate temporary directory.
logger.IsSystemGenerated determines whether a log entry is system generated.

We plan to make it easier to create deployers by providing pluggable system facilities (logging for example). We will achieve this by using component method calls to communicate between a weavelet and its envelope. This change starts us on this path by adding a logging component and changing the multi deployer to collect log entries via method calls made by the weavelet: 1. The weavelet contains a Logger component that logs to stderr. 2. The multi deployer has a different component multiLogger that has the same set of methods as the Logger component, but writes log entries to the multi deployer's database. 3. This multiLogger component is made available over a unix domain socket. 4. The multi deployer passes configuration information to the weavelet asking it to redirect calls made to the Logger component to the multiLogger component. 5. The weavelet calls a Logger method to write log entries (though for the benefit of other deployers, it falls back to using the envelope pipe protocol if a Logger redirect is not available). Other related improvements -------------------------- The multi deplopyer could previously lose log messages since the process could exit before the log messages written to the logs database could be read and printed to stderr. We now write log messages to stderr as soon as they arrive instead of routing them through the logs database. Other Details ------------- * Fix generator so it can handle components defined in the weaver package. * multi deployer creates a temporary directory and a unix domain socket inside it. * The Logger component method accepts a batch of log entries, which should help efficiency when the rate of logging is high. * Fixed comment with outdated example of logging output. * Added a list of redirects to `EnvelopeInfo` proto. Added helpers ------------- * `call.NewHandlerMap()` provides a single place to register ready method. * `FixedListener` provides a call.Listener that serves a fixed set of components that are not subject to any communication graph constraints. * `reflection.ComponentName[T]` returns the name of component T. * `codegen.Find` returns registration by name. * `dirs.NewTempDir` makes a provate temporary directory. * `logger.IsSystemGenerated` determines whether a log entry is system generated.

spetrovic77

Good job fitting this in so nicely.

spetrovic77 · 2023-09-20T22:43:46Z

internal/net/call/handlers.go

 fp := MakeMethodKey(component, method)
 hm.handlers[fp] = handler
 hm.names[fp] = component + "." + method
 }
+
+// addHandlers adds handlers for all methods of the component with the
+// specified name, The handlers invoke methods on the specified impl.


spetrovic77 · 2023-09-20T22:50:22Z

internal/net/call/listener.go

+ Addr() net.Addr
+}
+
+// FixedListener returns a listener allows all calls to supplied


This sentence doesn't parse.

Also, I'm not sure what listener to compare this one to, to understand what is its distinctive property. "Fixed" by itself doesn't describe that property. (I'm guessing the distinctive property is that it allows any/all clients to invoke it, without security checks.)

Fixed sentence.

I had intended "Fixed" to mean that a fixed set of components were being served, not the dynamic set in our normal mTLS based protocol.

I am happy to adopt a different name if that helps.

In the meantime I extended the comment:

// FixedListener returns a listener that allows all calls to supplied // components. FixedListener is intended for local system services and access // control happens at the net.Listener level (e.g., by using Unix domain // sockets). // // Note that this differs from the normal listener used for application // components (that listener only allows component A to call component B if A // has an edge to B in the component graph). // // Each components map entry has the full component name as the key, and the // component implementation as the value.

spetrovic77 · 2023-09-20T23:05:18Z

internal/weaver/logger.go

+ "github.com/ServiceWeaver/weaver/runtime/protos"
+)
+
+// remoteLogger sends log entries to a specified component method.


specified component method dst.

Redid comments on the type and on run().

spetrovic77 · 2023-09-20T23:06:49Z

internal/weaver/remoteweavelet.go

+ const loggerPath = "github.com/ServiceWeaver/weaver/Logger"
+ r, ok := w.redirects[loggerPath]
+ if !ok {
+ // For now fall back to sending over the pipe to the weavelet.


Nit: comma after now

spetrovic77 · 2023-09-20T23:10:17Z

runtime/dir.go

@@ -53,3 +53,19 @@ func DataDir() (string, error) {

 return regDir, nil
 }
+
+// NewTempDir returns a new directory, e.g., to hold Unix domain sockets for
+// internal communication. The new directory is not accessible by others.


nit: others seems too generic

spetrovic77 · 2023-09-20T23:15:26Z

internal/weaver/logger.go

+ return rl
+}
+
+func (rl *remoteLogger) send(entry *protos.LogEntry) {


send -> log ? (since we aren't sending yet.)

ghemawat

Thanks!

ghemawat · 2023-09-21T16:15:28Z

internal/net/call/handlers.go

 fp := MakeMethodKey(component, method)
 hm.handlers[fp] = handler
 hm.names[fp] = component + "." + method
 }
+
+// addHandlers adds handlers for all methods of the component with the
+// specified name, The handlers invoke methods on the specified impl.


ghemawat · 2023-09-21T16:21:18Z

internal/net/call/listener.go

+ Addr() net.Addr
+}
+
+// FixedListener returns a listener allows all calls to supplied


Fixed sentence.

I had intended "Fixed" to mean that a fixed set of components were being served, not the dynamic set in our normal mTLS based protocol.

I am happy to adopt a different name if that helps.

In the meantime I extended the comment:

// FixedListener returns a listener that allows all calls to supplied // components. FixedListener is intended for local system services and access // control happens at the net.Listener level (e.g., by using Unix domain // sockets). // // Note that this differs from the normal listener used for application // components (that listener only allows component A to call component B if A // has an edge to B in the component graph). // // Each components map entry has the full component name as the key, and the // component implementation as the value.

ghemawat · 2023-09-21T16:25:33Z

internal/weaver/logger.go

+ "github.com/ServiceWeaver/weaver/runtime/protos"
+)
+
+// remoteLogger sends log entries to a specified component method.


Redid comments on the type and on run().

ghemawat · 2023-09-21T16:26:25Z

internal/weaver/logger.go

+ return rl
+}
+
+func (rl *remoteLogger) send(entry *protos.LogEntry) {


ghemawat · 2023-09-21T16:27:24Z

internal/weaver/remoteweavelet.go

+ const loggerPath = "github.com/ServiceWeaver/weaver/Logger"
+ r, ok := w.redirects[loggerPath]
+ if !ok {
+ // For now fall back to sending over the pipe to the weavelet.


ghemawat · 2023-09-21T16:28:13Z

runtime/dir.go

@@ -53,3 +53,19 @@ func DataDir() (string, error) {

 return regDir, nil
 }
+
+// NewTempDir returns a new directory, e.g., to hold Unix domain sockets for
+// internal communication. The new directory is not accessible by others.


ghemawat requested a review from spetrovic77 September 20, 2023 16:43

spetrovic77 approved these changes Sep 20, 2023

View reviewed changes

ghemawat commented Sep 21, 2023

View reviewed changes

Small fixes in response to code review comments.

0e0ee96

ghemawat merged commit c841224 into ServiceWeaver:main Sep 21, 2023
7 checks passed

ghemawat deleted the logger branch September 21, 2023 16:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use component method calls for multi deployer logging. #630

Use component method calls for multi deployer logging. #630

ghemawat commented Sep 20, 2023

spetrovic77 left a comment

spetrovic77 Sep 20, 2023

ghemawat Sep 21, 2023

spetrovic77 Sep 20, 2023

ghemawat Sep 21, 2023

spetrovic77 Sep 20, 2023

ghemawat Sep 21, 2023

spetrovic77 Sep 20, 2023

ghemawat Sep 21, 2023

spetrovic77 Sep 20, 2023

ghemawat Sep 21, 2023

spetrovic77 Sep 20, 2023

ghemawat Sep 21, 2023

ghemawat left a comment

ghemawat Sep 21, 2023

ghemawat Sep 21, 2023

ghemawat Sep 21, 2023

ghemawat Sep 21, 2023

ghemawat Sep 21, 2023

ghemawat Sep 21, 2023

Use component method calls for multi deployer logging. #630

Use component method calls for multi deployer logging. #630

Conversation

ghemawat commented Sep 20, 2023

Other related improvements

Other Details

Added helpers

spetrovic77 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghemawat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment