0
# @scramjet/runner
1

2
@scramjet/runner is a runtime environment for sequence execution that provides communication mechanisms with the Scramjet Transform Hub host. It enables remote execution of data transformation sequences with comprehensive control, monitoring, and stream management capabilities.
3

4
## Package Information
5

6
- **Package Name**: @scramjet/runner
7
- **Package Type**: npm
8
- **Language**: TypeScript
9
- **Installation**: `npm install @scramjet/runner`
10

11
## Core Imports
12

13
```typescript
14
import { Runner, MessageUtils } from "@scramjet/runner";
15
```
16

17
For CommonJS:
18

19
```javascript
20
const { Runner, MessageUtils } = require("@scramjet/runner");
21
```
22

23
## Basic Usage
24

25
```typescript
26
import { Runner, MessageUtils } from "@scramjet/runner";
27
import { RunnerMessageCode } from "@scramjet/symbols";
28

29
// This package is primarily designed for use by the Scramjet Transform Hub
30
// infrastructure. The Runner class manages sequence execution internally.
31

32
// MessageUtils can be used for writing monitoring messages to streams
33
const monitoringMessage = [RunnerMessageCode.MONITORING, { healthy: true }];
34
MessageUtils.writeMessageOnStream(monitoringMessage, monitorStream);
35

36
// In sequence functions, the context is available as 'this'
37
export default function(this: RunnerAppContext<any, any>, input: DataStream) {
38
  // Use context methods for lifecycle management
39
  this.addStopHandler(async (timeout, canKeepAlive) => {
40
    console.log("Sequence stopping gracefully");
41
  });
42
  
43
  // Send keep-alive if needed
44
  this.keepAlive(5000);
45
  
46
  return input.map(item => ({ ...item, processed: true }));
47
}
48
```
49

50
## Architecture
51

52
@scramjet/runner is built around several key components:
53

54
- **Runner Class**: Core execution engine that manages sequence lifecycle, stream handling, and host communication
55
- **HostClient**: Network communication layer managing multiple streams to/from the Transform Hub host
56
- **RunnerAppContext**: Application context providing APIs for sequences including event handling, monitoring, and lifecycle management
57
- **MessageUtils**: Utility functions for encoding and writing monitoring messages to streams
58
- **Stream Management**: Comprehensive handling of input/output streams with content type detection and serialization
59
- **Communication Protocol**: JSON-based messaging system with multiple communication channels (control, monitoring, data streams)
60

61
## Capabilities
62

63
### Runner Execution
64

65
Core sequence execution environment that manages the complete lifecycle of data transformation sequences.
66

67
```typescript { .api }
68
class Runner<X extends AppConfig> implements IComponent {
69
  constructor(
70
    sequencePath: string,
71
    hostClient: IHostClient,
72
    instanceId: string,
73
    sequenceInfo: SequenceInfo,
74
    runnerConnectInfo: RunnerConnectInfo
75
  );
76
  
77
  main(): Promise<void>;
78
  logger: IObjectLogger;
79
  context: RunnerAppContext<X, any>;
80
}
81
```
82

83
[Runner Execution](./runner-execution.md)
84

85
### Host Communication
86

87
Internal network communication layer managing connections and data streams between runner and Transform Hub host. This is used internally by the Runner class and not exposed as part of the public API.
88

89
[Host Communication](./host-communication.md)
90

91
### Application Context
92

93
Runtime context for sequences providing lifecycle management, event handling, and host integration APIs. This context is accessible within sequence functions as `this` and through the Runner's `context` property.
94

95
```typescript { .api }
96
// Available through runner.context or as 'this' in sequence functions
97
interface RunnerAppContext<AppConfigType extends AppConfig, State extends any> {
98
  config: AppConfigType;
99
  logger: IObjectLogger;
100
  hub: HostClient;
101
  space: ManagerClient;
102
  instanceId: string;
103
  
104
  addKillHandler(handler: KillHandler): this;
105
  addStopHandler(handler: StopHandler): this;
106
  addMonitoringHandler(handler: MonitoringHandler): this;
107
  keepAlive(milliseconds?: number): this;
108
  end(): this;
109
  destroy(error?: AppError): this;
110
  on(eventName: string, handler: (message?: any) => void): this;
111
  emit(eventName: string, message?: any): this;
112
}
113
```
114

115
[Application Context](./application-context.md)
116

117
### Message Utilities
118

119
Utility functions for encoding and transmitting monitoring messages through streams.
120

121
```typescript { .api }
122
class MessageUtils {
123
  static writeMessageOnStream(
124
    message: EncodedMonitoringMessage,
125
    streamToWrite: WritableStream<any>
126
  ): void;
127
}
128
```
129

130
[Message Utilities](./message-utilities.md)
131

132
### Stream Processing
133

134
Internal stream processing utilities for handling different content types and stream formats. These are used internally by the Runner and not exposed as part of the public API.
135

136
[Stream Processing](./stream-processing.md)
137

138
## Core Types
139

140
```typescript { .api }
141
interface AppConfig {
142
  [key: string]: any;
143
}
144

145
interface SequenceInfo {
146
  sequenceId: string;
147
  config: Record<string, any>;
148
}
149

150
interface RunnerConnectInfo {
151
  appConfig: AppConfig;
152
  args?: any[];
153
}
154

155
interface RunnerProxy {
156
  sendKeepAlive(data: KeepAliveMessageData): void;
157
  sendStop(error?: AppError | Error): void;
158
  sendEvent(ev: EventMessageData): void;
159
  keepAliveIssued(): void;
160
}
161

162
interface IComponent {
163
  logger: IObjectLogger;
164
}
165

166
type EncodedMonitoringMessage = [RunnerMessageCode, any];
167
type EncodedControlMessage = [RunnerMessageCode, any];
168

169
type KillHandler = () => void;
170
type StopHandler = (timeout: number, canCallKeepalive: boolean) => Promise<void>;
171
type MonitoringHandler = (message: MonitoringMessageFromRunnerData) => Promise<MonitoringMessageFromRunnerData>;
172

173
interface MonitoringMessageFromRunnerData {
174
  healthy: boolean;
175
}
176

177
interface KeepAliveMessageData {
178
  keepAlive: number;
179
}
180

181
interface EventMessageData {
182
  eventName: string;
183
  message?: any;
184
}
185

186
interface PangMessageData {
187
  requires?: string;
188
  provides?: string;
189
  contentType?: string;
190
  outputEncoding?: string;
191
}
192

193
interface FunctionDefinition {
194
  mode: string;
195
  name: string;
196
  [key: string]: any;
197
}
198

199
interface IHostClient {
200
  logger: IObjectLogger;
201
  
202
  init(id: string): Promise<void>;
203
  disconnect(hard: boolean): Promise<void>;
204
  getAgent(): Agent;
205
  
206
  readonly stdinStream: Readable;
207
  readonly stdoutStream: Writable;
208
  readonly stderrStream: Writable;
209
  readonly controlStream: Readable;
210
  readonly monitorStream: Writable;
211
  readonly inputStream: Readable;
212
  readonly outputStream: Writable;
213
  readonly logStream: Writable;
214
  readonly packageStream: Readable;
215
}
216

217
interface WritableStream<T> {
218
  write(chunk: T): boolean;
219
}
220

221
interface ApplicationInterface {
222
  requires?: string;
223
  contentType?: string;
224
  (...args: any[]): any;
225
}
226

227
interface HasTopicInformation {
228
  topic?: string;
229
  contentType?: string;
230
}
231

232
enum InstanceStatus {
233
  STARTING = "starting",
234
  RUNNING = "running",
235
  STOPPING = "stopping",
236
  COMPLETED = "completed",
237
  ERRORED = "errored",
238
  KILLING = "killing"
239
}
240

241
enum RunnerExitCode {
242
  STOPPED = 0,
243
  KILLED = 1,
244
  SEQUENCE_FAILED_ON_START = 2,
245
  SEQUENCE_FAILED_DURING_EXECUTION = 3,
246
  UNCAUGHT_EXCEPTION = 4,
247
  CLEANUP_FAILED = 5,
248
  INVALID_ENV_VARS = 6,
249
  INVALID_SEQUENCE_PATH = 7
250
}
251

252
type MaybePromise<T> = T | Promise<T>;
253
```