0
# Metlo API Security Platform
1

2
Metlo is a comprehensive open-source API security platform that provides endpoint discovery, sensitive data scanning, vulnerability detection, and proactive security testing capabilities. With Metlo, organizations can create complete API inventories, detect vulnerabilities like unauthenticated endpoints returning sensitive data, perform automated security testing with OWASP Top 10 coverage, and detect API attacks in real-time.
3

4
## Package Information
5

6
- **Package Name**: metlo
7
- **Package Type**: github (multi-project)
8
- **Language**: TypeScript/JavaScript (Node.js)
9
- **Repository**: https://github.com/metlo-labs/metlo
10
- **License**: MIT
11
- **Version**: 0.1.1
12

13
## Core Imports
14

15
The Metlo platform consists of several npm packages that work together:
16

17
### CLI Package (@metlo/cli)
18
```bash
19
npm install -g @metlo/cli
20
```
21

22
```typescript
23
// CLI is used via command line
24
metlo init --backend_url https://your-backend.com --api_key your-key
25
metlo test generate --testType auth --endpoint /api/users --host api.example.com
26
```
27

28
### Testing Framework (@metlo/testing)
29
```bash
30
npm install @metlo/testing
31
```
32

33
```typescript
34
import {
35
  TestBuilder,
36
  TestStepBuilder,
37
  runTest,
38
  loadTestConfig,
39
  dumpTestConfig,
40
  AssertionType,
41
  ExtractorType
42
} from "@metlo/testing";
43
```
44

45
### Common Types and Utilities (@metlo/common)
46
```bash
47
npm install @metlo/common
48
```
49

50
```typescript
51
import {
52
  RestMethod,
53
  RiskScore,
54
  AlertType,
55
  AttackType,
56
  ApiEndpoint,
57
  ApiTrace,
58
  Alert,
59
  DataField
60
} from "@metlo/common";
61
```
62

63
## Basic Usage
64

65
### Setting Up the CLI
66
```bash
67
# Install the CLI globally
68
npm install -g @metlo/cli
69

70
# Initialize CLI with backend connection
71
metlo init --backend_url https://your-metlo-backend.com --api_key your-api-key
72

73
# Set up traffic mirroring for AWS
74
metlo traffic-mirror aws new --region us-east-1 --source-eni-id eni-12345 --target-eni-id eni-67890
75
```
76

77
### Creating and Running Security Tests
78
```typescript
79
import { TestBuilder, TestStepBuilder, runTest, AssertionType } from "@metlo/testing";
80

81
// Create a test configuration
82
const testBuilder = new TestBuilder("test-authentication")
83
  .setMeta({
84
    name: "Authentication Test",
85
    severity: "CRITICAL",
86
    tags: ["BROKEN_AUTHENTICATION"]
87
  });
88

89
// Add test steps
90
const step1 = TestStepBuilder.sampleRequest(endpoint, config)
91
  .assert({
92
    type: AssertionType.enum.EQ,
93
    key: "resp.status",
94
    value: 200
95
  });
96

97
const step2 = TestStepBuilder.sampleRequestWithoutAuth(endpoint, config)
98
  .assert({
99
    type: AssertionType.enum.EQ,
100
    key: "resp.status",
101
    value: [401, 403]
102
  });
103

104
const testConfig = testBuilder
105
  .addStep(step1.getStep())
106
  .addStep(step2.getStep())
107
  .build();
108

109
// Run the test
110
const result = await runTest(testConfig);
111
```
112

113
### Working with API Data Types
114
```typescript
115
import { ApiEndpoint, Alert, RestMethod, RiskScore, AlertType } from "@metlo/common";
116

117
// Type-safe API endpoint representation
118
const endpoint: ApiEndpoint = {
119
  uuid: "endpoint-123",
120
  path: "/api/users/:id",
121
  method: RestMethod.GET,
122
  host: "api.example.com",
123
  riskScore: RiskScore.HIGH,
124
  resourcePermissions: ["read:user"],
125
  isAuthenticatedDetected: false,
126
  // ... other properties
127
};
128

129
// Security alert representation
130
const alert: Alert = {
131
  uuid: "alert-456",
132
  type: AlertType.UNAUTHENTICATED_ENDPOINT_SENSITIVE_DATA,
133
  riskScore: RiskScore.HIGH,
134
  description: "Endpoint returns PII data without authentication",
135
  // ... other properties
136
};
137
```
138

139
## Architecture
140

141
The Metlo platform is built around several key components:
142

143
- **CLI Tools**: Command-line interface for deployment automation, traffic mirroring setup, and test management
144
- **Testing Framework**: Extensible security testing system with YAML-based test configuration and automated test generation
145
- **Common Library**: Shared TypeScript types and utilities used across all components
146
- **Backend Services**: REST API services for traffic analysis, vulnerability detection, and data management
147
- **Ingestors**: Traffic collection agents for various platforms (Node.js, Python, Java, Golang, Rust)
148
- **Frontend Dashboard**: React-based web interface for security monitoring and management
149

150
## Capabilities
151

152
### Command Line Interface
153

154
Complete CLI toolset for Metlo operations including initialization, traffic mirroring setup, and security test management.
155

156
```typescript { .api }
157
// Available as global CLI commands after npm install -g @metlo/cli
158
// metlo init --backend_url <url> --api_key <key>
159
// metlo test generate --testType <type> --endpoint <endpoint>
160
// metlo test run [paths...] --verbose
161
// metlo traffic-mirror aws new --region <region> --source-eni-id <id> --target-eni-id <id>
162
```
163

164
[Command Line Interface](./cli.md)
165

166
### Security Testing Framework
167

168
Comprehensive testing framework for automated security test generation and execution with support for OWASP Top 10 vulnerabilities.
169

170
```typescript { .api }
171
function runTest(config: TestConfig, context?: any): Promise<TestResult>;
172
function loadTestConfig(path: string): TestConfig;
173
function dumpTestConfig(config: TestConfig): string;
174
function generateAuthTests(endpoint: GenTestEndpoint, config: TemplateConfig): TestConfig[];
175

176
class TestBuilder {
177
  constructor(id: string);
178
  setMeta(meta: TestMeta): TestBuilder;
179
  addStep(step: TestStep): TestBuilder;
180
  setOptions(options: TestOptions): TestBuilder;
181
  addEnv(...items: KeyValType[]): TestBuilder;
182
  build(): TestConfig;
183
}
184

185
class TestStepBuilder {
186
  constructor(request: TestRequest);
187
  static sampleRequest(endpoint: GenTestEndpoint, config: TemplateConfig, name?: string): TestStepBuilder;
188
  static sampleRequestWithoutAuth(endpoint: GenTestEndpoint, config: TemplateConfig, name?: string): TestStepBuilder;
189
  assert(assertion: Assertion): TestStepBuilder;
190
  extract(item: Extractor): TestStepBuilder;
191
  addPayloads(payload: PayloadType): TestStepBuilder;
192
  getStep(): TestStep;
193
}
194
```
195

196
[Security Testing Framework](./testing.md)
197

198
### Data Types and API Models
199

200
Comprehensive type definitions for API security data including endpoints, traces, alerts, and attacks.
201

202
```typescript { .api }
203
interface ApiEndpoint {
204
  uuid: string;
205
  path: string;
206
  method: RestMethod;
207
  host: string;
208
  riskScore: RiskScore;
209
  resourcePermissions: string[];
210
  isAuthenticatedDetected: boolean;
211
  dataClasses?: string[];
212
  createdAt: Date;
213
  updatedAt: Date;
214
}
215

216
interface Alert {
217
  uuid: string;
218
  type: AlertType;
219
  riskScore: RiskScore;
220
  apiEndpointUuid: string;
221
  description: string;
222
  status: Status;
223
  createdAt: Date;
224
  updatedAt: Date;
225
}
226

227
interface ApiTrace {
228
  uuid: string;
229
  path: string;
230
  method: RestMethod;
231
  host: string;
232
  requestHeaders: PairObject[];
233
  requestBody: string;
234
  responseStatus: number;
235
  responseHeaders: PairObject[];
236
  responseBody: string;
237
  createdAt: Date;
238
}
239
```
240

241
[Data Types and API Models](./types.md)
242

243
### Traffic Analysis and Detection
244

245
Core functionality for analyzing API traffic, detecting sensitive data, and identifying security vulnerabilities.
246

247
```typescript { .api }
248
interface ProcessedTraceData {
249
  block: boolean;
250
  xssDetected: Record<string, string>;
251
  sqliDetected: Record<string, [string, string]>;
252
  sensitiveDataDetected: Record<string, string[]>;
253
  dataTypes: Record<string, string[]>;
254
  validationErrors: Record<string, string[]>;
255
}
256

257
interface DataField {
258
  uuid: string;
259
  dataClasses: string[];
260
  dataPath: string;
261
  dataSection: DataSection;
262
  dataType: DataType;
263
  dataTag: DataTag;
264
  matches: Record<string, string[]>;
265
  apiEndpointUuid: string;
266
}
267
```
268

269
[Traffic Analysis and Detection](./analysis.md)
270

271
### Backend REST API
272

273
Comprehensive REST API endpoints for managing API security data, alerts, traces, and platform configuration.
274

275
```typescript { .api }
276
// Endpoint Management
277
GET /api/v1/endpoints
278
GET /api/v1/endpoint/{endpointId}
279
PUT /api/v1/endpoint/{endpointId}/authenticated
280
DELETE /api/v1/endpoints
281

282
// Alert Management  
283
GET /api/v1/alerts
284
PUT /api/v1/alert/{alertId}
285
PUT /api/v1/alerts/batch
286

287
// Data Collection
288
POST /api/v1/log-request/single
289
POST /api/v1/log-request/batch
290

291
// Testing API
292
POST /api/v1/test/generate
293
POST /api/v1/test/run
294
GET /api/v1/test/templates
295
```
296

297
[Backend REST API](./backend-api.md)
298

299
### Ingestor SDKs and Integration Libraries
300

301
Multi-language SDK libraries for collecting API traffic across various programming languages and frameworks.
302

303
```typescript { .api }
304
// Node.js Ingestor
305
function init(key: string, host: string, opts?: ConfigOptions): void;
306

307
// Python Flask/Django Integration
308
class MetloFlask(app, api_key, base_url, rps, disable);
309

310
// Java Spring Boot Integration
311
@EnableMetlo // Automatic configuration
312

313
// Go HTTP Middleware  
314
func InitMetlo(metloHost, metloKey string) *Metlo;
315
func (m *Metlo) Middleware(next http.Handler) http.Handler;
316

317
// Rust Integration
318
struct MetloClient;
319
impl MetloClient { pub fn new(config: MetloConfig) -> Self; }
320
```
321

322
[Ingestor SDKs and Integration Libraries](./ingestors.md)
323

324
### Platform Integration and Ingestors
325

326
Traffic collection and platform integration capabilities for various environments and programming languages.
327

328
```typescript { .api }
329
interface TraceParams {
330
  request: Request;
331
  response: Response;
332
  meta: Meta;
333
  processedTraceData?: ProcessedTraceData;
334
  sessionMeta?: SessionMeta;
335
  encryption?: Encryption;
336
}
337

338
interface Meta {
339
  incoming: boolean;
340
  source: string;
341
  sourcePort: string;
342
  destination: string;
343
  destinationPort: string;
344
  originalSource?: string;
345
}
346
```
347

348
[Platform Integration and Ingestors](./integration.md)
349

350
## Types
351

352
### Core Enumerations
353

354
```typescript { .api }
355
enum RestMethod {
356
  GET = "GET",
357
  HEAD = "HEAD", 
358
  POST = "POST",
359
  PUT = "PUT",
360
  PATCH = "PATCH",
361
  DELETE = "DELETE",
362
  CONNECT = "CONNECT",
363
  OPTIONS = "OPTIONS",
364
  TRACE = "TRACE"
365
}
366

367
enum RiskScore {
368
  NONE = "none",
369
  LOW = "low", 
370
  MEDIUM = "medium",
371
  HIGH = "high"
372
}
373

374
enum AlertType {
375
  NEW_ENDPOINT = "New Endpoint Detected",
376
  PII_DATA_DETECTED = "PII Data Detected",
377
  OPEN_API_SPEC_DIFF = "Open API Spec Diff",
378
  QUERY_SENSITIVE_DATA = "Sensitive Data in Query Params",
379
  PATH_SENSITIVE_DATA = "Sensitive Data in Path Params",
380
  BASIC_AUTHENTICATION_DETECTED = "Basic Authentication Detected",
381
  UNSECURED_ENDPOINT_DETECTED = "Endpoint not secured by SSL",
382
  UNAUTHENTICATED_ENDPOINT_SENSITIVE_DATA = "Unauthenticated Endpoint returning Sensitive Data"
383
}
384

385
enum AttackType {
386
  HIGH_USAGE_SENSITIVE_ENDPOINT = "High Usage on Sensitive Endpoint",
387
  HIGH_ERROR_RATE = "High Error Rate", 
388
  ANOMALOUS_CALL_ORDER = "Anomalous Call Order",
389
  UNAUTHENTICATED_ACCESS = "Unauthenticated Access",
390
  BOLA = "Broken Object Level Authorization"
391
}
392

393
enum DataSection {
394
  REQUEST_PATH = "reqPath",
395
  REQUEST_QUERY = "reqQuery", 
396
  REQUEST_HEADER = "reqHeaders",
397
  REQUEST_BODY = "reqBody",
398
  RESPONSE_HEADER = "resHeaders",
399
  RESPONSE_BODY = "resBody"
400
}
401

402
enum Status {
403
  RESOLVED = "Resolved",
404
  IGNORED = "Ignored",
405
  OPEN = "Open"  
406
}
407

408
enum AuthType {
409
  BASIC = "basic",
410
  HEADER = "header",
411
  JWT = "jwt", 
412
  SESSION_COOKIE = "session_cookie"
413
}
414
```
415

416
### Utility Types
417

418
```typescript { .api }
419
interface PairObject {
420
  name: string;
421
  value: string;
422
}
423

424
interface Url {
425
  host: string;
426
  path: string;
427
  parameters: PairObject[];
428
}
429

430
interface Request {
431
  url: Url;
432
  headers: PairObject[];
433
  body: string;
434
  method: RestMethod;
435
}
436

437
interface Response {
438
  status: number;
439
  headers: PairObject[];
440
  body: string;
441
}
442

443
type JSONValue = string | number | boolean | { [x: string]: JSONValue } | Array<JSONValue>;
444
```