Configuration

Learn how to configure Probitas scenarios, steps, and clients. This guide covers all available options and common configuration patterns.

Project Configuration

Configure Probitas CLI defaults in a configuration file in your project root.

Configuration File Names

Probitas looks for configuration files in this priority order:

1. probitas.json - Standard JSON format
2. probitas.jsonc - JSON with comments (recommended)
3. .probitas.json - Hidden JSON format
4. .probitas.jsonc - Hidden JSON with comments

The first file found is used. JSONC files support // and /* */ comments.

Configuration Options

{
  // Glob patterns for scenario file discovery
  "includes": ["probitas/**/*.probitas.ts"],
  // Glob patterns to exclude from discovery
  "excludes": ["**/*.skip.probitas.ts"],
  // Output reporter (list, json)
  "reporter": "list",
  // Maximum parallel scenario execution (0 = unlimited)
  "maxConcurrency": 4,
  // Maximum failures before stopping (0 = unlimited)
  "maxFailures": 0,
  // Default timeout for scenarios
  "timeout": "30s",
  // Default step options for all scenarios
  "stepOptions": {
    "timeout": 30000,
    "retry": {
      "maxAttempts": 1,
      "backoff": "linear"
    }
  },
  // Default selectors for filtering scenarios
  "selectors": ["!tag:wip"]
}

*Default exclude patterns:

• **/node_modules/**, **/target/**, **/.venv/**, **/venv/**, **/__pycache__/**, **/vendor/**, **/build/**, **/bin/**, **/obj/**, **/.git/**, **/dist/**, **/coverage/**

maxFailures

Controls when the test runner stops execution:

• 0 (default): Run all scenarios regardless of failures
• 1: Fail-fast mode - stop immediately on first failure
• n: Stop after n failures

{
  // Fail-fast: stop on first failure
  "maxFailures": 1
}

stepOptions

Set default timeout and retry behavior for all steps across all scenarios. These defaults can be overridden at the scenario level or individual step level.

{
  "stepOptions": {
    // Default timeout for each step in milliseconds
    "timeout": 30000,
    // Default retry configuration for transient failures
    "retry": {
      // Maximum retry attempts (1 = no retry)
      "maxAttempts": 3,
      // Backoff strategy: "linear" or "exponential"
      "backoff": "exponential"
    }
  }
}

These project-level defaults apply to all scenarios unless overridden. Override precedence (highest to lowest):

1. Individual step options (highest priority)
2. Scenario-level stepOptions
3. Project-level stepOptions (this setting)
4. Framework defaults (lowest priority)

See Scenario Options and Step Options for how to override these defaults.

Selectors

Selectors filter which scenarios to run. The format is [!][type:]value:

Multiple selectors use OR logic. Comma-separated values within a selector use AND logic:

{
  "selectors": ["tag:api,!tag:slow"]
}

This runs scenarios with api tag AND without slow tag.

CLI Override

Command-line options override configuration file settings:

# Override reporter
probitas run --reporter json

# Override concurrency
probitas run --max-concurrency 1

# Fail-fast mode
probitas run --max-failures 1

# Add selectors (combined with config selectors)
probitas run -s tag:smoke

Re-running Failed Scenarios

Use the --failed (-F) flag to re-run only scenarios that failed in the previous run. This is useful for iterating on failing tests without waiting for all scenarios to complete.

# Run all scenarios
probitas run

# Re-run only scenarios that failed
probitas run --failed

# Short form
probitas run -F

How It Works

1. After each probitas run, the CLI saves the list of failed scenarios to .probitas/last-run.json
2. When --failed is specified, only scenarios matching the saved list are executed
3. The filter is applied after selectors (AND logic)

# Run failed scenarios that also have @api tag
probitas run -F -s tag:api

# Run failed scenarios excluding @slow tag
probitas run -F -s "!tag:slow"

State File

The .probitas/ directory contains machine-specific state and should be added to .gitignore:

# Probitas state directory
.probitas/

The state file (last-run.json) contains:

• Schema version for forward compatibility
• Timestamp of when the run completed
• List of failed scenarios (name, file path, and error message)

Unknown Argument Detection

Probitas CLI provides helpful error messages when you use unknown options. This helps catch typos and guides you toward the correct syntax.

Common Mistakes

The CLI recognizes common mistakes and provides contextual hints:

# Mistake: --tag instead of -s "tag:value"
$ probitas run --tag api
Unknown option: --tag
Did you mean '-s "tag:api"'? Use the selector option to filter by tag.

# Mistake: --name instead of -s "name:value"
$ probitas run --name Login
Unknown option: --name
Did you mean '-s "name:Login"'? Use the selector option to filter by name.

# Mistake: --filter instead of -s
$ probitas run --filter smoke
Unknown option: --filter
Did you mean '-s "smoke"'? Use the selector option to filter scenarios.

Typo Detection

For other unknown options, the CLI suggests similar known options using Levenshtein distance:

# Typo: --verbos instead of --verbose
$ probitas run --verbos
Unknown option: --verbos
Did you mean '--verbose'?

# Typo: --time-out instead of --timeout
$ probitas run --time-out 10s
Unknown option: --time-out
Did you mean '--timeout'?

Fallback Help

When no similar option is found, you'll be directed to the help command:

$ probitas run --unknown-flag
Unknown option: --unknown-flag
Run 'probitas run --help' for available options.

Scenario Options

Configure scenarios using the options parameter of `scenario()`.

import { scenario } from "jsr:@probitas/probitas";

scenario("My Test", {
  tags: ["api", "integration"],
  stepOptions: {
    timeout: 60000,
    retry: { maxAttempts: 3, backoff: "exponential" },
  },
})
  .step(() => {})
  .build();

Tags

Use tags to categorize and filter scenarios:

import { scenario } from "jsr:@probitas/probitas";

scenario("API Integration", {
  tags: ["api", "integration", "slow"],
})
  .step(() => {})
  .build();

Run scenarios by tag using selectors:

# Run only "api" tagged scenarios
probitas run -s tag:api

# Exclude "slow" scenarios
probitas run -s "!tag:slow"

# Combine filters (AND logic)
probitas run -s "tag:api,!tag:slow"

Step Options

Override scenario defaults for individual steps:

import { scenario } from "jsr:@probitas/probitas";

scenario("Mixed Timeouts")
  .step(
    "Quick check",
    async (_ctx) => {
      // Must complete in 1 second
    },
    { timeout: 1000 },
  )
  .step(
    "Slow operation",
    async (_ctx) => {
      // Can take up to 60 seconds
    },
    { timeout: 60000 },
  )
  .step(
    "Flaky external call",
    async (_ctx) => {
      // Retry up to 5 times with exponential backoff
    },
    {
      retry: { maxAttempts: 5, backoff: "exponential" },
    },
  )
  .build();

Timeout Behavior

When a step times out:

1. The step's ctx.signal is aborted
2. The step fails with TimeoutError
3. Retry logic applies if configured
4. Cleanup hooks still execute

import { scenario } from "jsr:@probitas/probitas";

const url = "https://api.example.com/data";

scenario("Timeout Example")
  .step(
    "Cancellable operation",
    async (ctx) => {
      // Pass signal to cancellable operations
      const response = await fetch(url, { signal: ctx.signal });
      return response.json();
    },
    { timeout: 5000 },
  )
  .build();

Retry Configuration

Configure automatic retries for flaky operations.

Backoff Strategies

Custom Retry Logic

import { client, scenario } from "jsr:@probitas/probitas";

scenario("Retry Example")
  .resource(
    "http",
    () => client.http.createHttpClient({ url: "http://localhost:8080" }),
  )
  .step(async (ctx) => {
    const { http } = ctx.resources;
    const res = await http.get("/endpoint", {
      retry: {
        maxAttempts: 3,
        backoff: "exponential",
        initialDelay: 500,
        maxDelay: 10000,
        retryOn: (error: Error) => {
          // Only retry on specific errors
          return error.message.includes("network") ||
            error.message.includes("timeout");
        },
      },
    });
    return res.json;
  })
  .build();

Client Configurations

HTTP Client

import { client } from "jsr:@probitas/probitas";

client.http.createHttpClient({
  url: "http://localhost:8080",
  headers: { "Content-Type": "application/json" },
  throwOnError: true,
  timeout: 10000,
});

PostgreSQL Client

import { client } from "jsr:@probitas/probitas";

client.sql.postgres.createPostgresClient({
  url: {
    host: "localhost",
    port: 5432,
    database: "testdb",
    username: "testuser",
    password: "testpass",
  },
  pool: { max: 10 },
});

Connection can also be a URL string:

import { client } from "jsr:@probitas/probitas";

client.sql.postgres.createPostgresClient({
  url: "postgres://user:pass@localhost:5432/mydb",
});

gRPC Client

import { client } from "jsr:@probitas/probitas";

client.grpc.createGrpcClient({
  url: "localhost:50051",
  metadata: { authorization: "Bearer token" },
  tls: { insecure: false },
});

GraphQL Client

import { client } from "jsr:@probitas/probitas";

client.graphql.createGraphqlClient({
  url: "http://localhost:4000/graphql",
  headers: { Authorization: "Bearer token" },
});

Redis Client

import { client } from "jsr:@probitas/probitas";

client.redis.createRedisClient({
  url: "redis://localhost:6379",
});

MongoDB Client

import { client } from "jsr:@probitas/probitas";

client.mongodb.createMongoClient({
  url: "mongodb://localhost:27017",
  database: "testdb",
});

RabbitMQ Client

import { client } from "jsr:@probitas/probitas";

client.rabbitmq.createRabbitMqClient({
  url: "amqp://guest:guest@localhost:5672",
});

SQS Client

import { client } from "jsr:@probitas/probitas";

client.sqs.createSqsClient({
  url: "http://localhost:4566",
  region: "us-east-1",
  credentials: {
    accessKeyId: "test",
    secretAccessKey: "test",
  },
});

Environment Variables

Probitas respects these environment variables:

Using Environment Variables

import { client, scenario } from "jsr:@probitas/probitas";

scenario("Production Test")
  .resource("http", () =>
    client.http.createHttpClient({
      url: Deno.env.get("API_URL") ?? "http://localhost:8080",
      headers: {
        Authorization: `Bearer ${Deno.env.get("API_TOKEN")}`,
      },
    }))
  .resource("pg", () =>
    client.sql.postgres.createPostgresClient({
      url: Deno.env.get("DATABASE_URL") ??
        "postgres://user:pass@localhost/testdb",
    }))
  .step(() => {})
  .build();

Configuration Patterns

Environment-Aware Configuration

import { client, scenario } from "jsr:@probitas/probitas";

const isProduction = Deno.env.get("ENV") === "production";

scenario("API Test")
  .resource("http", () =>
    client.http.createHttpClient({
      url: isProduction ? "https://api.example.com" : "http://localhost:8080",
      timeout: isProduction ? 30000 : 5000,
      retry: isProduction
        ? { maxAttempts: 3, backoff: "exponential" }
        : undefined,
    }))
  .step(() => {})
  .build();

Shared Configuration

import { client, scenario } from "jsr:@probitas/probitas";

const httpDefaults = {
  timeout: 10000,
  headers: { "Content-Type": "application/json" },
};

const dbDefaults = {
  pool: { min: 1, max: 5 },
};

scenario("Test")
  .resource("http", () =>
    client.http.createHttpClient({
      url: "http://localhost:8080",
      ...httpDefaults,
    }))
  .resource("pg", () =>
    client.sql.postgres.createPostgresClient({
      url: "postgres://user:pass@localhost:5432/mydb",
      ...dbDefaults,
    }))
  .step(() => {})
  .build();

Resource Factory Pattern

import { client, scenario } from "jsr:@probitas/probitas";

function createApiClient() {
  return client.http.createHttpClient({
    url: Deno.env.get("API_URL") ?? "http://localhost:8080",
    timeout: 10000,
    retry: { maxAttempts: 2 },
  });
}

function createTestDatabase() {
  return client.sql.postgres.createPostgresClient({
    url: {
      host: Deno.env.get("DB_HOST") ?? "localhost",
      port: Number(Deno.env.get("DB_PORT") ?? 5432),
      database: "testdb",
      username: "testuser",
      password: "testpass",
    },
  });
}

scenario("Test")
  .resource("http", createApiClient)
  .resource("pg", createTestDatabase)
  .step(() => {})
  .build();

Conditional Resources

import { client, scenario, Skip } from "jsr:@probitas/probitas";

scenario("Conditional Features")
  .resource(
    "http",
    () => client.http.createHttpClient({ url: "http://localhost:8080" }),
  )
  .resource("redis", () => {
    const redisUrl = Deno.env.get("REDIS_URL");
    if (!redisUrl) {
      throw new Skip("Redis not configured");
    }
    return client.redis.createRedisClient({
      url: redisUrl,
    });
  })
  .step(() => {})
  .build();