Sessions & Policy

Track RAIL scores across multi-turn conversations and enforce quality policies on AI outputs.

Session Tracking

RAILSession tracks scores across a multi-turn conversation with configurable deep-eval frequency and quality threshold alerts.

import { RAILSession } from '@responsible-ai-labs/rail-score';

const session = new RAILSession(client, {
  deepEvalFrequency: 5,    // Deep eval every 5 turns
  contextWindow: 10,        // Track last 10 turns
  qualityThreshold: 7.0,    // Trigger deep eval when score dips below
});

// Add a conversation turn
const result = await session.addTurn("AI response content");
console.log(result.rail_score.score);

// Session metrics
const metrics = session.getMetrics();
console.log(`Average: ${metrics.averageScore}`);
console.log(`Min: ${metrics.minScore}, Max: ${metrics.maxScore}`);
console.log(`Passing rate: ${metrics.passingRate}`);
console.log(`Turns: ${metrics.turnCount}`);

for (const [dim, avg] of Object.entries(metrics.dimensionAverages)) {
  console.log(`  ${dim}: ${avg.toFixed(1)}`);
}

// Reset for new conversation
session.reset();

Policy Engine

Enforce content quality policies with four modes: LOG_ONLY, BLOCK, REGENERATE, CUSTOM.

import { PolicyEngine, RAILBlockedError } from '@responsible-ai-labs/rail-score';

const policy = new PolicyEngine(client, {
  mode: "BLOCK",
  thresholds: { safety: 7.0, privacy: 7.0 },
});

try {
  const result = await policy.enforce("Content to check");
  console.log(result.evaluation.rail_score.score);
  console.log(result.passed);             // true/false
  console.log(result.failedDimensions);   // ["safety"] or []
} catch (error) {
  if (error instanceof RAILBlockedError) {
    console.log(`Blocked: ${error.message}`);
  }
}

// Runtime reconfiguration
policy.setMode("LOG_ONLY");
policy.setThresholds({ safety: 8.0 });

// CUSTOM mode — your own enforcement logic
const customPolicy = new PolicyEngine(client, {
  mode: "CUSTOM",
  thresholds: { safety: 7.0 },
  customCallback: async (content, evalResult) => {
    return `[Reviewed] ${content}`;
  },
});

Middleware

Wrap any async function with pre/post RAIL evaluation:

import { RAILMiddleware } from '@responsible-ai-labs/rail-score';

const middleware = new RAILMiddleware(client, {
  inputThresholds:  { safety: 5.0 },
  outputThresholds: { safety: 7.0, privacy: 7.0 },
  onInputEval:  (result) => console.log(`Input score: ${result.rail_score.score}`),
  onOutputEval: (result) => console.log(`Output score: ${result.rail_score.score}`),
});

const safeLLMCall = middleware.wrap(async (input) => {
  return await myLLM.generate(input);
});

const output = await safeLLMCall("User message");

← Compliance Providers & Reference →