RAIL Score MCP quickstart: score content and enforce a policy

The fastest way to understand the RAIL Score MCP server is to score one piece of text and watch a policy turn that score into a decision. This post does exactly that, in about thirty lines, against the live server.

There is no SDK to adopt and nothing to host. You point an MCP client at one URL and call tools:

https://mcp.responsibleailabs.ai/mcp

It speaks Streamable HTTP, so any MCP client works. Here we use the official Python mcp SDK.

Setup

python -m venv .venv && source .venv/bin/activate
pip install "mcp>=1.13"
export RAIL_API_KEY=rail_your_key   # from https://responsibleailabs.ai/dashboard

A small helper opens an authenticated session and calls a tool, returning its structured result:

import contextlib, os
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client

@contextlib.asynccontextmanager
async def rail_session():
    headers = {"Authorization": f"Bearer {os.environ['RAIL_API_KEY']}"}
    async with streamablehttp_client("https://mcp.responsibleailabs.ai/mcp",
                                     headers=headers) as (read, write, _):
        async with ClientSession(read, write) as session:
            await session.initialize()
            yield session

async def call(session, tool, **arguments):
    result = await session.call_tool(tool, arguments)
    if result.isError:
        raise RuntimeError(result.content[0].text)
    return result.structuredContent or {}

Score a piece of content

rail_evaluate rates content across eight dimensions of responsible AI: fairness, safety, reliability, transparency, privacy, accountability, inclusivity, and user impact. Use mode="basic" for fast gating (1 credit, under a second) and mode="deep" when you also want explanations and suggestions.

CLEAN = "Your order #4471 shipped today and should arrive within three business days."

ev = await call(session, "rail_evaluate", content=CLEAN, mode="basic")
scores = ev["result"]["dimension_scores"]
for dim, v in scores.items():
    print(f"{dim:15} {v['score']}")

Output:

accountability  6.1
fairness        7.7
inclusivity     6.8
privacy         8.0
reliability     7.2
safety          10.0
transparency    6.2
user_impact     7.6

Each dimension scores 0 to 10. The per-dimension scores live under result.dimension_scores, and each entry carries a score and a confidence.

Turn scores into a decision with a policy

Scores are useful, but most apps want a yes or no. Pass a policy and the server returns a decision. A rule fires when its dimension scores below the threshold, and the overall action is the most severe fired action: block beats flag beats warn beats allow.

POLICY = {"rules": [
    {"dimension": "safety", "threshold": 7.0, "action": "block"},
    {"dimension": "fairness", "threshold": 6.0, "action": "flag"},
]}

RISKY = "Our refund policy is the best and everyone else is a scam. Just trust us."

ev = await call(session, "rail_evaluate", content=RISKY, mode="basic", policy=POLICY)
outcome = ev["result"]["policy_outcome"]
print("action:", outcome["action"], "blocked:", outcome["blocked"])
for r in outcome["triggered_rules"]:
    print(f"fired: {r['dimension']} {r['score']} < {r['threshold']} -> {r['action']}")

Output:

accountability  4.0
fairness        3.9
inclusivity     3.4
privacy         2.0
reliability     4.6
safety          5.0
transparency    4.7
user_impact     4.2

policy action=block blocked=True
  fired: safety scored 5.0 < 7.0 -> block
  fired: fairness scored 3.9 < 6.0 -> flag

The risky text scores low across the board. Safety lands at 5.0, under the 7.0 block threshold, so the overall action is block. Both fired rules come back in triggered_rules with the exact score and threshold, which is what you log against the decision.

Where the outcome lives

One detail worth knowing up front: the per-rule outcome with action, triggeredrules, and blocked is at result.policyoutcome. There is also a top-level policyoutcome, which is the engine's overall score against its own threshold, a different and coarser view. Read result.policyoutcome for the rule-based decision.

If the API key's application has a dashboard policy enforced, that takes precedence over the policy you pass, and the outcome's source field says application instead of request. Either way the shape you read is the same.

Next steps

That is the whole loop: connect, score, decide. From here you can wrap a tool call with railevaluatetoolcall, screen untrusted input with raildetectinjection, or scan tool output with railscantoolresult. The full runnable script is quickstartscorepolicy.py. Get a key from the dashboard and run it against the live server.