ADOm8

Get Started: ADOm8 Pipeline Setup

Follow these steps to deploy the open-source stack in your own Azure environment using the automated Azure Pipeline. This is the preferred and fastest way to get started.

Automated PipelineEstimated time: 20-30 minAudience: DevOps / Engineering leads
Download Setup Checklist (.txt)Looking for the manual setup guide?

1. Prerequisites

5 min

Gather required information and create the necessary tokens before running the pipeline.

View detailed actions
  • You will need your Azure Subscription ID, ADO Organization URL, ADO Project Name, GitHub Organization, and GitHub Repository.
  • You will need an AI provider API key — at least one of: Anthropic (Claude) or OpenAI. The pipeline stores it securely in Key Vault and configures the Function App — the AI is not called during setup itself, only wired in so it’s ready for your first story. Google Gemini can be added post-setup in Function App configuration.
  • Create an Azure Service Connection in your ADO project (Project Settings → Service connections → New service connection). This is required — the pipeline uses it to authenticate with Azure and deploy the Resource Group, Key Vault, Storage Account, and Function App on your behalf. When prompted, select Azure Resource Manager → Identity type: App registration (automatic) → Credential: Workload identity federation — the modern, credential-free option. Set scope to Subscription, leave Resource Group blank, and check “Grant access permission to all pipelines”. For the connection name use something like adom8-azure-deploy — this becomes your AZURE_SERVICE_CONNECTION pipeline variable.
  • Create an ADO Onboarding PAT by following these steps:
    1. 1Click your avatar (top-right) then select User settings
    2. 2Click Personal access tokens
    3. 3Click + New Token
    4. 4Set Name: Adom8-Agent-Bot · Expiration: 30 days · Scopes: Custom defined
    5. 5Check: Work Items (Read & write), Code (Read & write), Build (Read & execute), Release (Read, write, & execute)
    6. 6Click “Show all scopes” and also check: Project and Team (Read, write, & manage), Service Connections (Read, query, & manage)
    7. 7Click Create. ⚠ Copy the token immediately — it will not be shown again. Paste it into the checklist scratch pad now.
  • Create a Fine-grained GitHub PAT. This is always created on your personal GitHub account — even if the repo is inside an organization.
    1. 1Go to your personal GitHub Settings
    2. 2Scroll to the bottom of the left sidebar and click Developer settings
    3. 3Click Fine-grained tokens → Generate new token
    4. 4Fill in Token name (e.g. adom8-pipeline), add a description, and set Expiration to 90 days (best practice — avoid “No expiration”).
    5. 5Resource owner: if the repo is in an organization, select the organization here (not your personal account). ⚠ The org owner must have enabled fine-grained PATs first — they can do this in Org Settings → Third-party Access → Personal access tokens → Allow fine-grained tokens.
    6. 6Under Repository access select Only select repositories and choose your target repo.
    7. 7Under Permissions grant Read & write to: Contents, Pull requests, Issues, Webhooks, Secrets. (Secrets is required so the pipeline can automatically write AZURE_FUNCTIONAPP_NAME, AZURE_FUNCTIONAPP_PUBLISH_PROFILE, and AZURE_STATIC_WEB_APPS_API_TOKEN to your repo — no manual secret setup needed.)
    8. 8Click Generate token. ⚠ Copy the token immediately — GitHub will never show it again. Paste it into the checklist scratch pad now.
  • Copy both tokens immediately after creation — you will not be able to see them again.
Prerequisites checklist and token creation

2. Import Pipeline

2 min

Fork the ADOm8 OSS repo, then create a new pipeline in ADO pointing at your fork.

View detailed actions
    1. 1Fork the ADOm8 OSS repo to your own GitHub account or org — fork it here. This gives you your own copy of the pipeline and Function App source that ADO will check out and build.
    2. 2In your ADO project go to Pipelines → New pipeline.
    3. 3On “Where is your code?” select GitHub (YAML). Do not choose the Enterprise sub-options — those are for ghe.com / self-hosted tenants.
    4. 4ADO redirects to GitHub to authorize. If the repo is inside an organization, click Grant next to the org — without this, org repos will not appear in the list. Missed it? Go to GitHub → Settings → Applications → Authorized OAuth Apps and grant from there.
    5. 5Select your forked adom8 repo from the list.
    6. 6On the Configure screen choose “Existing Azure Pipelines YAML file”, select branch main and path /adom8-onboarding-pipeline.yml.
    7. 7Click the dropdown arrow next to Run and choose Save — do not run yet. You need to add pipeline variables in Step 3 first.
Importing the YAML pipeline in Azure DevOps

3. Configure Variables

3 min

Set the required pipeline variables before running the setup.

View detailed actions
    1. 1Open your saved pipeline, click the Variables button (top-right of pipeline editor, before Run).
    2. 2

      Add each standard variable with its value:

      AZURE_SUBSCRIPTION_IDYour Azure subscription GUID — from Azure Portal → Subscriptions
      AZURE_DEVOPS_ORGYour ADO org name only, e.g. my-org (not the full URL)
      AZURE_DEVOPS_PROJECTUse spaces as-is, e.g. My Sample Project — not URL-encoded
      GITHUB_ORGYour GitHub org name or personal username
      GITHUB_REPORepo name only, e.g. my-project (no URL, no owner prefix)
      RESOURCE_GROUP_NAMESuggested: adom8-rg — will be created if it doesn't exist
      LOCATIONAzure region, e.g. eastus, westus2, eastus2
      AZURE_SERVICE_CONNECTIONThe service connection name you created, e.g. adom8-azure-deploy
      COPILOT_ENABLEDHighly recommended: set to true to enable GitHub Copilot Coding Agent delegation (default: true)
      GitHub Copilot (recommended): COPILOT_ENABLED=true turns on Copilot coding delegation and is strongly recommended for the best experience. Before enabling, make sure your GitHub organization/repository is eligible for Copilot, your team has Copilot access enabled, and your GITHUB_TOKEN has the required repo permissions (Contents, Pull requests, Issues, Webhooks, Secrets).
    3. 3

      Add each secret variable and check “Keep this value secret” for each one.

      ONBOARDING_PATThe ADO PAT you created in Step 1 (Adom8-Agent-Bot)
      CLAUDE_API_KEYAnthropic API key — set this OR OPENAI_API_KEY (at least one required; Claude preferred when both set)
      OPENAI_API_KEYOpenAI API key — set this OR CLAUDE_API_KEY (leave blank if using Anthropic)
      GITHUB_TOKENThe GitHub fine-grained PAT you created in Step 1 (adom8-pipeline) — must include Secrets (Read/Write) so the pipeline auto-sets deploy secrets in your repo
      AdoDashboardKeyA secure key you create to protect your dashboard (e.g. a long random string)
      Multi-provider tip: The pipeline sets up Claude as the default AI provider. Once onboarding is complete, you can enable OpenAI or Google Gemini by adding these to your Function App environment variables (Azure Portal → Function App → Configuration):
      • AI__ProviderKeys__OpenAI__ApiKey — OpenAI API key
      • AI__ProviderKeys__Google__ApiKey — Google Gemini API key
      You can also set per-agent model overrides (AI__AgentModels__StoryDesign, etc.) or pass OPENAI_API_KEY / GOOGLE_API_KEY env vars before running this pipeline to have them provisioned automatically.
    4. 4Click Save to close the variables panel. You are now ready to run the pipeline in Step 4.
Pipeline variables configuration for ADO-M8

4. Run Pipeline

10-15 min

Run the pipeline to automatically provision infrastructure, configure ADO, set up GitHub integrations, and deploy your free real-time ADOm8 Dashboard.

View detailed actions
  • Click 'Run' to start the pipeline.
  • Stage 1: Deploys Azure Infrastructure — Resource Group, Storage Account, Key Vault, Function App, and your Static Web App Dashboard.
  • Stage 2: Creates a dedicated ADOm8 Runtime PAT and stores it in Key Vault.
  • Stage 3: Stores all secrets securely in Key Vault and configures the Function App settings.
  • Stage 4: Customizes the ADO Process — creates inherited process, custom fields, states, and board rules.
  • Stage 5: Configures GitHub — registers webhook, creates .adom8 folder.
  • Stage 6: Creates an ADO Service Connection to GitHub.
  • Stage 7: Runs validation checks and outputs a summary.
  • If the pipeline fails, just fix the issue and re-run — every stage is idempotent. Resources that already exist are skipped, nothing gets duplicated.
The ADOm8 Real-time Dashboard deployed by the pipeline

5. Board Config & Verify

10 min

Create one AI Agent group on your User Story layout with 4 fields, show the active agent on board cards, then run a test story.

View detailed actions
  • The pipeline creates all custom fields and states automatically, but ADO does not allow adding fields to form layouts via API — this is a one-time manual step in Organization Settings. Only the core fields are needed to get started.
    1. 1Go to Organization Settings (bottom-left gear) → Boards → Process → find your inherited process (named after your project, e.g. Agile — My Sample Project) → click User Story → Layout.
    2. 2

      Click + New group and name it AI Agent. Then click + Add field to add all four fields:

      Autonomy LevelControls how far the agent runs autonomously (0–4)
      AI Minimum Review ScoreMinimum score (0–100) the Review agent must give before passing
      Current AI AgentWhich agent is currently active — shown on board cards
      AI Last AgentWhich agent last ran
    3. 3Go to your Boards view → click the gear icon ⚙ (top-right of the board) → Fields → add Current AI Agent to card display. This lets you see which agent is running directly on the board card.
    4. 4Test the end-to-end flow: Create a simple User Story, set Autonomy Level to 1 - Plan Only, and move the state to AI Agent. The agent should pick it up within seconds and transition it to Processing.
    5. 5Access your Dashboard: The pipeline automatically deployed a Static Web App in your resource group. Open it to view the live dashboard. Use the AdoDashboardKey you provided in the pipeline variables to log in. Azure generates a complex default URL for the Static Web App, but you can easily bookmark it or configure a custom domain in the Azure Portal under the Static Web App settings.
    Keeping your Dashboard Updated: The dashboard is yours to modify however you like! However, if you make modifications directly to your forked repository, you may lose the ability to easily sync your fork with the upstream repository. To get all the latest updates and bug fixes, we recommend keeping your fork synced and re-running the pipeline.
Post-setup board configuration — adding AI fields to User Story layout

6. Optional Fields

5 min

(Optional) Add per-agent model override fields and extended tracking fields to your User Story layout. These are not required for the core ADOm8 flow.

View detailed actions
  • The fields below exist in your ADO process already — the pipeline created them. This step just makes them visible on the User Story form. The tracking fields will remain blank until the agent output is extended to populate them (see project backlog).
  • Note: The pipeline creates all these fields automatically, but the tracking fields below are not yet wired up — they will remain blank until the agent output logic is extended. Adding them to the layout now means they’ll populate automatically once wired. See Backlog Item #3 in the project planning file.
    1. 1

      In your User Story Layout (same path as Step 5), add these to the AI Agent group. These fields let you tune model cost/quality globally or override per agent per story:

      AI Model TierBatch preset: economy / standard / pro — sets all agent models at once
      AI Planning ModelOverride the model for the Planning agent (e.g. gpt-4o)
      AI Coding ModelOverride the model for the Coding agent
      AI Testing ModelOverride the model for the Testing agent
      AI Review ModelOverride the model for the Review agent
      AI Documentation ModelOverride the model for the Documentation agent
      AI Coding ProviderOverride the provider for Coding agent (anthropic / openai / google)
    2. 2

      Add these extended tracking fields to the AI Agent group (or create a separate AI Tracking group if you prefer). The review score is posted as a work item comment today — these fields will populate automatically once the agent output is wired up (see backlog #3):

      AI Review ScoreScore from the Review agent (0–100) — currently surfaced in comments
      AI ModelModel used during the last run — informational
      AI Tokens UsedTotal token usage for this story
      AI CostEstimated cost in USD for this story
      AI Processing TimeTotal processing time in seconds
      AI ComplexityComplexity estimate from the Planning agent
      AI Files GeneratedNumber of files created/modified
      AI Tests GeneratedNumber of tests written
      AI PR NumberGitHub PR number if one was opened
      AI Critical IssuesCritical issues found during Review
      AI Deployment DecisionDeployment agent's go/no-go decision
    3. 3Optional — board card color rules: Board settings → Styles → add a rule for each agent value (e.g. Current AI Agent = Coding Agent) and pick a distinct color. Makes it easy to see at a glance which agent is actively working on a story.

7. Optional Kanban Setup

5 min

(Optional) Set up swimlanes and color-code cards based on the active AI Agent for better visibility.

View detailed actions
  • Note: This is a strictly optional suggested setup for your Azure Board. You can configure your Kanban board however you prefer, but this setup makes it incredibly easy to visually track the AI orchestrator as it moves stories through the pipeline.
    1. 1Add Fields to Cards: Go to Board Settings (gear icon) → Fields. Add Current AI Agent and Autonomy Level to your User Story cards. This lets you see exactly which agent is working on the story at a glance.
    2. 2Configure Card Styles: Go to Board Settings → Styles. Create a styling rule for each agent (e.g., Planning Agent, Coding Agent, Testing Agent) that changes the card color based on the Current AI Agent field. As the orchestrator hands off the story, the card will automatically change colors!
    3. 3Set up Swimlanes: Go to Board Settings → Swimlanes. Create a dedicated swimlane for AI Agents to separate automated work from human work. This keeps your board clean and organized.

Self-Host OSS

You control infrastructure, credentials, networking, and access policies in your Azure environment.

See How It WorksOpen OSS SetupSecurity Hardening