PSI Knowledge Base

Teams Notifications (PSI Notify Bot)

14 min readEdit this page

Teams Notifications (PSI Notify Bot)

The shared PSI alert channel. Any internal automation — scheduled tasks, CI/CD workflows, web apps, manual operator scripts — uses this bot identity to post Teams messages to a designated chat. One bot, one client, one source of documentation.

Quick Reference

ActionHow
Send an alert to an existing chatSend-NotifyMessage -ChatId <id> -BodyHtml <html> (module — when published). Until then, call send-teams-notification.ps1 patterns directly.
Bootstrap a new persistent alert chatscripts\Initialize-NotifyChat.ps1 -Topic <name> -MemberObjectIds <guids> -ChatSecretName psi-notify--chat-<purpose>
Add/remove a member to a persistent chatEdit chat membership in Teams (group-chat membership does not auto-sync with the Entra source group)
Rotate the client secretAzure Portal → App registrations → PSI.All Deploy Notifications → Certificates & secrets → New client secret. Update psi-notify--client-secret in ps-certificates-kv.

Bot identity (also used by Deploy ProApps): Azure Bot PSIDeployNotifyBot (RG PS-WEBAPPS, F0). App registration PSI.All Deploy Notifications (appId 5f54d8fc-fca1-4c78-9894-c267018efadb, tenant a83ae943-0a50-49cc-83c3-479b7a44b7fb). Teams app catalog id eb9901e7-e87c-437d-803a-b766ec007a16, published to org catalog.

The Teams-side app is being renamed from “PSI Deploy Bot” to “PSI Notify Bot” so it reads correctly for non-deploy consumers. Azure resource names stay as they are.

Chat patterns — pick one

PatternWhen to useExamples
Per-event group chatOne self-contained event with a scoped audience that won’t repeat. New chat per event.Deploy ProApps beta deploys — one chat per [Beta] AppName vX.X.X.X
Persistent group chatOngoing operational alerts with a stable audience. One chat, many messages over time.IT Critical Notifications (see below), future per-system alert chats

Don’t invent a third pattern. If you think you need one, talk through it before implementing.

Active persistent chats

ChatTopicMembership sourceKV secret holding the chat ID
IT Critical NotificationsPSI operational automation alertsSnapshot of DG - 170 - Operations Entra group, humans onlypsi-notify--chat-it-critical
PRGJSMES HelpOperator Get Help requests from psmes.progressivesurface.com (with in-chat resolve workflow via deep-link to /help/resolve/{id})Manual list: ADevereaux, DCooper, CHosakura (initial bootstrap 2026-05-13)psi-notify--chat-prgjsmes-help
CSM Board Managercsm-board Board Manager executive digests + Action-needed items (live since 2026-06-02). Becoming two-way via a dedicated interactive bot — see csm-board#41 / the consumer note belowManual list: ADevereaux (+ others as desired)psi-notify--chat-csm-board

When a new persistent chat is bootstrapped, add a row here.

Membership drift

Group-chat membership is a flat user list, not a live reflection of an Entra group. The bootstrap snapshots the source group at the moment the chat is created. When the source group changes:

  • Hire/role change inside Operations: add the person manually to the chat in Teams.
  • Departure: remove the person manually.

There is no reconciler. If drift becomes painful, the right escalation is a scheduled job that diffs current chat members against the source group and posts a “needs sync” alert — not a chat-per-membership-change pattern.

Secrets

All secrets live in ps-certificates-kv (RG PS-RG-01) following the psi-notify--* naming pattern:

Secret namePurpose
psi-notify--tenant-idAzure AD tenant (a83ae943-0a50-49cc-83c3-479b7a44b7fb)
psi-notify--client-idApp registration client ID (5f54d8fc-fca1-4c78-9894-c267018efadb)
psi-notify--client-secretApp registration client secret. Rotate every 2 years.
psi-notify--teams-app-catalog-idTeams app catalog ID (eb9901e7-e87c-437d-803a-b766ec007a16)
psi-notify--chat-<purpose>Chat ID for each persistent chat. Bootstrap script writes this.
psi-notify--acs-connection-stringAzure Communication Services connection string (voice/SMS). See Voice & SMS.
psi-notify--acs-endpointACS endpoint (https://psi-notify-acs.unitedstates.communication.azure.com)
psi-notify--acs-tollfree-numberACS toll-free number (+1 833-639-2948) used as caller ID / SMS sender

Consumers source these however they want:

Consumer shapeHow it fetches secrets
App Service@Microsoft.KeyVault(SecretUri=...) app settings, managed identity has KV read role
Scheduled task on PS-PROXYManaged identity → Get-AzKeyVaultSecret at task start, no on-disk secrets
GitHub Actions workflowOIDC federated credential → SP with KV read → fetch secrets in workflow step (preferred over GH Secrets so KV stays the single source of truth)
Ad-hoc operator scriptsaz keyvault secret show --vault-name ps-certificates-kv --name psi-notify--<x> --query value -o tsv

Legacy GH Secrets (PSI.All): GRAPH_TENANT_ID, GRAPH_CLIENT_ID, GRAPH_CLIENT_SECRET, TEAMS_BOT_CATALOG_ID still exist in PSI.All for the deploy workflow. These should migrate to the OIDC + KV pattern; until then KV holds the canonical values and GH Secrets are a mirror.

Graph API permissions (application)

PermissionWhy
Chat.CreateCreate group chats
Chat.ReadWrite.AllManage chat membership; required by Bot Framework to post into chats it didn’t create
User.Read.AllResolve SAM/UPN to AAD object IDs
TeamsAppInstallation.ReadWriteForChat.AllInstall the bot app into a newly created chat
TeamsAppInstallation.ReadWriteSelfForChat.AllSelf-install fallback
TeamsAppInstallation.ReadWriteSelfForUser.AllInstall bot into a single user’s personal scope (solo-recipient fallback). Added 2026-05-13 — admin consent still pending. Solo-tester DM fallback won’t work until consent is granted in Entra portal → App registrations → API permissions → “Grant admin consent for Progressive Surface Inc.”
AppCatalog.ReadWrite.AllManage the Teams app catalog entry (used for initial setup / re-upload only). Update operations require delegated auth — see “Updating the Teams manifest” below.
Teamwork.Migrate.AllGranted but unused (migration-mode channel posting)

Sending a message

Minimum surface

# Pseudo-code; the real module Send-NotifyMessage wraps this.
$graphToken = Get-OAuthToken -Scope "https://graph.microsoft.com/.default"        # not needed for steady-state send
$botToken   = Get-OAuthToken -Scope "https://api.botframework.com/.default"
 
$activity = @{
    type       = "message"
    text       = "<b>Alert</b><br>Something happened."
    textFormat = "xml"
} | ConvertTo-Json
 
$headers = @{ Authorization = "Bearer $botToken"; "Content-Type" = "application/json" }
Invoke-RestMethod `
    -Uri "https://smba.trafficmanager.net/amer/v3/conversations/$chatId/activities" `
    -Method Post -Headers $headers -Body $activity

The bot must already be installed in the chat. For chats this bot created, that’s automatic. For pre-existing chats, the bootstrap step POST /chats/{id}/installedApps is required (handled by Initialize-NotifyChat.ps1).

Message style

Plain HTML, no Adaptive Cards required for routine alerts. A sensible house style:

<b>{System}: {short subject}</b><br>
<br>
<b>What:</b> {one-line description}<br>
<b>When:</b> {timestamp}<br>
<b>Details:</b> {age, last error, etc.}<br>
<b>Log:</b> <code>{path or URL}</code>

Match the existing deploy-bot card style (see PSI.All/deploy/send-teams-notification.ps1) where it fits — consistency for recipients beats per-system creativity.

Bootstrapping a new persistent chat

scripts\Initialize-NotifyChat.ps1 (in C:\GIT\psi-notify-bot):

.\Initialize-NotifyChat.ps1 `
    -Topic "Some Alert Channel" `
    -MemberObjectIds @(
        'd7f24f06-eccc-4a3d-a095-def371873ea7',  # JTymes
        '3981fd99-e953-42ad-8813-b1b97601170c'   # BSummers
    ) `
    -ChatSecretName "psi-notify--chat-some-channel" `
    -BootstrapMessage "<b>This chat is now wired up for X alerts.</b>"

The script:

  1. Reads bot credentials from ps-certificates-kv.
  2. Acquires Graph + Bot Framework tokens.
  3. Validates ≥2 unique members (Graph requirement — see pitfalls).
  4. Creates the group chat, members bound by AAD object ID in [ordered]@{} blocks (see pitfalls).
  5. Installs the bot into the chat via POST /chats/{id}/installedApps.
  6. Sends the bootstrap message via Bot Framework so recipients have immediate context.
  7. Stores the chat ID in KV under the supplied secret name.
  8. Adds an entry to this page (manual — do it after running the script).

Resolving members from an Entra group:

az ad group member list --group "DG - 170 - Operations" `
    --query "[?upn!=null && !contains(upn,'.onmicrosoft.com') && !contains(upn,'test')].id" -o tsv

Adjust the filter to your group’s reality (exclude service accounts, test users, etc.).

Graph chat-creation pitfalls

Three non-obvious failure modes. All three produce the same misleading error: 'user@odata.bind' field is missing in the request. Always dump the exact POST body before the call (Write-Host $chatBody) so any future rejection is diagnosable from the log alone.

1. POST /chats members must use AAD object IDs, not UPNs

The user@odata.bind URL looks like:

https://graph.microsoft.com/v1.0/users('{id-or-upn}')

Graph’s docs say either works. In practice, the @ character inside a UPN (e.g. users('alice@progressivesurface.com')) is misparsed by Graph’s OData layer and the whole field is silently ignored. Resolve users to their AAD object GUID and bind by GUID.

2. Member hashtables must be [ordered]@{}

PowerShell’s plain @{} is unordered; ConvertTo-Json emits keys in hash-bucket order, often putting user@odata.bind before @odata.type. Graph’s OData parser requires the @odata.type discriminator first so it knows what shape to expect. Use [ordered]@{} on every member object AND on the chatBody itself.

3. chatType: "group" needs ≥2 unique AAD members

The bot does not count — it’s an installed app, not a conversationMember. When the recipient list dedupes to 1 unique person — common for solo-tested deploys — Graph rejects the call with the same misleading “user@odata.bind missing” error. The fallback is the bot’s personal 1:1 chat: install the bot into the user’s personal scope (POST /users/{upn}/teamwork/installedApps) and use the resulting user↔bot chat id. The user gets a DM instead of a group chat.

Don’t swallow errors

Don’t wrap notification steps in continue-on-error: true thinking it’s “best-effort.” It hides all three bugs above — the step reports success and the consumer thinks alerts are working, but no chat exists. If a notification is genuinely optional for a code path, skip the call with a real condition; don’t catch and ignore.

Consumers

ConsumerChat shapeNotes
Deploy ProAppsPer-event group chatOne chat per beta deploy; the original Bot Framework consumer
egnyte-stp-syncPersistent (IT Critical Notifications)Alerts on stuck files, token refresh failures, DFS access failures
PRGJSMES Get HelpPersistent (PRGJSMES Help)Operator-initiated; bot posts the original request and a follow-up “Resolved by X” card. App Service prgjsmes-prod pulls Support__* from psi-notify--* KV secrets via Key Vault references
csm-board Board Manager digestsPersistent (CSM Board Manager)First Python consumer — App Service csm-board on Linux can’t import PSI.Notify, so api/notify.py ports the token flow + Bot Framework send to Python (httpx). Pushes each groom-cycle executive digest as an Adaptive Card (Action-needed in its own container). Config-gated (CSM_TEAMS_NOTIFY), fail-safe, deduped. Secrets via Key Vault references resolved by the csm-board App Service MI. csm-board#24
csm-board Conversational (two-way)Persistent (CSM Board Manager)Dedicated interactive bot (NOT the shared Notify Bot — keeps it isNotificationOnly so other consumers can’t regress). Inbound POST /api/teams/messages validates a Bot Framework JWT per activity (audience = the dedicated bot app id, BF issuer, serviceUrl host allowlist + channel binding); messages route into the Board Manager’s .claude-inbox/, replies post back via the reply_to_teams tool. Phase B adds focus/snooze quick-reply card verbs; a dormant master is woken by resume (active-wake). Live since 2026-06-24 — dedicated bot csm-board-bot (App ID 2bb0664c-5ec8-4390-95af-1d08c12fb6a8), gated by CSM_TEAMS_INBOUND=1 (runbook). csm-board#41

When wiring a new consumer:

  1. Pick the chat pattern (per-event vs persistent).
  2. If persistent and the chat doesn’t exist yet, run Initialize-NotifyChat.ps1. If it does, look up the chat ID secret name on this page.
  3. In your consumer, fetch the four bot secrets + the chat ID from KV (or your runtime’s secrets source — see the table above).
  4. Call the Bot Framework activity endpoint with your HTML body.
  5. Add a row to the Consumers table here.

Updating the Teams manifest

The Teams app entry in the org catalog (PSI Notify Bot, catalog id eb9901e7-...) is updated by uploading a new app definition with a bumped version field. The manifest’s id must stay the same (the bot’s app reg id 5f54d8fc-...).

Process:

  1. Edit psi-notify-bot/teams-app/manifest.json — bump version, change names/descriptions/icons as needed. Don’t change id.
  2. Run psi-notify-bot/scripts/Publish-TeamsApp.ps1 from a workstation where the operator is signed in with Teams Admin or Global Admin (e.g. ADevereaux). The script:
    • Builds a zip from teams-app/
    • Connect-MgGraph -Scopes 'AppCatalog.ReadWrite.All' (interactive sign-in)
    • POST /appCatalogs/teamsApps/{catalogId}/appDefinitions?requiresReview=false with the zip body
  3. Future bot installs in chats show the new name/icons. Existing chats keep showing the version they were installed with.

Why delegated, not app-only. Microsoft Teams enforces an admin-role check on catalog updates that Graph permissions alone don’t satisfy. App-only calls fail with 403 "User not authorized" even with AppCatalog.ReadWrite.All admin-consented. The Microsoft-recommended path is to authenticate as a Teams Admin / Global Admin user.

Voice & SMS (Azure Communication Services)

For critical alerts that must wake someone up, Teams alone isn’t enough (Teams mobile honors phone Do Not Disturb). The PSI Notify Bot has a second delivery path built on Azure Communication Services (ACS): an outbound voice call (live now) and SMS (pending toll-free verification). Added 2026-06-17 for the criticalalerts@progressivesurface.com wake-up requirement.

ACS resources

ResourceNameGroupNotes
Communication Servicepsi-notify-acsPS-RG-01US data residency; host psi-notify-acs.unitedstates.communication.azure.com
Toll-free number+1 833-639-2948(on the ACS resource)SMS in/out + Calling in/out, $2/mo
AI Services (TTS)adeve-midqp8v8-eastus2rg-gpt5-9353ACS managed identity (9f59550a-…) holds Cognitive Services User here

Voice tier — how it works

Hosted in the psi-notify-listener Function App (PS-WEBAPPS, .NET 8 isolated, listener/VoiceFunction.cs):

EndpointAuthPurpose
POST /api/voice/callfunction keyInitiate a call. Body: { "to": "+1...", "message": "..." }
POST /api/voice/callbackanonymousACS Call Automation event webhook

Flow: CreateCall (caller ID = our toll-free, CognitiveServicesEndpoint set) → on CallConnected, speak the alert via TTS (SSML, 2.5s leading pause, message spoken twice) and recognize one DTMF digit → press 1 acknowledges (logged; the hook future escalation hangs off) → re-prompts once on no input (30s window) → hangs up. Voice needs no carrier registration of any kind.

SMS tier — pending verification

ACS enforces US A2P rules: an unverified toll-free number is blocked (not just filtered) for US SMS. So SMS is gated on toll-free verification — a free, single portal form (ACS resource → Regulatory Documents → Add), ~5–6 week carrier review. Application content + the 1-rejection-cause opt-in guidance are drafted in psi-notify-bot/docs/acs-tollfree-verification.md. Toll-free verification is a separate, free regime from 10DLC — never apply 10DLC to a toll-free number. Cost once live: ~$3/mo (number + low volume). The public opt-in page (required for the verification’s opt-in URL) is drafted at psi-notify-bot/opt-in-page/index.html, to be hosted on progressivesurface.com.

⚠️ Gotcha: Flex Consumption needs a Microsoft.App subnet for Key Vault

psi-notify-listener is a Flex Consumption function app (FC1). ps-certificates-kv is firewalled (defaultAction: Deny), so the app’s KV reads fail with ForbiddenByFirewall unless it egresses through a VNet subnet the vault trusts.

The catch: Flex Consumption VNet integration requires a subnet delegated to Microsoft.App/environments — not Microsoft.Web/serverFarms like the classic ps-webapps subnet the Basic/Standard apps use. A Flex app cannot attach to ps-webapps (you’ll get a misleading SubnetMissingRequiredDelegation error).

The fix (applied 2026-06-17):

  1. Created subnet ps-flexfunc (10.160.151.0/26) in ps-vnmain, delegated to Microsoft.App/environments, with the Microsoft.KeyVault service endpoint.
  2. az functionapp vnet-integration add the Flex app into ps-flexfunc.
  3. Added ps-flexfunc to the vault’s networkAcls.virtualNetworkRules.

This routes only KV traffic through the service endpoint (no route-all), so ACS/GitHub/Bot Framework egress stays direct. This also repaired the listener’s chat-command KV reads, which had the same latent problem.

The same applies per firewalled service. Reaching the alerts SQL DB initially failed with TCP error 35 even after the KV fix: privatelink.database.windows.net is linked to ps-vnmain, so SQL resolves to its private-endpoint IP, and that traffic wasn’t routed through the VNet (no route-all, and there was no SQL service endpoint). Fix: add the Microsoft.Sql service endpoint to ps-flexfunc and a SQL vnet-rule for it (az sql server vnet-rule create). Rule of thumb for this Flex app: each firewalled Azure service it must reach needs its own service endpoint on ps-flexfunc + the matching network rule on that service.

⚠️ Gotcha: iOS unknown-caller screening eats the prompt

iOS “Silence Unknown Callers” / Call Screening will answer with Apple’s screening assistant — ACS sees the call “connected” and plays the alert to the screener, so the human never hears it. Recipients must save +1 833-639-2948 as a contact (“PSI Critical Alerts”); pair with Emergency Bypass so it overrides Do Not Disturb (this is the real “wake me up” lever). CNAM/branded calling is a complement, not a substitute.

Interactive voice roadmap

  • Tier 1 (live): DTMF acknowledgement (press 1). Escalation (“no ack in N min → call next person”) drops onto the ack hook.
  • Tier 2 (future): conversational agent via ACS bidirectional audio streaming ↔ Azure OpenAI Realtime (gpt-4o-realtime) on the foundry resource — “ask the AI about the alert.” Whisper is batch transcription only; not for low-latency turn-taking.

Architecture diagram

            ┌──────────────────────────────┐
            │  ps-certificates-kv (PS-RG-01) │
            │  psi-notify--tenant-id         │
            │  psi-notify--client-id         │
            │  psi-notify--client-secret     │
            │  psi-notify--teams-app-catalog-id
            │  psi-notify--chat-<purpose>    │
            └──────────────┬───────────────┘
                           │ (secret fetch)
            ┌──────────────┴──────────────────┐
            │                                 │
   ┌────────▼────────┐               ┌────────▼────────┐
   │ PSI.All deploy  │               │ Operational     │
   │ workflow (GHA)  │               │ automations     │
   │ (per-beta chat) │               │ (persistent chat)
   └────────┬────────┘               └────────┬────────┘
            │                                 │
            └──────────────┬──────────────────┘
                           ▼
        ┌────────────────────────────────────────┐
        │ Microsoft Graph + Bot Framework        │
        │  - POST /chats (Graph, app-only)       │
        │  - POST /chats/{id}/installedApps      │
        │  - POST /v3/conversations/{id}/activities (Bot Framework)
        └────────────────┬───────────────────────┘
                         ▼
        ┌────────────────────────────────────────┐
        │ Teams: "PSI Notify Bot" group chats    │
        └────────────────────────────────────────┘

Operational notes

Why one bot, not many. Every additional Teams bot is a new app registration to rotate secrets on, a new Teams catalog entry to maintain, and a new UI surface that recipients have to recognize. Single identity is the sustainable shape.

Why group chat instead of Team channel. Group chats are more visible — they pop up in recipients’ chat list and the unread badge is the standard Teams alerting affordance. Channel posts are quieter and easy to miss. Trade-off: group-chat membership is a flat list with no live group sync (see “Membership drift” above). Acceptable for operations where membership changes are rare.

Why not Workflow webhooks. Webhooks would work for fire-and-forget posting but lose: sender identity (shows up as “Workflow” not the bot), the ability to @mention users, and consistency with the deploy-bot pattern. Reserved for cases where bot identity is genuinely irrelevant.

See Also

Last updated: 2026-05-13 — extracted from deploy-proapps.md when the bot became a shared resource. IT Critical Notifications chat bootstrapped same day.

Graph View

Backlinks