Mark HollandSenior AI Solutions Engineer

AI Use Case Submission Platform

ProductionBuilt at work for a national healthcare data company

Employees send in ideas for AI tools. This app screens each one for patient data first, turns the ideas it clears into scored, ranked cases before the review meeting, then tracks the approved ones from discovery to launch.

The dashboard: tiles counting ideas by outcome and a ranked queue of demo use cases with their scores.

What it does

  • VerifiedA five-step intake form with 45 fields and a live completeness meter.
  • VerifiedIdeas that touch patient or personal data are held for compliance review. They are never scored and never sent to a model.
  • VerifiedEvery other idea is rated on each rubric dimension with a one-sentence reason and a confidence value, then ranked. A person makes every decision.
  • VerifiedApproved ideas move through discovery, a build brief, a build kickoff that drafts a PRD and tickets for a person to approve, and delivery tracking, across 16 pipeline statuses.
  • VerifiedThe app drafts a plain-language monthly update on what shipped and what it saves, for a person to post.

From the app's own materials. Facts marked Verified were checked against the code or a working copy of the app.

Who it is for

Employees with an idea for an AI tool, the AI team that reviews ideas every two weeks, and the leaders who decide what gets built.

How it works

AI Use Case Submission Platform, step by step
  1. Ask

    A five-step form collects the problem, the current process, the value, the data, and the urgency, with a live completeness meter.

  2. Screen

    Ideas that touch patient or personal data are held for compliance review. They are never scored and never sent to a model.

  3. Score

    Every other idea is scored against a weighted rubric, with a reason for each rating, strengths, concerns, a value estimate, and a check for similar past ideas.

  4. Review

    The team opens a ranked queue. Any reviewer can change an outcome and leave a note, and the change is logged with their name.

  5. Decide

    A leader approves, defers, or declines, and the app prepares a message to the submitter for each. Ideas for customer-facing products get a one-page brief and go to the product team.

  6. Discover

    An eight-section questionnaire records stakeholders, data, tools, and a go or no-go call. A 12-week clock starts at the first discovery meeting.

  7. Specify

    A go produces a build brief: tier, scope in and out, success criteria, and a build plan. Starting the build drafts a PRD, scaffold notes, and tickets for a person to approve.

  8. Report

    Shipped tools show hours saved against what they cost to build. The same data feeds leadership reports and a plain-language monthly update.

The real screens

Captured from the first version of the app with its synthetic demo data. The current version scores with the team's own formula and adds the decision, discovery, and build stages described below.

The intake form on its first step, with a five-step progress bar and a completeness meter.
Anyone can propose an AI idea through a five-step form that shows how complete it is as they fill it in. Shown with demo data.
The dashboard: tiles counting ideas by outcome and a ranked queue of demo use cases with their scores.
Each cleared idea is scored against a weighted rubric and ranked, so the review meeting starts with a sorted list. Shown with demo data.
A workflow board with columns for new, in review, decided, and on hold, holding demo idea cards.
The team moves each idea through review, decision, hold, and done. Shown with demo data.
Case briefing mode showing one demo case with its dimension scores, the reason for each, and decision buttons.
Case briefing: every case shows its scores and the reason behind each one, and a person records the final call. Shown with demo data.
The cycles page listing frozen two-week snapshots with their size, average score, and outcome mix.
Each two-week cycle is frozen as a snapshot, so the team can see how the queue changed over time. Shown with demo data.
The impact page listing shipped demo tools with hours reclaimed, build cost, and payback.
After a tool ships, the app tracks hours saved against what it cost to build. Shown with demo data.

Before any scoring

The compliance gate

The starting assumption is that any free-text field could contain patient data, even when it should not. So the gate runs first, before the rubric and before any model call.

  • Held for compliance review

    If the regulated-data answer or the data-sensitivity fields signal patient or personal data, the idea is set to Hold for compliance review. It gets no score and is never sent to a model.

  • A blank is not a pass

    A blank regulated-data answer counts as a soft hold, so a skipped question cannot slip an idea through.

  • One signal for every gate

    The scoring hold and the build sign-off read the same patient-data signal, and once it is confirmed it stays set. An earlier version read two different fields; a code review caught it and it was fixed.

  • Redact before anything leaves

    A redactor masks names, emails, birth dates, and record-number-like tokens before any model prompt, before the build kickoff pack leaves the app, and before logs are written. The code describes it as a heuristic screen, not a guarantee.

  • Sign-off before build

    An idea that touches patient data needs a governance sign-off from an admin before it can enter the build stage.

  • Two audit logs

    Scores, overrides, weight changes, role changes, sign-ins, status changes, and handoffs are written to append-only logs.

How ideas are scored

The rubric

Each idea is rated 1 to 5 on every rubric dimension, and the weighted sum sets its place in the queue. Value carries half the weight. The first version used a rubric I designed; the current version implements the team's own scoring method.

  • A reason for every rating

    Each rating comes with a one-sentence reason and a confidence value, plus a summary, strengths, concerns, and a value estimate.

  • No invented value

    The value estimate uses the hours or revenue the submitter entered. If those are missing, it says the value needs a conversation to quantify instead of guessing a number.

  • Weights live in one file

    Weights and thresholds live in a configuration file. The app refuses to load a rubric whose weights do not add up to 100 percent, and a test checks it.

  • Edited by the team, logged

    An admin can change weights and thresholds from a page. Every save bumps the rubric version and writes an audit entry.

  • The model sees the live rubric

    The scoring prompt is built from the current rubric. If the model returns the wrong dimensions or unreadable values, the app falls back to its rule engine and flags the result.

  • The app does the math

    The model only rates the dimensions. The app computes the weighted score and the outcome, so the ranking always follows the published rules.

How it runs

Two engines, one answer

Scoring runs either on a Claude model through Amazon Bedrock or on a deterministic rule engine. Both return results in the same shape, so every screen, report, and export works the same way whichever one is running.

  • Rule engine

    Makes no network calls and gives the same answer every time, so demos and tests are reproducible.

  • Model path

    A Claude model on Amazon Bedrock rates the dimensions after the submission passes through the redactor. Each result records which provider and model produced it.

  • Fallback, flagged

    If a model call fails or returns something unusable, the rule engine answers and the case is marked as a fallback.

  • Swappable provider

    One function wraps every model call, with a timeout and a small number of retries on transient errors. Moving to another provider, such as Azure OpenAI, is a configuration change.

A tour

The pages

  • Dashboard

    Tiles by outcome and a ranked queue with search, filters, bulk actions, and export to CSV, PDF, and PowerPoint.

  • Submit

    The five-step intake form with its completeness meter.

  • Case detail

    The submission, the score breakdown with reasons, strengths and concerns, the outcome override, the stage tracker with sign-offs, and an activity timeline.

  • Case briefing

    A full-screen meeting mode that steps through the ranked cases with the arrow keys and records each decision in place.

  • Compare

    Two ideas side by side, with the difference on each rubric dimension.

  • Board and pipeline

    A drag-and-drop board and a pipeline view that follow each idea from submitted to shipped.

  • Cycles

    Each two-week cycle frozen as a snapshot, with its size, average score, and mix of outcomes.

  • Roadmap

    Ranked ideas fill each cycle's development budget, and ideas that share a data dependency are grouped so the work is built once.

  • Reports

    Analytics tabs, a drill-down of matching cases, and reports for each audience, exported to PDF, PowerPoint, and Excel.

  • Impact

    Shipped tools with hours reclaimed, build cost, and payback.

  • Ask AI

    A chat box on every page that answers questions about the queue, with a rule-based fallback when no model is connected.

The stack

How it is built

  • Server-rendered

    Python, FastAPI, and Pydantic models, with pages rendered on the server from Jinja2 templates. There is no separate front-end app.

  • Data

    SQLite for local work, demos, and tests; PostgreSQL with migrations when a database address is set. All data access goes through one store module.

  • Similar ideas

    Local text similarity, with the title weighted most, flags ideas that resemble past ones. It is advisory and never merges cases.

  • Exports

    PDF, PowerPoint, Excel, Word, and CSV, built in the app.

  • Workflow descriptions

    Arazzo documents describe the triage and build kickoff pipelines on top of the app's OpenAPI description.

  • One container

    The app ships as one Docker image that runs as a non-root user with a health check.

Why it works this way

Design decisions

  • The AI suggests, people decide

    Every score is a starting point. The override path is never removed, and every override is logged with a name.

  • Score before anyone meets

    Cleared ideas are scored when they arrive, so the meeting starts from a ranked, reasoned list and discovery starts from the submitter's own answers.

  • No launch date until it is earned

    The build brief will not set a launch date until the idea is viable, engineering is secured, and an owner is named. Its out-of-scope list can never be empty.

  • Nothing goes out without a click

    Build tickets stay drafts, scaffold notes are never applied to a code repository, and the monthly update is copied and posted by a person.

  • Pluggable edges

    Data sources, the model provider, and ticket systems each sit behind one interface, so adding a source means adding one class.

The decision that matters

Compliance is a gate, not a score

The first version counted risk as one small part of the score. The current version takes it out of the score entirely and stops any idea with patient or personal data before it reaches a model or the ranked list.

Built with

  • Python
  • FastAPI
  • Jinja2
  • PostgreSQL
  • Claude on Amazon Bedrock
  • Docker