{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# MuniSpot API Data-Quality Audit\n\n> **Open in Colab:** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/OWNER/REPO/blob/main/munispot-data-quality-audit.ipynb)\n> \n> _Replace `OWNER/REPO` with the public quickstart repo once chosen. Until then, upload this file to Colab via **File → Upload notebook**._\n\nA structured side-by-side comparison of MuniSpot against your current municipal data provider, on the entities **you** care about. It reports four things:\n\n1. **Coverage**: who has data for each entity+year\n2. **Freshness**: the latest fiscal year each provider has\n3. **Line-item match rate**: agreement on income-statement line items\n4. **Score correlation**: MuniScore vs your provider's score\n\nThe incumbent side runs through adapter functions you fill in (Section 0). Until you do, MuniSpot columns populate and incumbent columns show as missing, so the notebook is still useful on day one.\n\n_Requires a Pro or Enterprise key for the financials section._"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Setup"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "%pip install --quiet requests pandas"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "import getpass\nimport requests\nimport pandas as pd\n\nBASE_URL = \"https://www.munispot.com\"\nAPI_KEY = getpass.getpass(\"Paste your MuniSpot API key (hidden): \").strip()\n\nsession = requests.Session()\nsession.headers.update({\"Authorization\": f\"Bearer {API_KEY}\"})\n\ndef ms_get(path, **params):\n    # Return (status_code, json_or_None) for a MuniSpot API call.\n    r = session.get(f\"{BASE_URL}{path}\", params=params, timeout=30)\n    try:\n        return r.status_code, r.json()\n    except ValueError:\n        return r.status_code, None"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 0. Configure your evaluation\n\nSet the entities and fiscal year, then implement the incumbent adapters against your current vendor. Each adapter returns `None` when your provider has no data for that input."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# The EINs you actually care about: your portfolio or coverage universe.\nEINS = [\"366005820\", \"746002070\", \"746001164\"]\nFISCAL_YEAR = 2023\n\n# Fiscal years to probe when measuring freshness (newest first).\nFY_PROBE = [2024, 2023, 2022, 2021, 2020]\n\n# 1% relative tolerance for treating two dollar values as the same line.\nMATCH_TOL = 0.01"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# ---- Incumbent provider adapters: implement against YOUR current vendor ----\n# Return None whenever your provider has no data for the given input.\n\ndef incumbent_has_entity(ein, fiscal_year):\n    # True / False / None: does your provider cover this ein+FY?\n    return None  # TODO\n\ndef incumbent_latest_fy(ein):\n    # Latest fiscal year your provider has for this ein, or None.\n    return None  # TODO\n\ndef incumbent_line_items(ein, fiscal_year):\n    # dict {line_label: value} of income-statement lines, or None.\n    return None  # TODO\n\ndef incumbent_score(ein, fiscal_year):\n    # Your provider's numeric credit score for this ein+FY, or None.\n    return None  # TODO"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 1. Coverage\n\nDoes each provider return data for every entity+year you asked for?"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "rows = []\nfor ein in EINS:\n    status, body = ms_get(f\"/api/v1/entities/{ein}\", fiscal_year=FISCAL_YEAR)\n    ms_has = status == 200 and bool(body and body.get(\"data\"))\n    name = body[\"data\"].get(\"name\") if ms_has else None\n    rows.append({\n        \"ein\": ein,\n        \"name\": name,\n        \"munispot\": ms_has,\n        \"incumbent\": incumbent_has_entity(ein, FISCAL_YEAR),\n    })\ncoverage = pd.DataFrame(rows)\nprint(f\"MuniSpot coverage: {coverage['munispot'].sum()}/{len(coverage)}\")\ncoverage"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 2. Freshness\n\nLatest fiscal year each provider can serve for each entity. We treat a year as present when a non-null overall score exists."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "def ms_latest_fy(ein):\n    for fy in FY_PROBE:\n        status, body = ms_get(f\"/api/v1/entities/{ein}/scores\", fiscal_year=fy)\n        if status == 200 and body and body.get(\"data\", {}).get(\"overall_score\") is not None:\n            return fy\n    return None\n\nfresh = pd.DataFrame([\n    {\n        \"ein\": ein,\n        \"munispot_latest_fy\": ms_latest_fy(ein),\n        \"incumbent_latest_fy\": incumbent_latest_fy(ein),\n    }\n    for ein in EINS\n])\nfresh"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 3. Line-item match rate\n\nPull MuniSpot's General-Fund income-statement lines and compare them to your provider's values for the same entity+year. `match_rate` is the share of **shared** line labels whose dollar values agree within `MATCH_TOL`. Treat the ACFR PDF as ground truth and spot-check a few entities by hand."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "def ms_line_items(ein, fiscal_year):\n    status, body = ms_get(\n        f\"/api/v1/entities/{ein}/financials\",\n        fiscal_year=fiscal_year, statement_type=\"income_statement\",\n    )\n    if status != 200 or not body:\n        return None\n    out = {}\n    for it in body[\"data\"][\"line_items\"]:\n        if it.get(\"column_index\") == 1 and it.get(\"value\") is not None:  # General Fund\n            out[it[\"class_1\"]] = out.get(it[\"class_1\"], 0) + float(it[\"value\"])\n    return out\n\nrows = []\nfor ein in EINS:\n    ms = ms_line_items(ein, FISCAL_YEAR)\n    inc = incumbent_line_items(ein, FISCAL_YEAR)\n    if not ms:\n        rows.append({\"ein\": ein, \"munispot_lines\": 0, \"incumbent_lines\": None,\n                     \"shared_labels\": None, \"match_rate\": None})\n        continue\n    if not inc:\n        rows.append({\"ein\": ein, \"munispot_lines\": len(ms), \"incumbent_lines\": None,\n                     \"shared_labels\": None, \"match_rate\": None})\n        continue\n    shared = set(ms) & set(inc)\n    matched = sum(1 for k in shared if abs(ms[k] - inc[k]) <= MATCH_TOL * max(abs(inc[k]), 1))\n    rows.append({\n        \"ein\": ein, \"munispot_lines\": len(ms), \"incumbent_lines\": len(inc),\n        \"shared_labels\": len(shared),\n        \"match_rate\": round(matched / len(shared), 3) if shared else None,\n    })\npd.DataFrame(rows)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 4. Score correlation\n\nMuniScore vs your provider's score across the entities where both exist. Needs at least two overlapping entities for a meaningful number."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "pairs = []\nfor ein in EINS:\n    status, body = ms_get(f\"/api/v1/entities/{ein}/scores\", fiscal_year=FISCAL_YEAR)\n    ms_score = body[\"data\"][\"overall_score\"] if status == 200 and body else None\n    pairs.append({\n        \"ein\": ein,\n        \"munispot_score\": ms_score,\n        \"incumbent_score\": incumbent_score(ein, FISCAL_YEAR),\n    })\nscores_df = pd.DataFrame(pairs)\nvalid = scores_df.dropna(subset=[\"munispot_score\", \"incumbent_score\"])\nif len(valid) >= 2:\n    print(\"Pearson correlation:\", round(valid[\"munispot_score\"].corr(valid[\"incumbent_score\"]), 3))\nelse:\n    print(\"Implement incumbent_score() for >= 2 entities to compute correlation.\")\nscores_df"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## How to read the results\n\n- **Coverage gaps** on the incumbent side are MuniSpot's clearest win; gaps on the MuniSpot side are worth flagging to your account contact.\n- **Freshness**: a newer `latest_fy` means less lag between an ACFR being published and the data being available.\n- **Match rate** near 1.0 on shared labels means the two providers agree on the numbers; low rates are the entities to spot-check against the source ACFR.\n- **Correlation** measures whether the two scoring methodologies rank-order credit similarly, not whether they use the same scale.\n\nQuestions, or want the time-bounded enterprise key for this evaluation? https://www.munispot.com/api-docs (Sandbox & Evaluation)."
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.x"
  },
  "colab": {
   "provenance": []
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}