# Platform Improvement Plan: IETF Draft Analyzer → Standards Intelligence Platform

*Generated 2026-03-07 — Based on full codebase audit and architectural analysis*

---

## Current State Summary

| Dimension | What Exists | Assessment |
|-----------|-------------|------------|
| **Data** | 434 drafts, 557 authors, 419 ideas, 11 gaps, 4231 refs to 694 RFCs | Strong foundation |
| **CLI** | 20+ commands (fetch, analyze, embed, ideas, gaps, report, viz, wg, etc.) | Feature-rich |
| **Web UI** | 16 Flask pages with D3.js/Plotly visualizations | Good but disconnected |
| **Pipeline** | Observatory class with multi-source support | Built but manual |
| **Multi-SDO** | IETF complete, W3C fetcher written but unused | Partially built |
| **Tests** | Zero tests | Critical gap |
| **Deployment** | Manual Python setup | No containerization |

## The Transformation

**From**: A powerful CLI analysis tool that an expert runs manually
**To**: A living intelligence platform that monitors, alerts, and answers questions

---

## Phase 1: Quick Wins (All Parallelizable)

### 1.1 Global Search [S: 3-5h]
Add a unified search bar to the web UI that queries across drafts (FTS5), ideas, authors, and gaps simultaneously. Currently search is isolated to the drafts page.

- **Files**: `src/webui/app.py` (new `/search` route), `src/webui/data.py` (new `global_search()`), `src/webui/templates/base.html` (search in sidebar), new `search_results.html`
- **Why first**: Most common user action on any data platform

### 1.2 REST API [S: 2-3h]
Expose all existing `data.py` functions as JSON endpoints. Only 3 API endpoints exist today — extend to all 15+ data views.

- **Files**: `src/webui/app.py` (add `/api/ideas`, `/api/gaps`, `/api/ratings`, `/api/timeline`, `/api/landscape`, `/api/similarity`, `/api/drafts/<name>`, etc.)
- **Why**: Enables programmatic access, third-party tools, decouples data from presentation

### 1.3 Export (CSV/JSON) [S: 3-4h]
Add export buttons to web UI pages + `ietf export` CLI command.

- **Files**: `src/ietf_analyzer/cli.py` (new `export` command), API endpoints get `?format=csv` support
- **Depends on**: 1.2

### 1.4 Annotation System [M: 4-6h]
Add private notes and custom tags per draft, persisted in DB.

- **Files**: `src/ietf_analyzer/db.py` (new `annotations` table), `src/webui/app.py` (POST endpoint), `src/webui/templates/draft_detail.html` (inline edit), `src/ietf_analyzer/cli.py` (new `annotate` command)
- **Why**: Analysts need to layer their own context onto the data

### 1.5 Test Suite Foundation [M: 6-8h]
Create pytest infrastructure with ~30 tests covering DB layer, models, and web data functions.

- **Files**: new `tests/conftest.py`, `tests/test_db.py`, `tests/test_models.py`, `tests/test_data.py`, update `pyproject.toml`
- **Why**: Zero tests today. Every future change benefits from a safety net

---

## Phase 2: Core Platform

### 2.1 Semantic Search / "Ask" [M: 1-2 days] — THE SIGNATURE FEATURE
Natural language queries: "Which drafts address agent authentication?" → synthesized answer with citations.

- Embed the query via Ollama, compute cosine similarity against all 434 draft embeddings
- Merge with FTS5 keyword results using reciprocal rank fusion
- Optionally synthesize answer via Claude with top-K context
- **Files**: new `src/ietf_analyzer/search.py`, `src/ietf_analyzer/cli.py` (`ietf ask`), `src/webui/app.py` (`/ask` route), new template

### 2.2 Competitive Landscape Mapping [M: 1-2 days]
Auto-detect and compare competing proposals in the same problem space.

- Group drafts by similarity clusters, enrich with rating comparisons and WG adoption status
- Show head-to-head comparisons: where they agree, where they diverge
- **Files**: new `src/ietf_analyzer/competition.py`, `src/webui/app.py` (`/competition` route), new template

### 2.3 Standards Readiness Scoring [M: 1 day]
Composite 0-100 "readiness" score: WG adoption, revision count, reference density, cited-by count, author track record.

- **Files**: new `src/ietf_analyzer/readiness.py`, update `src/webui/templates/draft_detail.html` (gauge chart), update drafts listing

### 2.4 Scheduled Updates + Pipeline Health [M: 1 day]
Cron-based auto-fetch using existing Observatory, plus monitoring dashboard.

- **Files**: new `scripts/scheduled-update.sh`, enhance `src/webui/templates/monitor.html` (stage breakdown, cost tracking, failure log)

### 2.5 Comparison View [M: 1 day]
Side-by-side comparison of 2+ drafts with rating radar overlay, shared/unique ideas, shared/unique references.

- **Files**: `src/webui/app.py` (`/compare?drafts=...`), `src/webui/data.py` (`get_comparison_data()`), new template, add checkboxes to drafts listing

---

## Phase 3: Intelligence

### 3.1 Trend Forecasting [L: 2-3 days]
Predict which areas will grow based on submission velocity, revision activity, WG adoption signals.

- Per-category momentum signals, linear/exponential extrapolation, "Hot/Cooling/Emerging" tags
- **Files**: new `src/ietf_analyzer/forecasting.py`, new web page `/trends`

### 3.2 Change Detection & Diffing [L: 2-3 days]
Track what changed between draft revisions, summarize changes via Claude.

- New `draft_revisions` table to archive old versions
- Section-level diff with Claude-generated change summaries
- **Files**: new `src/ietf_analyzer/diff.py`, update `src/ietf_analyzer/db.py`, `src/ietf_analyzer/fetcher.py`
- **Depends on**: 2.4 (scheduled updates to catch new revisions)

### 3.3 Citation Graph [M: 1-2 days]
Visual dependency tree from the 4231 existing cross-references to 694 RFCs.

- D3.js force-directed graph (pattern from authors.html), PageRank-style influence scores
- **Files**: `src/webui/data.py` (`get_citation_graph()`), new template `citations.html`

### 3.4 Newsletter Generation [M: 1-2 days]
Automated weekly/monthly digest with new drafts, significant changes, trend shifts.

- **Files**: new `src/ietf_analyzer/newsletter.py`, `src/ietf_analyzer/cli.py` (`ietf newsletter`)
- **Depends on**: 3.2 (for change content)

---

## Phase 4: Scale

### 4.1 Docker Deployment [S: 3-4h]
Dockerfile + docker-compose.yml with Flask app + Ollama.

### 4.2 Complete W3C Integration [M: 1 day]
Wire existing W3C fetcher into Observatory pipeline, make prompts source-aware, add source filter to web UI.

### 4.3 IEEE + 3GPP Sources [L: 3-5 days]
New source fetchers following `SourceFetcher` protocol. Depends on 4.2 validating the pipeline.

### 4.4 Cross-SDO Analysis [L: 2-3 days]
Compare work across standards bodies. Embedding similarity between IETF/W3C/IEEE specs, gap analysis for topics only one body covers.

### 4.5 Plugin Architecture [L: 2-3 days]
Formalize extension points for sources, analyzers, report types via Python entry points.

---

## Recommended Agent Team Assignments

### Immediate Sprint (Phase 1 — all in parallel)

| Agent | Items | Focus |
|-------|-------|-------|
| **Coder A** | 1.1 + 1.2 | Global search + REST API |
| **Coder B** | 1.4 + 1.3 | Annotations + Export |
| **Coder C** | 1.5 | Test suite foundation |

### Next Sprint (Phase 2 — mostly parallel)

| Agent | Items | Focus |
|-------|-------|-------|
| **Coder A** | 2.1 | Semantic search / Ask (signature feature) |
| **Coder B** | 2.2 + 2.5 | Competition mapping + Comparison view |
| **Coder C** | 2.3 + 2.4 | Readiness scoring + Pipeline health |

---

## Impact vs Effort Matrix

```
HIGH IMPACT
    |
    |  2.1 Ask          3.2 Diffing
    |  1.1 Search       3.1 Forecasting
    |  2.2 Competition  4.4 Cross-SDO
    |  2.5 Compare
    |  1.2 API
    |  2.3 Readiness    3.3 Citations
    |  1.4 Annotations  3.4 Newsletter
    |  1.3 Export       4.2 W3C
    |  1.5 Tests        4.1 Docker
    |                   4.5 Plugins
    |                   4.3 IEEE/3GPP
LOW IMPACT
    +------------------------------------------
    LOW EFFORT                      HIGH EFFORT
```
