Developers
Architecture
AIDRAN is a unified TypeScript system running entirely on Vercel. One language, one deployment target, one database. No microservices, no message queues, no Python. Every pipeline stage is a serverless function triggered by a cron schedule.
One language, one platform, zero split-brain drift.
System Overview
Data flows through a six-stage pipeline. Each stage is decoupled — they communicate exclusively through the database. No runtime coupling, no shared state, no message bus.
Technology Stack
11 technologies, four layers. Every choice optimizes for simplicity and operational cost over theoretical scalability.
Runtime & Language
| Technology | Role | Why |
|---|---|---|
| Next.js 16 | Application framework | App Router with Server Components, ISR (revalidate=300), API route handlers for all cron jobs and REST endpoints. Turbopack for builds. |
| TypeScript | End-to-end language | One language for ingestion workers, AI pipelines, database queries, API routes, and the React frontend. No Python anywhere in the stack. |
Data Layer
| Technology | Role | Why |
|---|---|---|
| Neon | Serverless Postgres | Managed Postgres with pgvector for 512-dim embedding storage and cosine similarity search. Serverless driver for Vercel edge compatibility. |
| Drizzle ORM | Schema-as-code | Type-safe queries, schema definitions in TypeScript, zero-dependency runtime. All tables defined in a single schema file with Drizzle Kit for migrations. |
AI & ML
| Technology | Role | Why |
|---|---|---|
| Claude (Anthropic) | AI analysis & generation | Haiku for per-record analysis (sentiment, entities, key phrases). Sonnet for editorial narrative generation. Batch API for efficient bulk analysis. |
| Voyage AI | Embeddings | voyage-4-lite model producing 512-dimensional vectors. Used for semantic search, narrative clustering (K-means++), and deduplication (0.82 cosine threshold). |
| Vercel AI SDK | AI orchestration | Streaming text generation, structured outputs with Zod schemas, provider abstraction for Anthropic API. Used across all AI-powered pipeline stages. |
Infrastructure & Frontend
| Technology | Role | Why |
|---|---|---|
| Vercel | Deployment & runtime | Serverless functions for all cron jobs (18 scheduled tasks), ISR for editorial pages, edge CDN for global delivery. Single deployment target. |
| Tailwind CSS v4 | Styling | Dark-first design tokens, custom semantic color palette (ink, muted, accent, surface, raised, border), editorial typography scale. |
| PostHog | Observability | Privacy-respecting analytics for reader behavior. Server-side events for pipeline telemetry — ingestion counts, analysis durations, signal detection rates. |
| Framer Motion | Animation | Page transitions, scroll-triggered section reveals, staggered list entries. Consistent motion language across all editorial pages. |
Pipeline Data Flow
Data flows through 10 stages, each implemented as an independent serverless function. Stages communicate through the database — no runtime coupling, no shared state.
7 source adapters normalize public discourse into a shared schema
Voyage AI vectorizes new records (512-dim, cosine distance)
Claude Haiku extracts sentiment, entities, key phrases, emotional register
7 detectors scan for anomalies — volume spikes, sentiment shifts, entity surges
K-means++ groups related discourse into narrative threads with TF-IDF labels
5 detectors find cross-topic convergence, platform anomalies, aggregate trends
Claude Sonnet generates editorial stories from signals and clusters
Breaking news dispatches from high-severity signals
Saliency scoring, cadence enforcement, cross-platform distribution
Daily rollups per topic — volume, sentiment, source breakdown
Database Schema
All tables are defined in Drizzle ORM. The database is a single Neon Postgres instance with pgvector enabled.
| Table | Purpose |
|---|---|
discourse | Raw records from all sources — the core data table |
topics | Editorial beats with keywords, subreddits, category, status |
narratives | AI-generated stories, dispatches, and beat narratives |
signals | Detected anomalies — volume spikes, sentiment shifts, etc. |
entities | Named entities extracted from discourse analysis |
clusters | K-means++ narrative groupings with top terms |
meta_patterns | Cross-topic patterns and convergence signals |
embedding_queue | Records pending Voyage AI vectorization |
analysis_queue | Records pending Claude Haiku analysis |
seen_source_ids | Dedup table — prevents re-ingesting known records |
ingestion_log | Per-run observability for each source worker |
daily_topic_snapshots | Pre-computed daily rollups per topic per day |
publishing_log | Cross-platform publishing history and cadence tracking |
Cron Schedule
18 scheduled jobs orchestrate the full pipeline. All run as Vercel serverless functions with a 5-minute execution budget.
Ingest
Process
Detect
Generate
Distribute
Key Decisions
Every architectural choice optimizes for simplicity and operational cost over theoretical scalability. The system serves a single editorial product, not a platform.
Stack
Unified Next.js (TypeScript end-to-end)
Eliminates split-brain type drift between Python backend and TS frontend
Instead of Python backend + TS frontend
Database
Neon (serverless Postgres + pgvector)
Native Postgres compatibility, serverless driver for Vercel, embeddings co-located with metadata
Instead of Supabase, PlanetScale, local Postgres
ORM
Drizzle
Type-safe, schema-as-code, lightweight, zero-dependency runtime
Instead of Prisma, TypeORM, raw SQL
AI Narratives
Vercel AI SDK + Claude
Streaming, server-side generation, editorial voice via system prompts
Instead of OpenAI GPT-4, local models
Embeddings
Voyage AI (voyage-4-lite, 512-dim)
Consistent with v1, shared embedding space, cosine similarity
Instead of OpenAI ada-002, Cohere embed
Workers
TypeScript (Vercel Cron + serverless)
One language, deployable to same platform, 5-min execution budget
Instead of Python workers, separate microservices
Real-time
Server-Sent Events
Unidirectional, CDN-friendly, no extra infrastructure
Instead of WebSockets, polling, Pusher
Rendering
Server Components + ISR (5 min)
Narratives pre-rendered, not client-fetched. Data visualizations secondary to prose.
Instead of Client-side SPA, full SSR
Deployment
Vercel
ISR, edge functions, cron, single deployment target for everything
Instead of AWS, Railway, self-hosted