CORBEL-Automations/modules/A1-lead-enrich-pipeline/README.md

# Lead Enrich Pipeline — Module A1

Webhook-triggered lead enrichment pipeline. Accepts a lead via POST, enriches it through Apollo (or mock data in test mode), grades the profile, and passes the result to a configurable downstream actions workflow.

## Architecture

```
POST /webhook/lead-enrich
    │
    ▼
Input Mapper ─── normalise fields
    │
    ▼
Clean & Validate ─── trim, lowercase, email regex
    │
    ├── INVALID ──► Dead Letter Response (422)
    │
    ▼
Test Mode Check
    │
    ├── TEST_MODE=true ──► Mock Enrichment (static data)
    │
    ├── TEST_MODE=false ──► Apollo HTTP Request
    │                           │
    │                           ▼
    │                   Enrichment Mapper & Grader
    │
    ▼
Success Response (200) + Call Actions Workflow
```

**Two workflows:**
- `lead-enrich-core.json` — the pipeline above (this is the product)
- `lead-actions-example.json` — reference recipe showing downstream routing by grade (CRM sync, Slack notification, welcome email, audit log)

## Quick Start

1. Import `lead-enrich-core.json` into n8n (Settings → Import Workflow or via API)
2. Set `TEST_MODE=true` in your n8n environment variables
3. Activate the workflow
4. Test:

```bash
curl -X POST https://your-n8n-instance/webhook/lead-enrich \
  -H "Content-Type: application/json" \
  -d '{"email": "test@example.com", "first_name": "Test", "last_name": "User", "company_name": "Acme", "source": "manual"}'
```

You should get a 200 response with mock enrichment data and a "Full Profile" grade.

## Input Schema

POST JSON body:

| Field | Type | Required | Notes |
|---|---|---|---|
| `email` | string | Yes | Validated with regex. Lowercased automatically. |
| `first_name` | string | No | Passed to Apollo for matching accuracy |
| `last_name` | string | No | Passed to Apollo for matching accuracy |
| `company_name` | string | No | Passed to Apollo as `organization_name` |
| `source` | string | No | Defaults to "webhook". Use for tracking origin. |

Accepts fields at root level (`$json.email`) or nested in body (`$json.body.email`) — the Input Mapper handles both.

## Output Schema

200 response:

```json
{
  "lead": {
    "email": "...",
    "first_name": "...",
    "last_name": "...",
    "company_name": "...",
    "source": "...",
    "submitted_at": "ISO timestamp",
    "raw_payload": { ... }
  },
  "enrichment": {
    "provider": "apollo|mock",
    "person": {
      "title": "...",
      "linkedin_url": "...",
      "city": "...",
      "state": "...",
      "country": "...",
      "seniority": "..."
    },
    "organisation": {
      "name": "...",
      "domain": "...",
      "industry": "...",
      "estimated_employees": 250,
      "annual_revenue": 5000000,
      "linkedin_url": "...",
      "website_url": "..."
    },
    "enrichment_error": false,
    "credits_consumed": 1,
    "rate_limit_remaining": 95,
    "credits_low": false
  },
  "grading": {
    "profile_grade": "Full Profile|Partial Profile|Limited",
    "fields_populated": 13,
    "graded_at": "ISO timestamp"
  }
}
```

422 response (validation failure):

```json
{
  "error": true,
  "error_reason": "Invalid email format: ...",
  "lead": { ... },
  "timestamp": "ISO timestamp"
}
```

## Profile Grading

| Grade | Criteria |
|---|---|
| **Full Profile** | Company name + job title + employee count all present |
| **Partial Profile** | Company name present but missing title or employee count |
| **Limited** | No company data, enrichment error, or no Apollo match |

## Environment Variables

Set these on your n8n instance (Docker env vars, not n8n Variables which require a paid plan):

| Variable | Required | Description |
|---|---|---|
| `TEST_MODE` | Yes | `"true"` for mock data, `"false"` for live Apollo |
| `APOLLO_API_KEY` | For live mode | Apollo People Enrichment API key. **Requires a paid Apollo plan** — the free tier does not include the `people/match` endpoint. |

See `.env.example` for the full list including downstream action credentials (HubSpot, Slack, SMTP).

## Apollo API Notes

- **Endpoint:** `POST https://api.apollo.io/api/v1/people/match`
- **Free tier limitation:** The `people/match` endpoint is **not available on the free plan**. You need at least a Basic plan ($49/month) or equivalent.
- **Rate limits:** 100 requests/minute on paid plans. The workflow reads `x-minute-requests-left` from response headers and sets `credits_low: true` when below 50.
- **No-match behaviour:** Apollo returns a successful response with `person: null`. No credits are consumed. The grader assigns "Limited" grade.
- **Retry logic:** The HTTP Request node retries up to 3 times with 5-second backoff on failure.

## Swap Points (for Client Deployment)

These are the nodes you'd modify when deploying for a client:

1. **Webhook path** — change `lead-enrich` to a client-specific path
2. **Apollo API key** — client provides their own key via env var
3. **Grading thresholds** — adjust in the Enrichment Mapper & Grader Code node
4. **Actions workflow** — replace `lead-actions-example` with client-specific routing

## Apollo Simulator (Test Mode)

When `TEST_MODE=true`, the workflow routes through an **Apollo Simulator** Code node instead of the real API. The simulator generates Apollo-shaped responses with varying data quality based on the email prefix:

| Email prefix | Simulated response | Expected grade |
|---|---|---|
| `full@` (or any other) | All person + org fields populated | Full Profile |
| `partial@` | Company name present, title and employee count null | Partial Profile |
| `limited@` | Person data but empty organisation object | Limited |
| `nomatch@` or `empty@` | `person: null`, zero credits consumed | Limited |
| `error@` | HTTP 429 rate limit, zero remaining | Limited (error) |

The simulator output goes through the **same Enrichment Mapper & Grader** as real Apollo responses. This means test mode exercises the actual parsing logic, not a bypass.

## Test Results (2026/03/28)

### Validation Tests

| Test | Result | Notes |
|---|---|---|
| Invalid email (bad format) | PASS — 422 | Correct error message returned |
| Missing email | PASS — 422 | "Email is missing or empty" |

### Grading Path Tests (via Apollo Simulator)

| Test | Result | Notes |
|---|---|---|
| Full Profile (`full@test.com`) | PASS — 200 | Grade: Full Profile, 13 fields, provider: apollo |
| Partial Profile (`partial@test.com`) | PASS — 200 | Grade: Partial Profile, 8 fields, title: null, employees: null |
| Limited - no org (`limited@test.com`) | PASS — 200 | Grade: Limited, 6 fields, company: null |
| No match (`nomatch@test.com`) | PASS — 200 | Grade: Limited, 0 fields, credits: 0, no error |
| Rate limit error (`error@test.com`) | PASS — 200 | Grade: Limited, error: true, rate_remaining: 0, credits_low: true |
| Lead data in response | PASS | Lead object present in all paths |

### Throughput Tests

| Concurrent | All 200s | Wall time | Throughput | Notes |
|---|---|---|---|---|
| 10 | Yes | 543ms | ~18 req/sec | Avg 345ms per request |
| 25 | Yes | 989ms | ~25 req/sec | Avg 575ms per request |
| 50 | Yes | 3.95s | ~12 req/sec | Tail latency 3.3s (n8n queueing). Acceptable for lead volumes. |

## Files

```
A1-lead-enrich-pipeline/
├── README.md                          ← this file
├── .env.example                       ← required credentials
└── workflows/
    ├── lead-enrich-core.json          ← the product (Workflow 1)
    └── lead-actions-example.json      ← reference recipe (Workflow 2)
```

## Pricing (Template Marketplace)

| Tier | Price | What's included |
|---|---|---|
| Core template | £39–49 | Raw workflows + this documentation. Buyer wires own adapters. |
| Ready-made bundle | £79–99 | Pre-configured for Tally → Apollo → HubSpot + Sheets + Slack. |
| Done-for-you | £350–600 | Jake deploys, configures, tests, hands over. 7-day support window. |