agent: session-23 — Module A1 end-to-end tested and documented
- Fixed lead data loss bug in Enrichment Mapper (fullResponse overwrites input data — now references upstream node) - Full README with architecture diagram, input/output schemas, swap points, test results, pricing tiers - Live test results: validation ✓, mock enrichment ✓, Apollo path ✓ - Apollo free plan blocks people/match — documented as known limitation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jake
@@ -1,5 +1,186 @@
|
||||
# Lead Enrich Pipeline — Module A1
|
||||
|
||||
> Full documentation coming after workflow build is complete. This placeholder exists so the directory structure is committed.
|
||||
Webhook-triggered lead enrichment pipeline. Accepts a lead via POST, enriches it through Apollo (or mock data in test mode), grades the profile, and passes the result to a configurable downstream actions workflow.
|
||||
|
||||
See spec: `docs/superpowers/specs/2026-03-27-module-a1-lead-enrich-pipeline-design.md` in CORBEL-Furnished-House.
|
||||
## Architecture
|
||||
|
||||
```
|
||||
POST /webhook/lead-enrich
|
||||
│
|
||||
▼
|
||||
Input Mapper ─── normalise fields
|
||||
│
|
||||
▼
|
||||
Clean & Validate ─── trim, lowercase, email regex
|
||||
│
|
||||
├── INVALID ──► Dead Letter Response (422)
|
||||
│
|
||||
▼
|
||||
Test Mode Check
|
||||
│
|
||||
├── TEST_MODE=true ──► Mock Enrichment (static data)
|
||||
│
|
||||
├── TEST_MODE=false ──► Apollo HTTP Request
|
||||
│ │
|
||||
│ ▼
|
||||
│ Enrichment Mapper & Grader
|
||||
│
|
||||
▼
|
||||
Success Response (200) + Call Actions Workflow
|
||||
```
|
||||
|
||||
**Two workflows:**
|
||||
- `lead-enrich-core.json` — the pipeline above (this is the product)
|
||||
- `lead-actions-example.json` — reference recipe showing downstream routing by grade (CRM sync, Slack notification, welcome email, audit log)
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. Import `lead-enrich-core.json` into n8n (Settings → Import Workflow or via API)
|
||||
2. Set `TEST_MODE=true` in your n8n environment variables
|
||||
3. Activate the workflow
|
||||
4. Test:
|
||||
|
||||
```bash
|
||||
curl -X POST https://your-n8n-instance/webhook/lead-enrich \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"email": "test@example.com", "first_name": "Test", "last_name": "User", "company_name": "Acme", "source": "manual"}'
|
||||
```
|
||||
|
||||
You should get a 200 response with mock enrichment data and a "Full Profile" grade.
|
||||
|
||||
## Input Schema
|
||||
|
||||
POST JSON body:
|
||||
|
||||
| Field | Type | Required | Notes |
|
||||
|---|---|---|---|
|
||||
| `email` | string | Yes | Validated with regex. Lowercased automatically. |
|
||||
| `first_name` | string | No | Passed to Apollo for matching accuracy |
|
||||
| `last_name` | string | No | Passed to Apollo for matching accuracy |
|
||||
| `company_name` | string | No | Passed to Apollo as `organization_name` |
|
||||
| `source` | string | No | Defaults to "webhook". Use for tracking origin. |
|
||||
|
||||
Accepts fields at root level (`$json.email`) or nested in body (`$json.body.email`) — the Input Mapper handles both.
|
||||
|
||||
## Output Schema
|
||||
|
||||
200 response:
|
||||
|
||||
```json
|
||||
{
|
||||
"lead": {
|
||||
"email": "...",
|
||||
"first_name": "...",
|
||||
"last_name": "...",
|
||||
"company_name": "...",
|
||||
"source": "...",
|
||||
"submitted_at": "ISO timestamp",
|
||||
"raw_payload": { ... }
|
||||
},
|
||||
"enrichment": {
|
||||
"provider": "apollo|mock",
|
||||
"person": {
|
||||
"title": "...",
|
||||
"linkedin_url": "...",
|
||||
"city": "...",
|
||||
"state": "...",
|
||||
"country": "...",
|
||||
"seniority": "..."
|
||||
},
|
||||
"organisation": {
|
||||
"name": "...",
|
||||
"domain": "...",
|
||||
"industry": "...",
|
||||
"estimated_employees": 250,
|
||||
"annual_revenue": 5000000,
|
||||
"linkedin_url": "...",
|
||||
"website_url": "..."
|
||||
},
|
||||
"enrichment_error": false,
|
||||
"credits_consumed": 1,
|
||||
"rate_limit_remaining": 95,
|
||||
"credits_low": false
|
||||
},
|
||||
"grading": {
|
||||
"profile_grade": "Full Profile|Partial Profile|Limited",
|
||||
"fields_populated": 13,
|
||||
"graded_at": "ISO timestamp"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
422 response (validation failure):
|
||||
|
||||
```json
|
||||
{
|
||||
"error": true,
|
||||
"error_reason": "Invalid email format: ...",
|
||||
"lead": { ... },
|
||||
"timestamp": "ISO timestamp"
|
||||
}
|
||||
```
|
||||
|
||||
## Profile Grading
|
||||
|
||||
| Grade | Criteria |
|
||||
|---|---|
|
||||
| **Full Profile** | Company name + job title + employee count all present |
|
||||
| **Partial Profile** | Company name present but missing title or employee count |
|
||||
| **Limited** | No company data, enrichment error, or no Apollo match |
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Set these on your n8n instance (Docker env vars, not n8n Variables which require a paid plan):
|
||||
|
||||
| Variable | Required | Description |
|
||||
|---|---|---|
|
||||
| `TEST_MODE` | Yes | `"true"` for mock data, `"false"` for live Apollo |
|
||||
| `APOLLO_API_KEY` | For live mode | Apollo People Enrichment API key. **Requires a paid Apollo plan** — the free tier does not include the `people/match` endpoint. |
|
||||
|
||||
See `.env.example` for the full list including downstream action credentials (HubSpot, Slack, SMTP).
|
||||
|
||||
## Apollo API Notes
|
||||
|
||||
- **Endpoint:** `POST https://api.apollo.io/api/v1/people/match`
|
||||
- **Free tier limitation:** The `people/match` endpoint is **not available on the free plan**. You need at least a Basic plan ($49/month) or equivalent.
|
||||
- **Rate limits:** 100 requests/minute on paid plans. The workflow reads `x-minute-requests-left` from response headers and sets `credits_low: true` when below 50.
|
||||
- **No-match behaviour:** Apollo returns a successful response with `person: null`. No credits are consumed. The grader assigns "Limited" grade.
|
||||
- **Retry logic:** The HTTP Request node retries up to 3 times with 5-second backoff on failure.
|
||||
|
||||
## Swap Points (for Client Deployment)
|
||||
|
||||
These are the nodes you'd modify when deploying for a client:
|
||||
|
||||
1. **Webhook path** — change `lead-enrich` to a client-specific path
|
||||
2. **Apollo API key** — client provides their own key via env var
|
||||
3. **Grading thresholds** — adjust in the Enrichment Mapper & Grader Code node
|
||||
4. **Actions workflow** — replace `lead-actions-example` with client-specific routing
|
||||
|
||||
## Test Results (2026/03/28)
|
||||
|
||||
| Test | Result | Notes |
|
||||
|---|---|---|
|
||||
| Invalid email (bad format) | PASS — 422, correct error message | |
|
||||
| Missing email | PASS — 422, "Email is missing or empty" | |
|
||||
| Test mode (mock enrichment) | PASS — 200, Full Profile, 13 fields, provider=mock | |
|
||||
| Live Apollo (valid email) | PASS — 200, Limited grade, provider=apollo | Apollo free plan blocks people/match endpoint. Workflow handles gracefully. |
|
||||
| Lead data in response | PASS — lead object present in all paths | Bug fixed: upstream node reference for Apollo path |
|
||||
|
||||
## Files
|
||||
|
||||
```
|
||||
A1-lead-enrich-pipeline/
|
||||
├── README.md ← this file
|
||||
├── .env.example ← required credentials
|
||||
└── workflows/
|
||||
├── lead-enrich-core.json ← the product (Workflow 1)
|
||||
└── lead-actions-example.json ← reference recipe (Workflow 2)
|
||||
```
|
||||
|
||||
## Pricing (Template Marketplace)
|
||||
|
||||
| Tier | Price | What's included |
|
||||
|---|---|---|
|
||||
| Core template | £39–49 | Raw workflows + this documentation. Buyer wires own adapters. |
|
||||
| Ready-made bundle | £79–99 | Pre-configured for Tally → Apollo → HubSpot + Sheets + Slack. |
|
||||
| Done-for-you | £350–600 | Jake deploys, configures, tests, hands over. 7-day support window. |
|
||||
|
||||
Reference in New Issue
Block a user