AGENT 03 / 07
Extracting
The extraction worker turns pages, documents, and browser output into structured data — contacts when you need them, or clean inputs for any downstream workflow.
extracting · autonomous
01What it does
Raw sources in, structured outputs out
Everything in one pass
Emails, phones, social profiles, persons and companies from text, HTML or a URL — each item carrying a confidence score (0.0–1.0) and a source label: text, mailto link, tel link, html:alt.
Anti-scraping defeated
"name [at] domain [dot] com", (at), {at}, HTML entities @ / . and even word forms like "bob at acme dot com" are reconstructed into real addresses — tagged 0.70 confidence vs 0.95 for plain ones.
Company sites, decoded
Fetches the homepage, locates contact / team / about pages by path hints, reads Schema.org JSON-LD (Organization name, description, size) and harvests team member cards.
Fast structured parsing
CSS-selector extraction in five modes — texts, html, attr, links, tables. The same parser turns pages, exports, and documents into structured inputs for downstream workers.
Structured API inputs
Dot-path queries over JSON: data.items.0.name for deep access, results[*].email to iterate every element. REST responses and exports become inputs for any worker.
Contacts inside pictures
A vision model OCRs screenshots, banners and rendered images; img alt/title attributes are scanned too. Emails hidden from plain-text scrapers still surface.
02How it works
Six passes over every source
Ingest
Text, HTML or a URL. Fetches run behind an SSRF guard with per-hop redirect validation; the document is parsed once and reused by every extractor.
Mine
Deterministic regex passes sweep visible text, mailto:/tel: links, alt/title attributes and raw markup — JSON-LD blocks included.
Deobfuscate
[at]/(at)/{at}/@ tokens and word-form addresses are reconstructed; asset TLDs like .png and .woff2 are rejected as fake emails.
Normalize
Phones go to E.164 via libphonenumber with region fallback; socials are keyed by platform + handle. Duplicates drop — the highest confidence survives.
Entities
Optional LLM pass returns strict JSON at temperature 0.0 with injection guards. Deterministic selectors run first when a page exposes recognizable cards.
Emit
Structured contacts with confidence and provenance flow on — and the runtime autosaves them to the contact DB before the model can move on.
03Under the hood
Regexes, thresholds and confidence
| Email confidence ladder | mailto link 0.98 · plain text 0.95 · raw HTML/JSON-LD 0.90 · obfuscated 0.70 |
| Email plausibility | local part ≤ 64 chars, total ≤ 254, asset TLDs rejected (png, jpg, svg, woff2, mp4, pdf…) |
| Obfuscation forms | [at] (at) {at} @ and [dot] (dot) {.} . · word forms need whitespace, so prose like "look at this" never matches |
| Phone candidates | 7–15 digits · dates, IPs and round figures rejected · E.164 via libphonenumber · regions tried RU → US → DE → GB |
| Phone confidence | +-prefixed valid 0.9 · local valid 0.8 · tel: link 0.98 · invalid parse 0.35 |
| Social profiles | LinkedIn, X/Twitter, Instagram, Telegram, Facebook · reserved paths filtered · @handles read a ±60-char context window |
| LLM entity pass | 12,000-char input cap · temperature 0.0 · strict JSON with code fences tolerated · prompt-injection defense on untrusted web text |
| Team detection | 14 card selectors (.team-member, [class*="person-card"], [class*="team"] li…) · names ≤ 6 words and ≥ 70% letters |
| Limits | parse_html caps each call at 500 items; selectors and output shapes stay explicit for predictable handoffs |
04Tools
The extraction toolbox
Feeds downstream workers
Structured output is reusable fuel
Every extracted value leaves with a confidence score and source, so downstream workers know what to trust and what to verify. The runtime autosaves results as soon as the extraction tool returns — nothing evaporates when the session ends. Other workers can validate, write, or act on the same structured output.
extract_contacts(url: "https://acme.io/team")
→ emails: [email protected] 0.98 mailto link
→ phones: +74957107580 0.90 RU · E.164
→ socials: linkedin.com/in/ann 0.9
→ persons: Ann Petrova — CTO @ Acme
[auto-persisted 4 contact(s): 4 new, 0 merged]
# → Structuring dedups into one record
# → Cleaning verifies quality and confidence
# → Operations prepares the next governed action