getmethaturl.pensanta.com - agent URL fetch/normalize API
HTTP API on venus: an agent sends a URL plus a desired format, the API fetches it and returns html and/or txt and/or md and/or yaml and/or zip to the requesting agent. Motivating case: InfoLeg (servicios.infoleg.gob.ar) returns 403 to plain curl/wget from agent hosts; the same GET with a browser UA + Referer succeeds, so agents currently hand-roll WAF workarounds per site. Consumers: Claude Code and Codex sessions fleet-wide. Auth: api-key, first key issued to pm-medida-odaia-cc and stored in ~/.claude/ltm.md. Domain: getmethaturl.pensanta.com, Hostinger DNS, same apex as bwi.pensanta.com. Implementation decided by llmmsg blackboard round with bin-venus-cc, nw-venus-cc, bin-whey-cc. HARD CONSTRAINT: this service fetches arbitrary URLs on request, so it is an SSRF primitive by construction. Egress denylist (RFC1918, link-local, 169.254.169.254, loopback), redirect cap, response size and time limits, and per-key rate limiting are in scope and not optional - venus is the fleet hub host and whey holds the only GCABA VPN tunnel.
Sub-tickets — 5/5 done
Activity
-
Transport: REST, converged. bin-whey-cc withdrew its MCP-tool suggestion and bin-venus-cc rejected it independently: a global MCP entry launches in every session on the host for every agent, which is what got github removed on 2026-08-09, and this is a fetch tool most sessions will never call. REST behind Caddy costs one vhost and no per-session process.
-
API dropped by Elazar 2026-09-04 in favour of a local script. A network service was the wrong shape: the failure this was built for is a WAF User-Agent check, which a documented curl/python recipe fixes on every host at zero infra cost. The API's only real adds over that were a centralised egress IP, format normalisation and a shared cache - the first is a NEGATIVE on venus (datacenter ASN draws harder Cloudflare challenges than residential whey), the second is a script not a network service, the third was speculative. Replaced by getpage.py in ~/.local/bin (sh.git, already on PATH on whey/venus/lezama), tracked in project agent-ops. Net effect: no SSRF primitive on the fleet hub host, no DNS record, no TLS, no API key to issue or store, no systemd unit to maintain.