Remote
Remote
Senior
Full Time
13 days ago
AILLMPlatform EngineeringRemotePythonGCPInfrastructure as CodeOpen Source
Requirements
- •Experience shipping production LLM agent systems with operational history and real users
- •Ability to design evaluations with golden sets, behavioral assertions, judge rubrics, pass thresholds, and CI gates
- •Deep API work against systems where work lives and authoring MCP servers
- •Ownership of infrastructure end to end in Python on GCP with cloud warehouse and infrastructure as code
- •Taste about how software contacts humans treating notifications as spending limited trust
What You'll Do
- •Ship the agent platform to production including event-triggered queue, headless model-agnostic runtime, durable state, human review gate, atomic rollback, and complete run logging
- •Own the evaluation layer and implement gates based on it with golden suites, behavioral assertions, judge criteria, safety cases, and CI gates
- •Build and register the agent portfolio across executive, team-lead, and individual-contributor layers plus a meta layer
- •Enforce governance in code with risk tiers, least-privilege credentials, tool-permission gates, decision audit logs, and autonomy classes
- •Design how the system contacts people with interruption budgets, message bundling, and useful communication structure
- •Own the platform tooling including compiler, validator, inventory integrity, and distribution paths
- •Compute operating measures from production data with pipelines and maturity grades per team
- •Instrument the platform's own return with a ledger logging work absorbed and computing monthly figures
Nice to Have
- •Public work in AI agent space such as open-source agent frameworks, MCP servers, evaluation harnesses, or cited writings on agent reliability
- •Experience with LLM observability and cost instrumentation including tracing agent runs and attributing spend
- •Built internal platforms adopted voluntarily by non-engineers and improved based on user feedback
Benefits
- •Fully remote work with global hiring and WeWork membership or co-working allowance
- •ESOP (equity ownership) for all team members
- •Tech allowance for work environment setup
- •Health benefits covering 100% employee and 80% dependents' insurance globally
- •Annual off-sites for company connection and collaboration
- •Flexible asynchronous work with self-managed time
- •Annual professional development allowance for education
