United States (Remote)•US (Remote)
Remote
Senior
Full Time
2 days ago
AIQuality AssuranceLeadershipRemote
Requirements
- •5+ years of experience in Quality Engineering, Deployments, or Technical QA, ideally within an AI or high-growth SaaS environment
- •Strong technical intuition and curiosity about how LLMs work, comfortable with concepts like LLM, RAG, prompt logic, and multi-turn conversational flows
- •Proven ability to lead large end-to-end technical projects through partners
- •Ability to find edge cases and deliver solutions with urgency
- •High empathy and consultative mindset to represent the human element of customer support interactions
- •Ability to thrive in fast-paced environments and turn ambiguity into execution
What You'll Do
- •Architect and scale AI evaluation systems using LLM-driven methodologies including automated simulations, LLM-on-LLM rubrics, and adversarial red-teaming
- •Drive deployment excellence by partnering with engineers and PMs to triage issues and create new test cases
- •Conduct manual UAT and voice-call testing to represent the end-customer experience
- •Lead and scale a team of QA analysts and partners, defining best practices and communication loops
Nice to Have
- •Experience with CCaaS, telephony, or STT/TTS technologies
- •Background in Conversation Design or SDET roles
- •Experience leading teams with direct reports
Benefits
- •Comprehensive medical, dental, and vision coverage
- •Flexible PTO
- •Paid parental leave
- •Retirement savings plan
- •Remote work setup budget
- •Monthly wellness and communication stipend
- •In-office meal program and commuter benefits for onsite employees
About Cresta
Cresta uses artificial intelligence to assist sales and service agents in the quality of customer service.
Palo Alto, CA, US
250 - 500
AI & Machine Learning