Senior Software Engineer, GenAI & ML Evaluation Frameworks - Grafana Ops, AI/ML (Remote, USA)
Skills
- Experience in evaluation frameworks for AI/ML systems
- Familiarity with prompt engineering and structured output evaluation
- Ability to work in rapid iteration and experimental environments
- Strong collaboration and translation of team goals into testable criteria
Responsibilities
- Design and implement robust evaluation frameworks for GenAI and LLM-based systems
- Develop tooling for automated evaluation of model outputs and prompts
- Define metrics aligning with product goals and operational constraints
- Lead dataset management processes and guide best practices for evaluation
Education
- Bachelor's degree in Computer Science, Engineering, or related field
Benefits
- Equity participation
- Bonus opportunities
- Comprehensive health benefits
- Flexible work hours
Average salary estimate
If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.
Become a key sales leader at Grafana Labs as the Regional Sales Director, driving revenue and mentoring your team in a remote role.
As an Observability Architect at Grafana Labs, you'll lead the evolution of customer experience while ensuring effective adoption and ROI of our innovative solutions.
Join 1Password as a Developer Intern, contributing to innovative onboarding and activation experiences for a secure digital future.
Advanced your career as a Senior Software Engineer focusing on Graph Database technology at Tanium, a leader in real-time endpoint management.
Canvas Medical is looking for an Applied AI Software Engineer to lead evaluation efforts for AI agents that automate tasks in the healthcare sector.
Join HeyTaco as a Full Stack Developer to build impactful software that enhances workplace culture through gratitude.
Be a part of a dynamic team at Aptiv as a Software Developer and help shape the future of mobility.
Experienced developer needed at Citibank to lead application development and programming activities within a hybrid work environment.
Elevate your career as a Software Engineer at Fifth Third Bank, where innovation meets banking excellence.
As a Senior Software Engineer at Penn Mutual, you will play a crucial role in developing innovative web applications for financial planning, working collaboratively in a thriving remote environment.
Join Notion as a Software Engineer to shape the developer experience and enhance engineering productivity within the team.
Join Zipline as a Senior Full Stack Software Engineer and help revolutionize the logistics industry with cutting-edge technology.
Join LMArena as a Senior Security Engineer to lead innovative projects in safeguarding real-time AI evaluation against sophisticated risks like Sybil attacks.
Lead the development of cutting-edge cybersecurity solutions as a Principal Software Engineer at Palo Alto Networks.
Experienced Fullstack Software Engineer II opportunity at Chewy to build backend services for an innovative veterinary B2B marketplace.
Grafana Labs is the company behind Grafana, Loki, Mimir and Tempo, the leading open source software for visualizing operational data.
230 jobs
Subscribe to Rise newsletter
🔮
Hi, I'm Risa! Your AI Career Copilot
you, just ask me below!
