AI Data Services Solution Architect

Are you interested in working with the World’s leading AI-powered Quality Engineering Company? Ready to advance your career, team up with global thought leaders across industries and make a difference every day? Join us at Qualitest! 

We are looking for an AI Data Services Solution Architect to join our growing team in United States!

AI Data Services Solution Architect (Human-in-the-Loop)

Role Overview:
The AI Data Services Solution Architect supports the Sales and Delivery teams by designing data-focused AI solutions for customers. This role combines client engagement with practical expertise in data sourcing, annotation workflows, model evaluation, safety, and human‑in‑the‑loop (HITL) services. The architect leads discovery discussions, defines solution patterns, supports proposals and RFPs, and ensures technical clarity for customer-facing programs.

Key Responsibilities:

• Lead customer discovery sessions to understand requirements across data sourcing, annotation, evaluation, and HITL review processes.
• Design end‑to‑end solution architectures including data pipelines, annotation workflows, QA processes, and evaluation frameworks.
• Define data governance considerations—provenance, labeling guidelines, redaction, PII handling, and compliance alignment.
• Develop HITL workflows: task routing, reviewer processes, sampling methods, gold sets, and quality controls.
• Create evaluation approaches for LLMs/ML models, including metrics for grounding accuracy, hallucination checks, and safety compliance.
• Recommend safety guardrails for privacy, toxicity, bias, and content compliance.
• Contribute API integration patterns for data ingestion, annotation platforms, and evaluation tools; support basic UI prototypes.
• Support Sales with solution documentation, diagrams, estimates, SOW inputs, and presentation materials.
• Partner with Delivery for technical handoff, success criteria, and readiness planning.

Required Skills & Experience:

• 6 - 10+ years in solution architecture, data engineering, ML, or related technical roles with client-facing experience.
• Strong understanding of:
  
  • Data pipelines (Spark/Databricks, ADF/Glue, SQL/Python).
  • Annotation/HITL platforms (Label Studio, SageMaker Ground Truth, Scale AI, Appen, etc.).
  • LLM/ML evaluation concepts and safety best practices.
  • API integrations and modern front-end frameworks (React/Next.js).
  • Ability to communicate complex topics to both technical and business stakeholders.
  • Solid documentation and diagramming skills (Visio, Lucidchart, Miro).

Must Have:

• Execute API Integration.
• Understanding of AI models and evaluation.
• Sales experience.

Preferred Qualifications:

• Experience integrating CRM, support systems, or enterprise content repositories.
• Familiarity with Azure OpenAI, AWS Bedrock/SageMaker, or Vertex AI.
• Knowledge of Responsible AI, data privacy, and compliance frameworks.
• Success Indicators.
• High-quality client discovery and clear solution documentation.
• Strong POC and pilot scoping with measurable success criteria.
• Annotation quality metrics (sampling accuracy, agreement rates) aligned to expectations.
• Effective evaluation frameworks reducing hallucinations and improving model quality.
• Smooth coordination between Sales and Delivery teams.
• Location: West Coast Preferred.

Benefits:
Why QualiTest?

• Be a part of a company who strives to support for diversity and inclusion in the workplace – we are one, we are many at Qualitest. Celebrate culture, share knowledge with engineers from around the globe, and inspire each other through our differences. We have more than 40% women and around 120 different nationalities. Local and global opportunities – we offer you internal rotation and international mobility opportunities to grow your career.
• Clear view of your career and progression with the company – Qualitest is growing massively (since 2021 – tripled our employees base – we now have more than 8,000 engineers) and giving you the opportunity to grow with us.
• Work hard and play harder with our flexible and casual culture. Take a break from work and join an employee event, or enjoy the amenities and games provided from one of our Employees Centers. Save your earnings and prepare for your future by enrolling in our 401k plan where Qualitest will match your contributions accelerating your savings plan.
• Take care of health with enrollment into one of our competitive healthcare benefits. Qualitest will match towards your HSA if you choose to participate.
• Never stop experimenting and learning with QCraft – our Learning & Development platform: 50,000+ courses, 300+ virtual labs, mentorship and leadership programs, professional tribes, sponsored certifications, and much more.
• Stay active and get rewarded with our Corporate Wellness Program. We pay your Gym membership and giving you opportunities to Earn additional vacation times for attendance the gym!
• Earn bonuses via our Client Referral and Employee Referral Program’s. Refer and earn – tap your network for net-worth.
• We recognize our employees work via our Qudos platform - You can earn bonuses and spot awards by celebrating your and your peers’ achievements.
• Planning a vacation? Looking for car insurance? Get access to Qualitest Employee Perks for discounts on anything from travel to electronics. With so many offerings the savings are endless!
• A Competitive pay, the salary range for the role is $170,000 - $190,000.
• Intrigued to find more about us?

Visit our website at https://www.qualitestgroup.com/
If you like what you have read, send us your resume and let’s start talking!