Senior Data Scientist
Content + Source + Freshness • 14 Dec 2025 • 95% confidence
Offer value
High value reflected by a competitive salary, strong brand recognition, and work on cutting-edge AI technologies.
- Salary range: $119,800 - $234,700/year
- Significant role in AI product development
- Opportunity for substantial career growth
- Requires considerable experience and technical skills
Pros
- Attractive salary range ($119,800 - $234,700/year) indicating strong compensation in data science roles.
- Opportunity to work with high-profile AI products and collaborate with various teams.
- Focus on continuous improvement and direct impact on product quality.
Cons
- Extensive experience required (e.g., 5+ years depending on education level).
- Expectations for high technical proficiency and adaptability in a fast-paced environment.
- Potentially limited remote work due to in-office requirements.
Who it's for
Senior • Hybrid / office with some trips
Good fit
- Experienced data scientists with an interest in AI.
- Professionals eager to influence product quality.
- Candidates skilled in statistical analysis and data evaluation.
Not recommended for
- Entry-level candidates.
- Applicants looking for fully remote positions.
- Individuals unprepared for fast-paced analytics roles.
Motivation fit
Key skills
About the job
Senior Data Scientist
Redmond, Washington, United States
Save
Share job
Overview
As a Senior Data Scientist part of Cadets, you will own evaluation analytics end‑to‑end: curate datasets from customer and production signals; author binary‑first rubrics; build LLM (Large Language Model)‑as‑judge graders and work on high‑quality synthetic data generation to scale evaluations with experience in human‑match rates. You’ll partner with PM/Eng/Design and VIP customers to ship quality gains and AI features with confidence.
You’ll Thrive Here If You Have:Evaluation proficiency for LLM/agent systems: dataset curation, rubric design, human‑in‑the‑loop grading, judge prompts with quantitative agreement goals.
Experience in analytics & experimentation skills (statistical inference, A/B), plus Python/SQL for large‑scale trace analysis.
LLM fundamentals: prompt engineering, few‑shot design, retrieval metrics, multi‑turn/agent trace evaluation.
Data quality mindset: trace hygiene, metadata design, policy/PII awareness, and principled guardrails.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Qualifications
Required Qualifications:
- Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR equivalent experience.
- Experience with building data pipelines, performing large-scale analysis, and implementing ML workflows using Python and SQL.
- Experience in developing models or designing evaluation frameworks, including A/B testing or prompt-based assessments for LLMs.
Other Requirements:
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Preferred Qualifications:
- Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 7+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results) OR equivalent experience.
- Experience building graders that score persona/tone, contract/formatting (e.g., JSON validity, schema), and tool‑use correctness.
- Background with structured synthetic data generation and vendor annotation programs; familiarity with judge mutation/optimization loops.
- 2+ years customer-facing, project-delivery experience, professional services, and/or consulting experience.
- AI & Technical Fluency: You don't need to train models, but you know how they work, how to test them, and how to build great products on top of them.
- Experience in communication and stakeholder management skills.
- Ability to work in a fast-paced, ambiguous environment and deliver results under tight deadlines.
Data Science IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $158,400 - $258,000 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
Microsoft will accept applications and processes offers for these roles on an ongoing basis.
#MSAI
#M365Core
#M365Copilot
Responsibilities
- Evaluation & Feedback Analysis
- Convert multi‑source feedback (dogfood, VIP customers, production traces) into a prioritized dataset of 10–100 tasks per scenario, each with prompts and golden outputs; maintain a living failure taxonomy prioritized by volume × impact × fixability.
- Rubrics & LLM‑as‑Judge
- Author crisp, binary‑first rubrics across 7–30 dimensions (e.g., correctness/completeness, refusal calibration, tool‑use quality, formatting/contract, persona/tone, trace hygiene).
- Build grader prompts (with few‑shots and counter‑examples) that achieve ≥80% human‑match rate, track TPR/TNR on held‑out sets, and prevent reward hacking.
- Synthetic & Human‑Labeled Data
- Design structured tuples to scale high‑signal synthetic data; orchestrate vendor/partner annotation sprints and live calibrations to align shared judgment.
- Ensure datasets are reproducible with linked artifacts and robust metadata/trace hygiene.
- Customer‑Grounded Scenarios
- Partner with PMs/solution architects to co‑develop evals with VIP customers so tasks reflect real outcomes and workflows; quantify lift from fixes and inform the next hill‑climb.
- Team Leadership & Ways of Working
- Co‑own the Cadets “feedback flywheel” with PM/Eng (instrumentation, taxonomy, guardrails vs. evaluators) and help operationalize weekly checklists, change logs, and judge refresh cadence.
