“Included as a consequential modern Indian ai data employer with relevant product, engineering, data or AI roles.”
4
stages
2–5 weeks
end to end
20
practice questions
Hiring now
Karya has 13 open roles right now
landed applies to roles like these for you, on each company’s own site, with answers written for the role.
Research Associate - AI Evaluations
Bengaluru
Research Engineering Intern, AI Evaluations
Bengaluru, India
Partnerships Associate
Bengaluru, India
01
The interview, stage by stage
- 1
Founder or hiring manager screen60 min
data qualityannotationPrepare one quantified example demonstrating data quality.
- 2
Practical role exercise60 min
annotationfair work designPrepare one quantified example demonstrating annotation.
- 3
Technical/product deep dive60 min
fair work designmultilingual coveragePrepare one quantified example demonstrating fair work design.
- 4
Team values and ownership round90 min
multilingual coverageevaluationPrepare one quantified example demonstrating multilingual coverage.
02
What decides the offer
Data quality
25%
Shows specific decisions and measurable evidence for data quality.
Annotation
20%
Shows specific decisions and measurable evidence for annotation.
Fair work design
20%
Shows specific decisions and measurable evidence for fair work design.
Multilingual coverage
20%
Shows specific decisions and measurable evidence for multilingual coverage.
Evaluation
15%
Shows specific decisions and measurable evidence for evaluation.
They look hardest for data quality, annotation, fair work design, multilingual coverage.
03
Your four weeks
Days 1–2
Company and market
- ·Map products, users, revenue model and competitors
- ·Write a one-page company thesis
Days 3–5
Core role skills
- ·Practice data quality
- ·Practice annotation
- ·Practice fair work design
Days 6–8
Company-context cases
- ·Design a high-quality Indic-language data pipeline.
- ·Measure annotator agreement and bias.
- ·Use active learning without narrowing coverage.
Days 9–11
Technical simulations
- ·Complete two timed exercises
- ·Run one architecture/product mock
- ·Practice follow-up pressure
Days 12–14
Stories and final loop
- ·Prepare six quantified ownership stories
- ·Rehearse project deep dive
- ·Prepare interviewer questions
04
Practice these
- Design a production RAG system for ten million documents.Recommended
- How do you evaluate outputs when human raters disagree?Recommended
- Compare supervised fine-tuning, preference optimization, and reinforcement learning from feedback.Recommended
- Design an ablation study for a new agent architecture.Recommended
- Design reliable synchronization between a CRM, data warehouse, and enrichment providers.Recommended
- How would you audit whether an AI system treats demographic groups fairly?Recommended
- How would you detect memorization or sensitive-data leakage from a model?Recommended
- How would you build a feedback loop without amplifying user bias or abuse?Recommended
- Evaluate LLMs on a toy task and use LLMs to generate additional evaluation data.Recommended
- Design a language model that minimizes harmful outputs while remaining useful and expressive.Recommended
- Assign human labelers, tasks, and models so every pairing is balanced.Recommended
- Represent a one-nearest-neighbor classifier using a feed-forward neural network.Recommended
- Design a machine-learning system.Recommended
- What approaches would you use to improve transformer efficiency and performance?Recommended
- How does PyTorch Fully Sharded Data Parallel work?Recommended
- Design a production speech recognition system from data collection through serving.Recommended
- How would you improve the quality, latency, and cost of a speech recognition system without masking regressions?Recommended
- Design a production text-to-speech system from data collection through serving.Recommended
- How would you improve the quality, latency, and cost of a text-to-speech system without masking regressions?Recommended
- Design a production voice cloning system from data collection through serving.Recommended
05
Where people slip
- !Generic enthusiasm about startup culture or AI
- !No understanding of the company business model
- !Ignoring Indian price sensitivity, regulation or operational variance
- !Architecture without failure recovery and observability
- !Claiming a recommended case was actually asked
06
Ask them this
- ?What is the exact loop for this team and level?
- ?Which rounds permit AI tools?
- ?What would I own in the first 90 days?
- ?What is the hardest product or model constraint the team faces in India?
- ?How does the team measure quality after launch?
- ?How are ESOPs valued and what is the exercise policy?
