12 generative AI interview questions: RAG, agents and evaluation

RAG retrieves relevant information and supplies it as context for generation. It is useful when an answer needs a specific collection of documents or changing external knowledge. Describe the stages—ingestion, retrieval, context assembly and answer generation—and where errors can occur. Google Cloud provides a technical overview.

Follow-up to practise: how would you tell whether the wrong answer came from missing evidence or from the model misusing good evidence?

Use retrieval when the main need is access to external knowledge with source traceability. Fine-tuning changes model behaviour through training and may help with patterns or task behaviour. They can be combined. First define the problem and compare against a baseline; neither approach automatically fixes inaccurate answers.

Start with document structure and the kinds of questions users ask. Preserve headings and source metadata; avoid splitting information that needs to be read together. Compare candidate approaches on representative questions rather than declaring one chunk size universally best.

A good answer explains the tradeoff between enough context and irrelevant material, and how retrieval and final-answer quality would be checked separately.

Define representative questions, expected evidence and a scoring rubric. Test retrieval separately from the final answer. Include unsupported, ambiguous and permission-restricted questions. Google Cloud’s evaluation guidance is useful background.

State sample size, selection method and failure categories. Hold back some cases during development, and compare changes with a baseline. If a model grades answers, validate its judgments against human review.

Improve evidence quality, make source boundaries clear, evaluate retrieval and teach the application to abstain when evidence is missing. Check that citations actually support the answer. Avoid promising that a prompt or RAG eliminates hallucinations.

Example: when two policy versions conflict, a good system should use explicit version rules or ask for review, rather than silently combining them.

Consider an agent when the next step depends on intermediate results and a model’s dynamic choice adds value. Prefer a fixed workflow when the sequence is predictable and easier to verify. Anthropic’s workflow/agent distinction is useful background. Explain the extra cost and failure paths introduced by dynamic tool use.

Enforce permissions in the application and underlying service. Validate arguments, restrict available actions and require approval for consequential changes where appropriate. A model’s instruction to behave safely is not an authorisation boundary.

Be ready to explain what happens if retrieved material tries to redirect the agent or if a user requests another person’s information. OWASP’s risk guidance provides relevant security context.

Set timeouts and bounded retries. Record task state, distinguish retryable errors from permanent failures and design operations to avoid duplicate effects. Stop and escalate when progress is uncertain. In a portfolio, demonstrate the failure path rather than merely listing these controls.

An agent may produce an attractive explanation while leaving the wrong final state. Evaluate the completed task, intermediate actions, permission use and resource consumption. Anthropic’s agent-evaluation article discusses why traces and grading design matter.

For a support agent, checking the final draft is only one part: also inspect whether it touched the correct record, used permitted tools and respected the approval step.

Measure where time and spending go before changing the architecture. Consider reducing unnecessary context, caching where freshness and permissions allow it, choosing simpler paths for easy tasks and avoiding redundant calls. Re-run quality checks after each change.

Explain which tradeoff is acceptable for the user. A cheaper answer that requires repeated corrections may increase total workflow cost.

Clarify users, data sources, access rules, update frequency and acceptable failure modes. Sketch ingestion, retrieval, generation, evidence display and monitoring. Define how deleted or changed documents propagate and how the system responds when it cannot answer.

Discuss a baseline such as search plus a curated FAQ. Present evaluation and rollout as part of the design rather than adding them at the end.

Describe the goal, your role, the observed failure, the evidence and the change you made. Explain what remained unresolved. If the project was simulated or personal, say so. A useful answer demonstrates learning and ownership without inventing production scale.

Prepare a short walkthrough with one architecture sketch, one result table and one failure example. For a mock interview, have a peer challenge assumptions and ask you to reproduce a result. Use technical vocabulary only when you can explain what it means in your system.