AI Prompt Engineer Jobs in Canada
About The Role AI prompt engineers design, test, and evaluate prompts that steer LLMs toward accurate, safe, and useful outputs across production workflows. You will collaborate with engineering and AI/ML data operations to improve model behavior through prompt iteration, evaluation, and RLHF-adjacent feedback.
What You Will Do Design system prompts, user prompts, and tool instructions for LLM-powered products (chat, search, summarization, extraction, coding assistants).
Run prompt evaluation and QA evaluation using rubrics to measure correctness, groundedness, safety, and style adherence.
Create and maintain datasets for prompt testing, including data labeling and preference data aligned to RLHF objectives.
Analyze failures (hallucinations, instruction gaps, unsafe content) and propose prompt, policy, or data fixes.
Write and enforce annotation guidelines; ensure training data quality and compliance.
Support LLM training pipelines, regression testing, and model performance improvement.
Contribute to content safety labeling, refusal behavior testing, and jailbreak/misuse resistance.
Document prompt libraries, evaluation sets, and decision logs for reproducible experiments.
Required Qualifications Mid-Senior experience building or optimizing prompts for LLM applications in production or rigorous evaluation settings.
Strong writing and reasoning skills to translate product intent into precise instructions and constraints.
Hands-on experience with rubric-based prompt evaluation and QA evaluation.
Understanding of RLHF concepts (preference data, calibration, helpfulness/harmlessness tradeoffs).
Ability to define measurable criteria for quality and safety under ambiguity.
Preferred Qualifications Experience with NLP tasks (summarization, classification, NER, extraction) and RAG evaluation.
Familiarity with content safety labeling, policy writing, and red-teaming.
Experience building evaluation datasets and running QA programs.
Basic scripting/analytics (spreadsheets, SQL, Python) for tracking and error analysis.
Compensation Hourly rate: $30–$50/hr (BASE_SALARY). Remote-ready roles across Canada time zones.
