Senior Test Engineer

Artificial Agency
Location: Edmonton, Alberta (Remote possible for exceptional candidates)

Job Description

Artificial Agency, the leading player-facing agentic AI company for games, is seeking an experienced Senior Test Engineer to help define, build, and operate the quality systems for our AI-powered game agents, engine plugins, internal demos, and customer-facing content.

This is a senior individual contributor role for someone who can work across QA, game design, agent development, and engineering. The successful candidate will help ensure that our Behavior Engine and game integrations are tested across the full development pipeline, from early unit and functional testing through to integration, end-to-end, manual, automated, regression, and customer-facing validation.

The role is not limited to traditional QA execution. It requires technical judgment, strong root-cause analysis, practical test design, and the ability to build shared quality practices across teams. The successful candidate will directly design, implement, maintain, and execute tests, and build the reusable engine- and game-specific harnesses needed to support agent evaluation and regression testing. Hands-on manual and exploratory testing are part of the role, but routine manual test execution is not the primary focus.

You will also help define the systems, standards, tools, dashboards, and workflows that allow the broader team to understand and improve product quality. You will work closely with all of our technical teams (Game Technology, Platform, Agents, ML). A separate evaluation-infrastructure engineer on the platform team will own shared test execution and results infrastructure, working with you to connect game-specific tests to our existing development and evaluation workflows.

Artificial Agency is building agentic AI for games, and this role will help define how intelligent agents are tested, evaluated, debugged, and made reliable enough for production use. You will work with a team of ex-DeepMind researchers, experienced game developers, and product builders on a new category of AI-powered entertainment.

Responsibilities

• Design, implement, and maintain manual and automated tests for agents, game content, plugins, and supporting systems across Unreal, Unity, and other environments, covering unit, functional, integration, end-to-end, and player- and developer-facing validation.

• Partner with technical teams to define acceptance criteria and create in-engine scenarios that validate agent behaviour, task performance, content setups, and configuration changes.

• Build reusable game- and engine-specific test harnesses for scenario setup, reset, execution, outcome assessment, and diagnostics, making new cases easy to add across example maps, internal projects, and customer games.

• Integrate tests with shared infrastructure for local and distributed execution, CI workflows, and headless or rendered runs. Work with the evaluation-infrastructure engineer on interfaces, results, and diagnostic artifacts, and document patterns others can extend.

• Run regular manual and automated regression tests and exploratory testing. Investigate root causes with engineering, verify fixes, maintain reliable test suites, and turn discovered defects into regression coverage.

• Improve testability through game-side tooling, logging, telemetry, and instrumentation, partnering with engineers on supporting systems. Use diagnostic data to reproduce issues and uncover quality risks beyond routine test passes.

• Develop dashboards and reporting with the platform team to track coverage, defect trends, build health, and readiness. Assess and advocate for the quality of internal content, demos, sample projects, and customer-facing examples.

• Establish practical defect-reporting, triage, prioritization-support, and lifecycle-tracking practices. Stay current with quality engineering standards and emerging testing methods, particularly for AI-driven systems.

Quality remains a shared responsibility. You are accountable for the testing capabilities, coverage, and assessments you deliver. Engineering and content teams own their implementations and fixes; product and engineering leadership own prioritization and release decisions. Other technical teams will support you and contribute tests and infrastructure.

Requirements

• An AI-first approach to engineering, with hands-on experience using AI tools for development, testing, debugging, or analysis. You are excited to make AI-driven workflows central to how you work, actively experiment with new capabilities, and adapt your approach as tools improve, while taking responsibility for the quality of what you deliver.

• Experience in game quality, quality design, technical QA, software quality engineering, gameplay systems, tools, automation, or a closely related discipline, with substantial hands-on test development.

• Strong hands-on experience with Unreal, Unity, or comparable game-engine environments. Deep experience in one engine and the ability to work across additional integrations are more important than prior experience with every engine we support.

• Strong programming skills appropriate to game-engine development, such as C++ or C#, and demonstrated ability to build and maintain reusable test harnesses, automation frameworks, or developer tooling.

• Experience designing, implementing, and executing manual, automated, regression, functional, integration, and end-to-end tests for complex software or gameplay systems.

• Ability to create, debug, and maintain in-engine test scenarios using engine code, Blueprints, scripting, editor utilities, automation frameworks, validation tools, or equivalent systems.

• Strong understanding of defect reporting, triage, root-cause analysis, regression testing, release quality, and content validation.

• Technical fluency with logs, telemetry, CI systems, dashboards, automation workflows, and collaboration with engineering teams, including integrating tests with shared development infrastructure.

• Ability to work effectively across engineering, design, product, agent development, and leadership stakeholders.

• Strong judgment about when to use manual testing, automated testing, exploratory testing, telemetry analysis, or designer-driven validation.

• Comfort working in a fast-moving startup environment where systems, responsibilities, and best practices are still being defined.