About Starya
Starya develops AI for complex operations. We connect artificial intelligence, software, and data to transform real business processes with security, reliability, and control.
Our platform, NebulaOS, is the foundation for building, integrating, and operating these solutions. Our work combines engineering close to customers, product development, and a foundation of data, infrastructure, and security.
The challenge
Help the team understand the risk of a change and verify product behavior before and after delivery. An AI operation involves responses, decisions, and actions: evaluation must reach the result in the target system, the permissions used, and the recovery paths available to people.
What you will do
- Build a risk map and a test plan for each journey with PM, Design, and engineering. Prioritize situations with the greatest operational effect, such as an improper approval, a duplicate action, or an execution without confirmation, linking each risk to the necessary checks.
- Automate interface, API, and integration checks according to risk. Check state in the target system and observable effects; simulate a timeout after a write, repeated events, and resumed execution to verify duplicates and handling of unknown outcomes.
- Test authorization boundaries and isolation between organizations with appropriate data. Cover context switches, references to another tenant’s resources, and attempts to execute actions without permission, recording evidence that enables reproduction and correction.
- Deliver versioned datasets and contract and regression suites integrated into CI. Prepare known states and investigate flaky tests, separating environment failures from product failures so that results can inform change decisions.
- Connect AI evaluation and functional quality with AI Research. Use reference cases to observe incorrect responses, missing information, and inappropriate tool use, distinguishing acceptable variation from violations of rules, permissions, or outcome criteria.
- Investigate production issues with service owners. Deliver a reproduction, expected and observed results, and a regression case. Complement automation with exploration and report completed checks, limitations, and risks that remain for the release.
What we look for
- Experience testing applications and integrations, with the ability to derive scenarios from rules and risks. Examples of defects found and how the evidence guided a fix.
- Ability to code test automation and organize repeatable checks. Understanding of data preparation, scenario isolation, and diagnosis of failures in the test or product.
- Knowledge of APIs, states, and operational effects to investigate differences between responses and results. Ability to discuss contracts, repetition, and failure conditions with developers.
- Ability to reason about authorization, isolation, and test data, using appropriate environments and information. Attention to evidence traceability and protection of sensitive content.
- Clear communication of risk, reproduction, and coverage while working with other specialties. Ability to prioritize checks by the effect of a change and revisit the strategy based on usage.
Additional experience
- Experience with Playwright or equivalent tools, contract tests, and CI pipelines, with examples of suite maintenance and investigation of intermittent results.
- Experience evaluating AI applications, versioned datasets, or probabilistic behavior, connecting model analysis with verification of product journeys and actions.
- Experience with accessibility, performance, or event-driven integrations, helping select specific checks when these factors present a material operational risk.
Your impact
When repeating a call after a timeout, the suite verifies that only one change occurred in the target system and that the interface shows the correct result. Cases like this make important failures detectable during delivery. Tracking escaped regressions and flaky tests guides improvements to coverage and automation.
Who you build with
Quality is built with SWE, PM, Design, and Client Engineering throughout delivery. You share cases and criteria with AI Research while preserving its dedicated evaluation of models and methods. With SRE and security, you connect incidents and operational risks to verification strategy and regression prevention.