Turnkeel GitHub (opens at launch)

Docs The app

Testing

Testing holds builds and their test steps. An agent or a member posts a build, testers tick steps and write what they saw, and the feedback goes back to whoever posted it.

How a build gets here

A build is a shippable artifact with its test run, such as testflight-46. Its steps are grouped in features. A build is posted in one of three ways:

  • an agent calls post_test_steps with the build, its features and their steps;
  • turnkeel post-steps <file.json> posts the same body from a file;
  • turnkeel post-steps --template <file> --build <key> posts a step template from the repository (below).

Each needs testing:post for an agent. Posting the same build again updates it. Steps are matched by their key: a changed step is marked changed, and a step that was left out is marked removed, with its notes kept. An agent can attach its own run of a step, a pass or a fail with a note and screenshots, which testers see under the step.

A new build is given its testers: the people with the Tester role on the project, or, when there are none, the posting agent's owner or the posting member. A Member, Admin or Owner can change the list in the Testing header.

Step templates

A step template is a test run written once and posted for each build. It lives in .turnkeel/steps/:

---
title: First run
platform: ios
---

## Onboarding
- id: onboarding

What a new person sees before they have an account.

### Open the app for the first time
- id: first-open
- expect: The welcome screen shows. Nothing asks for a sign-in yet.

Each ## heading is a feature and each ### heading a step, with expect saying what should happen. Every one needs an id. Post it for a build:

turnkeel post-steps --template .turnkeel/steps/first-run.md --build testflight-46

--title, --version, --note and --platform are optional. A new --build starts a new run with the same steps. turnkeel sync does not send step templates; a template is not a build until someone posts it.

The Testing view

A build switcher, a header that says "X of Y steps done, A of B features done", the features in a rail (a strip across the top on a phone), and the chosen feature's steps.

A step shows a tick with its number, its text, "Should happen: …" when it has an expected result, "Updated" when it changed since you last saw it, the agent's own run, the replies, and a box: "What did you see? Saved as you type." Each person ticks their own steps, or a whole feature with "Mark feature done". Members see who ticked what.

Notes on steps that have left the build are kept under "Feedback on steps that have since been removed".

Notes are never lost

A note is saved 700 ms after the last keystroke, when the box loses focus and when the page is hidden. Until the server confirms it, the text stays on the device and is tried again. The box says "Saving…", "Saved 14:02" or "Not saved yet. It stays on this device and keeps trying." If someone else's device changed the note meanwhile, both texts are shown and nothing is overwritten. The same rule covers every text saved in place: a card's title and body, notes on assets, a post's text, a release's changelog and notes, and a decision's context.

Feedback back to the agent

The header counts what is new on each side: "3 new for the agent, 1 new for you".

  • For the agent. A note or a mark is new while it is newer than the build's read time. read_feedback (or turnkeel feedback) returns what is new, each with its step's text, and moves that time; --peek reads without moving it, and all returns everything. The read time is one for the build, shared by all agents: feedback is read once, by whichever agent reads it first.
  • For a person. A reply, a new step or a changed step is new until you open that feature.

A step waits on each tester who has neither ticked nor written on it. Then, while there is feedback the poster has not read, it waits on whoever posted the build. A member who posted it clears that with "Mark feedback read". Agents answer under a step with reply_to_step or turnkeel reply.

Testers

A Tester is invited to one project by email and sees only Testing for it: the builds, their own notes and marks, and the replies under steps they wrote on. They do not see other testers' notes, any other tab, the member list or the activity log. A Tester's Testing count is the steps that wait on them.

Only a person can archive a build. Where the workspace has set up the assistant, the header also has Summarise feedback, which shows a summary of what testers reported for the person to read; it saves nothing.

The source of this page will be on GitHub when the repository opens. Something wrong or unclear? Write to us.