How a study runs

This page follows one study from the first idea to the published data. Your own study may skip steps: a study without teams, for example, has no team or chat steps.

1. Set the study up

   In the panel you create a study and switch on the parts it needs: teams, chat, a shared document, keystroke logging, questionnaires. You write the information text and the consent form in Turkish and English, build the sign-up form, and add the source texts participants will translate.

   A checklist shows what is still missing before you can open the study.

2. Rehearse it as a pilot

   A pilot is a copy of your study with test participants. You can move its clock forward to see what happens on day 3 or after a deadline, without waiting. Pilot data are marked everywhere and never mixed with real data.

3. Recruit participants

   You share the sign-up link. People read the information, answer the eligibility questions, agree to the consent items one by one and confirm their email address. Each one gets a participant code. Names and email addresses go into a separate, locked store; the rest of the system only ever sees the code.

4. Form the teams

   If your study works in teams, the ToolKit proposes teams that are balanced on what you choose (for example language pair or years of experience). You check the proposal and commit it. Each team gets its own chat, each member an account with a nickname, and a shared document is prepared for every task.

5. Participants do the work

   Participants install Stone Guest on their computer and start a session when they translate. Stone Guest records their key presses and which kind of window they are in (the translation, a dictionary, a browser). At the end of the session it saves a file, which they upload on their portal.

   Meanwhile they talk to their team in the chat, write in the shared document, fill in questionnaires and hand in drafts and final versions. Reminders arrive by email or chat when you have set them up.

6. Keep an eye on it

   The panel shows who has uploaded, who is active in the chat, and which files need attention. Quality flags point at problems such as a missing upload or a very short session. You can note incidents and write memos as you go. Every message you or the system send to participants is recorded as an intervention, so you can separate it from what participants did on their own later.

7. Analyse

   You choose which process indicators you will use, and lock that choice before you look at the results, much like a preregistration. The ToolKit then computes the indicators for every session and gives you tables ready for R or Python. You can also replay any session, have coders label parts of it, look at pauses and bursts, rate the translations, and see who replied to whom in the team chats.

8. Report and share

   The protocol report drafts the methods section of your paper from the study's own settings: design, participants, allocation, instruments, analysis plan and every deviation. A dataset release packs the data you choose for a repository such as OSF, with new codes for every participant and a description of every column.

   When the study is closed, the participants' names and email addresses are deleted after the retention period you set. The research data stay, keyed by codes only.

Who does what

Everyone who works in the panel has a role. A role decides what they may see and change.

Everyone signs in with a password and a second factor (a code from an app on their phone).