A rebuilt rehearsal experience

Why we rebuilt it

The original rehearsal system was responsible for several things at once: playing the scene partner, knowing whose line was active, opening and closing the microphone, recognizing speech, deciding when a line was finished, preparing Line Notes, and responding when the actor paused, skipped, or rewound.

As LineCheck grew, improving one part could expose delays or disagreements somewhere else. A previous line might still be finishing its work while the next line was beginning. Speech recognition could delay the next cue. A correct reading could produce an unnecessary Line Note. The screen could briefly show a state that did not match what the microphone was actually doing.

Rather than continuing to add fixes around those symptoms, we rebuilt the rehearsal system so every part follows the same current line and the same rehearsal state. That is the main reason this update took longer than a typical release.

How we chose the new speech-recognition approach

During development, we tested the speech-recognition systems available on iPhone and Android devices, the fully local Sherpa system, Whisper, and several approaches that combined them.

We compared how each option handled:

• Long speeches and short lines
• Dramatic pauses and changes in pace
• Names, locations, and unusual spellings
• Similar spoken forms such as “O.K.” and “okay”
• The speed of moving from one line to the next
• Incorrect or unnecessary Line Notes
• Consistency across different phones
• Keeping rehearsal speech private

We ultimately chose real-time Whisper for private speech recognition and Line Notes, using open-source technology that runs directly on the phone. We then adapted it specifically for rehearsal rather than ordinary dictation.

A separate on-device script follower helps LineCheck decide when a spoken line is far enough along to move the rehearsal forward. Keeping that timing decision separate means Whisper can prepare detailed Line Notes without holding up the next cue.

LineCheck can use the expected script line to better understand what the actor may be saying. For unusually long lines, it preserves the opening, ending, and important spellings without exceeding what the recognizer can use effectively.

What feels different

The rebuilt system follows each rehearsal line as one continuous flow: from the scene partner’s cue, through your response, and into the next cue.

• A scene beginning with your line now waits for you to press Play.
• LineCheck can begin preparing to listen while the scene partner finishes speaking.
• Line Notes are prepared separately so they do not make the next line wait.
• Pausing, skipping, rewinding, leaving the scene, or retrying stops unfinished work from the previous line.
• An earlier line cannot advance the scene or attach feedback to a newer attempt.
• The microphone and waveform more closely reflect what is actually happening.
• Scenes with available partner audio no longer incorrectly ask you to regenerate it.
• Spoken equivalents such as “O.K.” and “okay” are treated consistently.
• Long lines retain the parts of the script that are most useful to recognition.

What has been tested

The rebuilt system has passed the complete automated test suite and focused speech-recognition tests. It has also been exercised through simulator and physical-device testing.

That does not mean physical-device acceptance is complete. This TestFlight build begins broader testing on iPhones with real actors, voices, scripts, pacing, and rehearsal conditions.

One of the remaining measurements is the time between the scene partner finishing and the microphone being ready. Our goal is for that handoff to happen in no more than a fraction of a second. We will measure it from the actual TestFlight build rather than claiming it from simulator results.

Other improvements included

Because the previous TestFlight build is older, this version also includes improvements previously delivered through smaller updates:

• Recovery when scene-partner audio is missing
• More reliable libraries and scene lists
• Safer cleanup of protected shows and local rehearsal files
• Clearer scene-upload and scanning steps
• Improved privacy information and support reporting
• Clearer Line Notes and rehearsal review
• More reliable off-book rehearsal credit
• Better handling of protected-script access