New Direction for Acknowledgement

This is a continuation of my previous post.

I talked with one of my beta testers and asked him what percentage of the clips he’s consciously aware of. He said it was maybe only 20%, which I thought was discouraging, but he pointed out that it’s 20% more awareness than he would have without the tool.

Still, I want to improve on that. That reveals a flaw in the tool, in that there’s no way to measure how successful it is. Currently, the only way a user has to tell if the tool is working is through clip retirement. That is, when you feel that you’ve learned or internalized a particular clip, and it’s no longer useful to you, you can mute it or “retire” it to a separate session that you don’t play. This measure of success is rather coarse, and I would like something more fine tuned.

I called this feature Acknowledgement. When it’s enabled, you have to consciously acknowledge each clip as it plays.

My first design implemented this as a tap-to-acknowledge feature. After each clip played, you would have to tap the screen to acknowledge that you heard the clip. I knew this would be a little tricky. After all, one of my original design goals for Reiterate was that it wouldn’t get in the way of your game. I wanted it to be hands-free, because if you have to take your hands off your keyboard or mouse, even for a moment, it would be distracting at best, and possibly degrade your gameplay at worst.

So I made it as easy as possible to tap. The tap area on the screen was positively huge. It was impossible to miss. And I made the timing very forgiving. You could even tap the screen before the clip had even finished playing, and it would record the acknowledgement.

I tested it myself, in game after game, and wow was it awful.

No matter how I adjusted it, or how easy it was to tap, even taking your attention off the game for a moment just wrecks your focus. Reiterate had gone from a tool that helped me, to a tool I actually resented. I would shudder with revulsion every time I started it up.

I wanted to make it work. But I eventually had to admit, tap-to-acknowledge was just a bad idea.

Still, I was reluctant to give it up completely. I still liked the idea of acknowledgment, but I didn’t like the implementation. Was there any way I could fix this?

And then, I got an idea. What if, when you used Reiterate, you actually… reiterated? What if, instead of using your fingers to acknowledge, you repeated the clip back to the app?

In order to make it work, I’d have to bring the the Speech Recognition framework. Then the app would be able to tell what you were saying, and match it against the clip. It would be completely hands-free. It would be even better than hands-free. I was reminded of a Reddit post, several posts actually, where players had mentions that vocalizing their actions had helped them play better. Some people imagined they were streaming to an imaginary audience, explaining what they were doing and why, and it made them play better.

That makes sense from a narrative conscious point of view. My favorite philosopher, Daniel Dennett, believes consciousness evolved through a kind of speaking to ourselves. When you voice out loud what you’re doing, you’re forcing yourself to become conscious of that action. It’s the opposite of autopiloting.

Bringing in speech recognition adds a lot of complexity to this one optional feature, but I think I’m on to something here and that’s what I’ve been working on these past few weeks. I already have the app transcribing existing clips, and soon I hope to have it recognize my reiterations live as I play.