Monday, 4 May 2009

In search of the perfect App keyboard

Mobile music is cool, but you've got to be able to get what's in your head into your device. Whilst there are loads of "funky" new paradigms courtesy of the iPhone's touch screen and accelerometer, in the main we're looking at piano keys or guitar strings. For this post I'm looking at piano keys. 

When the iPhone was announced I was excited at the prospect of multi-touch for one reason - chords! Although I love my Glofiish's pinpoint stylus accuracy, and my DS's ability to detect touch pressure, neither of them allowed me to play chords on a virtual keyboard. 

There have been many different approaches to the virtual keyboard on the iPhone, so I'm going to go through some of the good practice and pitfalls I have encountered along the way, in search of the best possible solution (that no-one has as-yet implemented).
1. Noise.IO uses a split row design, giving good width keys and access to two octaves. Key length obviously suffers here, but that's unavoidable. Changing octaves requires a trip to a separate screen, which isn't ideal but it does work. There's also a very handy modulation slider area to the side, along with a volume slider. Horizontal slides along the keyboard trigger the next note (I think that is technically called a Glissando). Initial vertical key position affects (I think) "key pressure" and can be modified afterwards with a vertical slide. Overall an excellent solution. In addition Noise.IO
 makes good use of the accelerometer for configurable modulation of just about every parameter when you move the device around.

2. Nlog Synth has a single row of keys with configurable key width, giving from 13 to 22 keys on screen (not sure why they stopped short of two full octaves). Configuration is a key (ho ho) requirement I think - everyone plays in a different way, and I personally have broad fingers. Nlog could easily fit in a second row of keys though, for a performance mode, giving access to 44 keys. To get around the octave issue, Nlog allows you to smoothly slide the keyboard left/right as you play it. Very impressive, but means you cannot slide up and down the keys, as the keyboard moves with you. So close to
 perfection! Nlog also has vertical key position mapped to key pressure, and allows a vertical key slide to control a configurable modulation. This is incredibly expressive. It is "sort of" polyphonic when modulating, but triggering a new key seems to reset the modulation for the held key.

3. Vocoder SV5 takes an interesting approach to the octave issue. In normal play, a horiontal
 slide triggers the next note. If you hold down the octave button though, the horizontal slides scroll the keyboard. Very useful. Also well worth a note is the sprung pitch wheel. I love it! There are other UI innovations I've writen about in a previous post, such as "sticky" rotary controllers. No controllable modulation is a minus, but there is a full-screen dual row keyboard option, which has some useful spare space for, for example, a future mod wheel.

4. iSyn offers a highly playable full-screen
 two-octave keyboard with switchable Legato and multiple scales. Changing octaves is via octave up/down buttons, which I'm not a big fan of. Vertical key modulation works brilliantly although it's tied to the same modulation as the Y axis of the XY pad - I'd like to see an independent option. Interestingly, sliding up and down the keyboard doesn't retrigger the next note, it slides the pitch smoothly even across the width of a single key, which is great for leads and "finger wobble" pitch modulation, and reminds me a lot of Bebot. It would be good to have this optional like it is in Bebot (when set to "Snap" mode). The current note display is a nice touch. By default the keyboard has "virtual wide" black keys. This means you cannot trigger a white key from the black key area (scale dependent), which I personally like.  Wavesynth also has this, but it is optional.

5. Speaking of Wavesynth then, it also features a lovely big sprung pitch bend wheel and a whole heap of configurable options (along with the black key mentioned above): pitch bend range, single or double layer keyboard, "shake to modulate" (which I find a little gimmicky), configurable portamento and switchable mono mode. The only missing option here is configurable retrigger/don't retrigger envelope (Legato). A great little synth but a bit overlooked (presumably because it is touted as a general midi rompler).

6. Megasynth. Keyboard is very basic, and lacks a lot of the features mentioned in this article, but it is worth noting the slidable octave display that lets you see and choose your keyboard range.

7. Pianofly. A new App, and a great sound engine, but what I really want to mention here is the keyboard. Switchable to full screen is nothing new, but I absolutely love the novel approach to the octave issue. Playing the keyboard is similar to Nlog - silky smooth scrolling left/right - but you can "hold" the keyboard still with another finger (either on a key or on the black scroll area at the bottom) and now your other fingers are free to slide up and down the notes. Genius.

8. Grand Pro. An honourable mention to this app for offering optional 1, 2 and 4 octaves on screen at once! Also has optional note names on each key, which is a useful feature.

Ok, there's loads of apps I've not mentioned, and I may have made a few mistakes for which I
 apologise but hope you can correct me.

So what would make the perfect keyboard for me then? Hmm let's see:

  • Configurable key width
  • Configurable split/single key row
  • Pianofly's scroll lock
  • Optional note names (Grand Pro and makes a brief appearance on Wavesynth)
  • Initial vertical position configurable to a parameter (would generally choose pressure, but options are nice)
  • Vertical slide configurable to a parameter (Nlog)
  • On screen pitch wheel with configurable range (Wavesynth and Vocoder SV5)
  • On screen mod area (Noise.IO)
  • Optional "virtual wide" black keys (Wavesynth and iSyn)
  • Optional smooth slide portamento (I don't know what to call this as it doesn't exist on a real keyboard, but on a fretless bass or trombone! - iSyn and Bebot). It is different to normal keyboard portamento as it is controlled by the player, not a predefined time.
  • Configurable Legato (iSyn)
  • Configurable scales (iSyn and Bebot!)
  • Optional mono/poly (Wavesynth)
  • Configurable portamento (Wavesynth)
Maybe I'll mock-up a few screenshots of a keyboard and settings screens?


  1. Excellent review. I found that really useful. I hope some developers read it too.

  2. Thanks for that "Genius" comment; good motivation there!

    Why don't users ever ask for fewer features?

    In general, there are two schools of thought on "configurable" design. The school I belong to is this: design is about making decisions; it is a mistake to just make everything configurable and leave all of those decisions to the user. That isn't to say you can't take a great design and then add some options after the design is finished, but start with the good design.

    To be more specific, Being able to access 4 octaves on screen would be great, but given the size of the human finger, it isn't ever going to be a great experience on the iPhone. You're going to have to do a lot more convincing before I'd consider letting users make the keys any smaller in Pianofly.

    We've been looking at a "double keyboard" option (instead of full screen) for Pianofly, but this would be more about mixing two timbres. Some options for controlling key pressure or after touch are reasonable, but not something we wanted to surprise first time users with.

    This will all get a lot more interesting if Apple ever gives us access to the raw touch data so we can start approximating touch pressure...

    Thanks for the thoughtful article.


  3. Rob ... just learned of your blob via Palm-Sounds. Great to see it. There is great need for more good iPhone/iPod touch + Music blogs! Looking forward to lots of great posts.

  4. Thanks for the comments. Chad, I do agree entirely with your views on design being a process of making decisions. Sometimes, of course, simplicity and/or minimalism can help to inspire creativity. Anyone who says otherwise has never seen a great black ink sketch on a piece of paper.

    Grand Pro does do a good job of getting more octaves on screen, but it's an option - that is what impresses me most about it.

    Users will never ask for less, because we're not the ones who have to program it :o)

    Glad you've got future plans for Pianofly, and I look forward to Apple "opening up" a bit too. External midi via the dock port? Count me in!

  5. no Janko layout!? iphone would be excellent for Janko layout.

  6. You missed FingerBeat keyboard!