What is a portable usability laboratory? And could KDE development make use of one? Pete Gordon, an engineer behind such a one, shares his thoughts in this interview on exactly that topic.
(Frans Englich): Pete, feel free to introduce yourself.
(Pete Gordon): Thanks, Frans for having me discuss Portable Usability it is definitely a passion of mine.
Just a little bit about how I got into this. I like most of your readers am a coder; I was thrown into customer interactions when a 120 employee company I was working for went down to six employees of which I was one. The insight that I gained from working with the customers on a regular basis was amazing up to that point I was in the Software Engineering group and we had a couple dozen that pounded out code.
Immediately, I saw from the customers so much that was missing in the functional specification documents, and design documents that were passed into the Software Engineering group and developed from. So much, that we had spent months working on that did not meet the customers needs and that they were not using, and so much that could be done for little or no effort to meet those same customers needs.
At first reading this you may think that the company was slow moving and didn't have a good process in place. Where in fact, we were practicing XP for the last year or so of our large existence and were focused on the "Voice of the Customer" our representative of the customer. I don't think the "Voice of the customer" was necessarily the problem, I think the problem was far greater than that.
But, the fact remains that when developers can talk to, interact with, and see real people using their software it is enlightening. And, that is what lead me to this. I wanted to come up with a way to communicate the user experience throughout the organization (all the way to developers, sales people, everyone). That's the goal of UsersFirst and VisualMark.
What is a portable usability lab, and in what way does it aid usability development?
Portable Usability Labs have been around for a while, mostly made up of large suitcases to transport analog video/audio equipment to remote locations to set up a temporary observation room. Here's some links:
This equipment was very specific, developed and sold only for the purpose of observation setups. The idea that someone can go into the field (into someone's home or wherever) to watch interactions is not new. The most famous comes from anthropology. Watching and Observing Chimps in the wild like Jane Goodall. The art of watching and learning from peoples interactions soon followed in Product Design look at companies like IDEO, Fitch, Frog Design which focus on Product Innovation and Design. You really innovate and get creative thoughts and understanding from stepping out from where you are and watching and learning from others.
It is not surprising with the PC revolution of the 80's that a need to watch others interacting with PCs was needed. That said, we have now moved into a point where watching people is secondary to usability, I believe. The most important thing is communicating those interactions to all involved with a product, to then come up with creative solutions to most accurately meet the users needs. This is a little bit of a different perspective, but similar to Joel On Software's recent comments on Social Interface Design. Bottom line, it aids Communication, allowing you to communicate what you have observed to others involved.
But is it at all necessary? Isn't it enough to test it by oneself, and make sure any Human Interface Guidelines are followed? Why is user testing important?
The answer to this lies in the huge benefit and huge disappointment to free (open source) software. The huge benefit, when developers develop for themselves they already have a deep understanding of the user needs, and the software is amazing in meeting those needs.
Look at Linux, MySQL, PHP, Perl, and so much more (I personally love Mac OS X Desktop Manager--Richard Wareham you are amazing!) these are great examples of where developers develop for themselves, and the community at large benefits amazingly.
The other side is when developers develop for themselves and it only meets a small portion of the user needs; or even worse the code (solution or whatever you want to call the product) goes in the wrong direction from meeting the primary users needs in favor of the few. This is where some software dies on the vine, where other software can push through it just fine, because people want it bad enough they will put up with some inconvenience and difficulty or lack of it meeting the need perfectly.
The key I believe is striking the balance. The balance between the users primary goals and the solution to meet those goals. The way that this balance is reached is by understanding your users, whether it be yourself alone, or the world.
So, to answer your question. Is it all necessary? Depends. If your the only user and you will always be the only user and your writing it yourself--no it is not at all necessary to have others test it. But, if you are to have others use it and you want others to use it, make sure that you understand what it is those others want and don't go it alone. That is why user testing is important to get a clear understanding of user needs.
Great. Here's where I can get into the corporate BS, thanks. The biggest difference right now, is you can download mine and try it out. Please do! I love to get feedback and if you think it would work on a project drop me a line. The product is in beta and I want more and more people to use it.
Other differences include the focus on communicating the experience. The other solutions spit out multiple files that you can edit together or use their proprietary viewer to give to others. I went with Quicktime because I can get everything video,audio, screen all into one Quicktime Movie (still maintaining separate tracks for editing, overlays, etc.) but simplifying the output, thus making it easier to communicate it to others. No matter what lab you look at, the files are huge, capturing pixel perfect screen as video is costly in terms of bytes, so having it all neatly available as a Quicktime Movie that can be burned simply to DVD is a big advantage.
Another big benefit is the multi-platform nature of my solution. I have based the VisualMark Lab on a VNC Client, so it does the screen capture using RFB (Rectangle Frame Buffer) coming from any VNC Server. Although, it may get replaced eventually because of performance, we'll see what the VNC 4.0 stuff is like.
The system is pretty simple in its usage, turn on your inputs (screen,audio,video) log some observation notes and time stamps, close your session, and check out your HTML/Quicktime, maybe make some edits changes at that point, then burn it to DVD or put it on the network were your entire team can view it. I really want it to be simple, and the focus again is to communicate the user experience.
What is the implementation details? For example, what is the data format of the result, and how is the screen capture handled?
Well, I guess I just answered this mostly.
The output is MPEG4 for the videos, I recommend IMA4:1 for the audio inputs (but it can be anything in the QT SoundManager), and the Screen is captured as Animation Codec. Again, these all get thrown into the same multi-track Quicktime Movie file, so the output is simple, big advantage of quicktime there.
The video inputs come from IIDC webcams (like iSight) and the screen comes over the network as RFB (Rectangle Frame Buffer). For Windows previewing/capturing I have built a Rendezvous wrapper application around UltraVNC which allows you to execute but not install UltraVNC Server off of CD, and it is dynamically located by the VisualMark App on the Mac, so there is no need to even know the IP address of the subjects computer.
This is cool, I would like to have it on Linux and Mac also (right now you have to code the IP address in the lab to do those), but there are bigger issues related to performance and mouse movements right now capturing those platforms. Stay tuned, or contacting me about getting involved.
What is the license terms for Visual Mark? It is closed source?
Yes for the time being. : ) But, if people are interested in helping out, and eventually open sourcing it I would love to talk to them about it. I could use some help especially with the low level performance stuff. It is written in Objective-C and C.
Often mentioned but often forgotten is to develop user oriented software, in contrast to only designing for the developers themselves. In what way does Visual Mark put the usability engineer in focus, and make the usage easy and simple?
Much of what I have already said. Single file output for everyone to see and use is definitely easy and simple. HTML and Quicktime are ideal. I think it really empowers "usability engineers" to grab a lot of information quickly, that's why it has multiple camera inputs in addition to the screen capture. If you wanted to just setup multiple video cameras around a focus group, you could get a lot of information about users and what they thought of a conceptual product or prototype really fast, and communicate that information really quickly to others; even if it was just a paper prototype or something.
The key is in developing user oriented software. When we are developing software for others besides ourselves, we need to understand others. And, the Usability Engineer or the developer that has a free couple hours and can meet with users, can capture that user experience and communicate it back to others.
Now maybe Usability Professionals will frown on me saying developers can do it, but I can't help to say they can do it--I am a developer and I do it--a lot. Granted some people are more suited to do it than others.
You need to be able to listen more than talk, you need to watch carefully rather than wanting to lead and teach, and you need to be empathetic to the user. But, if you can do those things why can't you capture the user experience and communicate it to the team. And, all the better if you can do it in a way that they can watch it for themselves, and begin to think creatively also about the solution to the users problems, that's were a lab is useful.
What is the typical development mistakes a usability lab session reveals?
Wow, tough one to answer. I think more than anything here we are talking about qualitative analysis. Where there is a misunderstanding between what the developer has developed or the concepts the developer used and the users goals? Often times the developer develops because he/she can, and users just want to get a goal done. These sometimes line up and sometimes they don't.
I have seen screens extremely cluttered with functionality, when the users primary goals could be boiled down to just a few interaction points. I would say the biggest mistakes in smaller developer led projects are building too much into a product without understanding what the user really wants to do (their primary goals). This comes back to things like Alan Cooper ("The Inmates are Running the Asylum") and User-Centered Design of software. Don't build it without understanding what they want.
Do you think open source projects can make use of such an usability tool, considering their extensive use of online communication?
I sure do, think they can! I would love to see my system grow into a more robust system to help with this need even more than it does. I want things like remote real-time streaming of the information, and chatting among observers, etc. But that is a bigger application, and more robust in its feature set. And, I must stay focused on the beta and performance issues to get to general release before getting into more features.
Sorry, back to your question. Definitely. If anybody wants to use my tool for open source let me know, I want to help where I can. It is really powerful to see people/users interacting with your software it gives you a sense of really meeting peoples needs and allows you to tap into those needs really clearly. It allows developers to be empathetic, really taking on the users concerns, goals, thoughts--and that makes for better solutions.