Jen Simmons’ HTML5 APIs session was an interesting look into the back-end of HTML5, an area that I’m not as familiar with. Lots of new goodness about how we can make HTML not only create a basic layout but actually devise interactions, storage and flexibility.

Presenters: Jen Simmons (@jensimmons)
Hashtag: #html5apis

Subway as a Competing Standards Metaphor

In NYC, part of issue was that 3 competitors were creating own subway lines – created confusing gridwork of paths

HTML5 API for Communication – Web Socket

  1. Original web needed http, URL and HTML – http wants the web to be a brochure rack
  2. AJAX allowed asynchronous updating
  3. what we want is a open connection = web socket
    • Doesn’t replace http – supplements by sharing an experience between two (or more) windows
    • Ex. Page on mobile acts as control for web page on desktop
  4. What can you do?
    • Real-time updates
    • Multiple people sharing a page
    • One person using multiple web windows on multiple devices at the same time
    • Ex. Multiple people watching a movie together as though it was a shared broadcast, incl. one person pausing for group
    • See “The Web Ahead #5″

HTML5 APIs for Storage

  1. Web Storage
    • Previously would create HTML, images and FTP it up
    • Then we went into separating HTML, CSS and data (style from content from database)
    • Web 2.0 – people could add information to your site
    • Web 3.0 – databases on the servers AND on the client side
    • Local Storage, Session storage, key|value pairs, IndexDb, WebSQL
    • What’s it for?
      • Allowing users to save data without an Internet connection
      • WordPress saving locally as a scratch pad until upload to main server
  2. Application Cache
    • Allows users to continue to use a site even without connection to web
    • Can download the pieces to work offline
    • Great potential for preventing data loss on connection loss
  3. Files
    • Web seems separate from computer – this helps connect them
    • File API, File Reader / Writer / System, Blob URLs / Blob Builder, Drag and Drop -gives a website the chance to understand file concepts
    • See “The Web Ahead #14″

HTML5 APIs for Content

  1. Device APIs
    • Websites are able to directly interact with devices
    • Ex. Vibration API on Webkit
    • Website can also read data FROM device (volume on/off, orientation, battery life)
  2. WebGL
    • Allows complex graphics to be rendered directly in browser – tTop use idea is gaming, but what are other applications?
  3. Vehicle API – to interact with car data so car could send diagnostics or mpg data but could also help people do self-diagnostic

“An innovator isn’t someone who creates something out of nothing. An innovator is someone who wakes up to the constraints caused by false assumptions, and breaks them.”

My Tweets During Session

  • @jensimmons says Web Socketing is a way 2 augment FTP using a bidirectional, always open connection. Think auto-refresh on speed #html5apis
  • Application Cache allows use of a site even without connection to web; great potential for preventing data loss @ connection loss #html5apis


SxSW: Teaching Touch (Josh Clark)

Josh Clark’s presentation on touch design was one of the better presentations of the week – a combination of humor and good information, presented in one of the most charismatic sessions I saw. As I said after:

After seeing @globalmoxie speak, I need to tweak my #SxSW presentation with more humor/fart jokes

Here are the highlights of the session.

Presenters: Josh Clark
Hashtag: @globalmoxie

In the coming years, interfaces will lose the cursor, leaving only you and the device – or you and the content.

  1. Traditional point and click interfaces suck on large screens (Fitt’s law)
    • Even on iPad email have to click small ‘back’ button several (million) times
    • Buttons not always wrong – gives affordance of action but gestures can take over for some things AS SUPPLEMENTS
    • Gestures are the keyboard shortcuts for touch

Big Screens Invite Big Buttons – buttons were needed as an abstraction for functionality and have a role still but there may be other, useful methods like touch, voice, etc

  1. Win8 supports gestures – goodbye passwords: Win8 will allow patterns on top of a picture
  2. Twitter on iPad almost did away with buttons entirely – can sweep instead (card metaphor)
  3. We need two things to make gestures on the web workable – real support for gestures across browsers and libraries, and gesture conventions.
    • JavaScript only has limited gesture options – tap, swipe
    • Conventions have yet to be established – apps all have own conventions, web apps have to avoid interfering with OS standard gestures
    • JavaScript libraries can add some gestures
      • JQuery mobile adds 3 gesture controls
      • SenchaTouch adds 7-10 more for Webkit only
      • Touchy.js – library adds function

Experiments in Touch

  1. IKEA app concept –
  2. Clear app –
  3. TouchUp app ex. – to change brush size, can’t use pressure because inexact; this app uses pressure to change canvas size instead.
  4. Massive Swiss army knife example – Clarity trumps density; with design, more features = more controls

How do users discover gestures?

  1. Inherently invisible; the less it resembles a physical action, the harder to discover
  2. Google maps – 2-finger tap zooms out – how would you know that?
  3. Importance of Visual Cues – theOCDChef cutting board ex.
  4. Many aps often start with instruction screens on apps – have to then remember them, plus no context to the instructions
  5. Should include reference, but don’t make it a learning guide
    • Best interfaces don’t need instructions
    • Don’t make me use buttons if you create a gesture-style interface; if it looks like a physical object, people will try to interact accordingly
    • Magazine examples are too literal – long swiping PDFs, but remove key pieces like a gesture to get to TOC
    • Voice Memo app = bad metaphor for interaction: entire activity is focused on single small button in lower corner
  6. Amazing how quickly toddlers pick up interaction with an iPad – very young and very old pick it up quickly because they don’t have learned bad habits from years of cursor-based use

Play More Video Games

  1. Coaching
    • Basic instructions and demonstration, practice
    • Help them in the moment (overlays with instructions)
      • Web version = coach marks
      • Clippy issue: bad advice at wrong times
    • Gesture without a visual aid is a suitcase without a handle
  2. Leveling up
    • Teach when a skill is needed vs all at once – think of app having levels, and when you need to lead
    • OS Lion scrolling ex. Upside down from previous version
  3. Power ups
    • Areas where you can add skills for expert users
    • Twitter app ex. – make them learn the long way, but after a time introduce a better way

My Tweets During Session


SxSW: Designing for Context

It’s been almost a month so I thought I would finish doing write-ups about the sessions I attended, going in chronological order and following Luke Wroblewski’s bullet point method to simplify it.

Presenters: Ben Fullerton – Method (@benfu), Leah Buley – Intuit (@ugleah), Nate Bolt – Bolt|Peters (@boltron), Ryan Freitas – AOL/ (@ryanchris), Andrew Crow – GE (@AndrewCrew)

Hashtag: #DforC

In 1995, the Decision Theory and Adaptive Design Group at Microsoft studied the ultimate frustration point for users, which resulted in “Clippy”

  1. Clippy was diseigned for a single context – today there are far more contexts to design for.
  2. Ways to design for diff locations, environments, relationships and product ecosystems
  3. Weirdest contexts?
    • ??: couldn’t talk to the users at all
    • NB: astronauts in space (apps in space and shuttle): had to deal with zero-G, cosmic rays require frequent replacements of tablets
    • LB: checking finances on the toilet
    • LB: take a browser-based context into mobile context: advantage of having data of desktop usage to inform mobile
      • reject 1:1 context mapping
      • add contextual elements (location)

What should we be aware of when designing for context?

  1. Time (length or instances of time)
    • LB: thought of linear time w/o interruption – but people get distracted AND multitask
    • NB: working with NYT on news contexts – intercepting users to understand time-based contexts, resulting in NOW as an important time.
      • BF: need to understand longer tasks but also microtasks
      • AC: worst idea is that it must be the same design for beginner and advanced user
  2. Ecosystem: how devices interact with other apps, devices and even competitors
    • RF: TeachingChannel – understand time issues teachers have and solve for those
    • LB: Mint – how to bring from desktop to mobile, or those who are coming directly from mobile (using app) so may not have previous experience to leverage.
  3. Location – frame of reference of external factors
    • a) Victoria’s Secret made unique experiences for iPhone and iPad: iPad was browsing, picking at home; iPhone: don’t bring iPad with them but gave way to photograph tag to get more information on product
    • think of location in a radiating circle instead of absolute position
    • LB: if designing for use outside, make sure you can actually SEE them outside
  4. Form & Technology
    • Often start at lowest common denominator (Mobile First) – good in general principle but lowest common denominator is not always a mobile phone.
    • LB: likes to do opp. thinking of of devices’ extra capabilities first – location, camera : leads to other opportunities.
  5. Brand & Relationships
    • BF: brand is usually viewed with suspicion when they try to establish a relationship
    • Bedsider example: contraception tool that goes beyond simple info and lists – did fact or fiction videos “Douche and Dont’s”

What have we learned?

  1. If you’re going to extend, figure out which pieces are most important at each contextual point
  2. Adopt a service mentality to figure out points of synergy between contexts
  3. Expose and understand he metrics you have to find intersections
  4. Find ways to truly get into real users’ context
  5. If you try to design FOR users outside of knowing context, you will miss something

Dealing with User Frustration

  1. Messaging within context – but can be rude to tell the truth
  2. Can turn it into a design aspect -ex. show what you can do, then interrupt when absolutely necessary; people respond to clarity and honesty

Users/Contexts on the edge -how to move between contexts

  1. Where experiences break down most
  2. Clarity and directness is best policy (guided transition vs. forced)
  3. Add personality to cushion the transition

My Tweets During Session

  • Think of location in a radiating circle instead of absolute position (city, store) #DforC
  • 5 aspects of designing for context: time, ecosystem, location, form & technology, brand & relationship #DforC


Official page:

Lanyrd logo Designing for Context on Lanyrd

SxSW Accessibility Survey

As I stated here, I have the honor of presenting “How the iPad Can Help Save Accessibility” at SxSW 2012 in March. In preparation for this talk, I wanted to get a better idea of how people with different disabilities cope with the web in general, and how they deal with the mobile web. I also want to get an idea of what issues non-disabled people have with the web/mobile web to help contrast it.

If you could help me out, I’d greatly appreciate it.

Accessibility Surveys

For users with Visual Disabilities »
For users with Hearing Disabilities »
For users with Mobility Disabilities »
For non-disabled users »

Thanks for your help!

Update 1-20-2012

Thanks for the response so far – I’ve gotten some good information but looking for more answers.

Non-disabled: 20 entries
Vision disabilities: 29 entries
Hearing disabilities: 11 entries
Mobility disabilities: 8 entries

Also, if anyone is having trouble with the survey itself accessibility-wise, please leave a comment here or on the form. I tried using Wufoo because they seemed the most accessible of the survey services but some blank entries have me wondering.

SxSW Bound!

I’m really excited – I was chosen to speak at the 2012 SxSW Interactive Festival next year, presenting How the iPad Can Save Accessibility. I got the notice some time ago, but things have been so hectic that I haven’t gotten around to posting it. I tried for both SxSW and IxDA’s Interaction 2012 in Dublin, Ireland¹, and received my notice of success just after I saw my name on the announcement list.

Here is the original synopsis I came up with (which ended up being shortened to fit in the 1,500-character maximum):

How the iPad Can Save Accessibility

A long time ago, in a galaxy before HTML5…

Usability has come a long way since the dark days before “Designing with Web Standards”, with advocates pushing web usability to the forefront of almost all web design projects. Now, nearly all companies see the value of UX in their digital designs. But despite heightened focus on user-centered development, accessibility hasn’t had quite the same level of acceptance – noted, understood perhaps but ultimately implemented as a matter of convenience when it didn’t affect ROI or timing.

So, those who have accessibility needs have persevered and pushed, gaining traction as advocates pushed the envelope in creating sites that were beautiful, compelling AND accessibility. Still, users struggled through image-heavy sites missing alt tags, functional sites that required a mouse to use, designs that they couldn’t read and videos they couldn’t understand because there was no text alternative.

Enter smartphones like the iPhone and Android … and then the iPad. With the proliferation of non-desktop devices and browsers, suddenly a more people were finding that the web wasn’t as nice and clean as they remembered. Broken formatting, too small text, buttons and functionality didn’t work because they couldn’t hover. And entire swaths of the web rendered as Flash-based wastelands that millions couldn’t access.

The people cried out and developers listened, and things began to change. They called them ‘iPhone versions’, they tested in a wider range of browsers, they made things work better. And strangely, when they made these fixes, they actually also made things better for accessibility. And as we’ve figured out how to make our sites & web apps work better on the new splinter web, we’re figuring out that helping our iPad users helps all users.

For better or worse, by solving for many of the issues that iOS and other mobile users have with our websites, we can address the same needs that have always been there but are now more exposed. Better yet, we can take advantage of the accessibility capabilities that are built into mobile devices to in some cases make these newest devices in some cases better than the old web.

I’m pretty stoked to get back to SxSW (I had a great trip last year), and hope that I can come up with an engaging presentation of my own this time around.

Update 12/30/2011

I just put up a survey to find out about people’s habits on the web and mobile web, and more specifically about how the disabled use the web/mobile web. If you have a short bit of time, I’d love to get your feedback.

¹I submitted 2 proposals to IxDA including this same one and another called “Twitchers & Twitter: What Birding Can Teach Us About Content Strategy”.

Mobile Lab Testing Apps

When I had my recent TIA incident, I was put on blood thinners that require me to be tested regularly to measure my levels. To get a record of my labs, I went out looking for a lab results app (specifically one for Quest Diagnostics) and I saw a mention of a new app they had called Gazelle (Why? Got me). I figured I would try it out.

Gazelle App

This app was designed to help users have a medical record on their phones, including medical history, emergency info and lab results:

Much of this information is saved so that it can be accessed by anyone (such as EMTs and the like), although they would need to know you have the app.

Other information is locked behind a password system, though I’m not entirely sure what use it would be without the password, except for one key thing – the most important factor for getting this app, which is that it would allow me to get (and store) my lab results. This is accomplished through the “Lab Results” section via two different steps.

First, you have to request the lab results by entering the date and requesting physician, which sends a request to the lab to have the results posted to your account. It takes 2-3 days, but once the results are available you receive an email notification, and when you log back in, you see the results of your tests, as well as a history of previous tests.

So, how’s it work?

1) Medical Information
Entering your information is a multi-step process with form fields NOT formatted to gracefully accept inputs from a mobile device (such as dates, numbers, etc.). But it serves its purpose.
Information display of key medical information is good, showing the key information needed in an emergency (image, right)
2) Lab Results
Getting your lab results is fine, and the information is handy to have for physician visits. I have only been getting a single type of test result, but theoretically different types would have different displays and information.
I don’t understand why a mobile app only gives you e-mail based notifications; there are no push-based notifications from the system.
3) Other
Login – Understandably, you have to log in to get to certain information, including the lab results and some medical data. But unfortunately, the system doesn’t have multi-tasking support, so leaving the app automatically logs you out, which is somewhat annoying considering the “Remember Me” checkbox
Toolbar – The toolbar has 3 icons: Home, Share App and Share Information – but none are labeled to let you know what they are. Clicking either of the latter two results in you being taken to an email client (which also logs you out of your account).
Make an Appointment – Strangely, this link takes you to a website to download a completely different application (Quest Appointment Scheduler). It would make more sense to have this incorporated into the application, in my mind.

Overall, this app is nice but has some serious UX issues that need to be fixed in order to make it more useful. It’s a free app, so it’s simple to get for your iPhone or Blackberry models, and soon for Android.

Final Grade: B-

Quest Appointment Scheduler

So, since I have to go to several appointments, that sent me to looking for a way to schedule my appointments so that I could avoid the long waits at the lab. This second app has that more singular purpose – to make appointments at Quest Diagnostics’ offices. If you’ve ever been to Quest (or any other lab, I think), you know that having an appointment can cut a significant amount of time off of your visit (sometimes over an hour). While many people can’t really take advantage of this, as they usually don’t have time to make an appointment beforehand, people in my situation find it valuable.

Step 1: Location
Upon loading the application, you select to search for a local Quest Diagnostics lab via geo-location or Zip Code, City or State.

Step 2: Type of Test
Step 2 involves selecting what type of testing you need done, such as routine labs, glucose tolerance, pediatric testing, drug screening or ‘blueprint for wellness’. After selecting one, you can choose “Find location” or “Make an appointment”, although they are basically two paths to the same end result.

Step 3a
If you choose “Find Location”, a map comes up with pins for all nearby locations, although it’s hard to select them easily. A list of locations can be opened via the menu, which is much easier.

Step 3b: Calendar
Next, a calendar comes up where you select the location and time you want to make the appointment. The interface of this is a bit confusing – there is a way to bring up a more standard calendar (by clicking the date) but it’s not evident, and clicking the arrows is slow as it checks each date for availability. You then select the location (unless you did that first) and it takes you to the appointment details screen.

Step 4: Confirmation
Once you make your appointment, you see a confirmation screen. However, if you need to make another appointment it takes you back to the beginning of the process – and remembers nothing you did before, a huge failing for someone who needs to make several appointments at once (admittedly a fringe case, but something that should be easy enough to do). Strangely, the online version of the process does the same thing – maddening.

Also, there’s an option in the toolbar for “Appointments”, but that only shows your appointments for the current session – once you leave the app, the appointments disappear (and half the time, even the in-session save doesn’t seem to work). This is likely because this app has no login, so they can’t protect privacy but that is another reason that integrating this functionality into the Gazelle app makes more sense. Similarly, the app doesn’t save the appointment to any calendars, including your iPhone or Google calendar.

There are two other toolbar options that have little value to the user. One is the ability to share the app with friends, which I doubt many people would do. The last is information about the app, but little of it is of use to the user.

While this app has some utility, its limited functionality means that most people will quickly remove it after one use. With the addition of some minor functionality – such as saving your previous location while making a second appointment, or actually having information appear in your “Appointments” list, it could be more valuable.

Final Grade: C-

Other Apps

I went out to see if other companies have similar apps. The biggest competitor for Quest Diagnostics is LabCorp doesn’t seem to have a consumer-based app, although they do have a Lab Test Results app for physicians (as does Quest Diagnostics). I couldn’t find any other similar apps.

Afterthoughts of IxDA10

Just a quick summary of things I saw, heard and learned at IxD10 in Savannah:

  • “Meaningful Interaction” is my new mantra
  • iPad = running joke
  • UX folk may be considered geeks by some, but they are just as highly friendly as they are intelligent … and can drink with the best of them
  • Read more

When the ‘Standards’ Are Wrong

I was doing some research for a Usability/Accessibility Guidelines document I’m developing at work, when I stumbled across a peculiar piece of information while searching for reference sources. In general, the body of work that is presented on the website is pretty good; getting a little dated in spots, but overall very good. Read more

CSS3 Attribute Selectors Query

The other day I was working on an internal UX Guidelines website and instituting one of the very things that I was advocating when I ran into a problem. Essentially, I was implementing the code to create a file type icon at the end of links so that all PDFs would show a PDF icon, all SWFs would have a Flash icon and all external links would have an external link icon, using the CSS3 attribute selectors. Read more

REVIEW: Web Form Design – Filling in the Blanks

Web Form Design - Filling in the Blanks by Luke Wroblewski I just got done a great book on building usable forms – Luke Wroblewski’s “Web Form Design – Filling in the Blanks”. It was a really well-done, interesting book on a less than interesting topic (well, to most people) – in fact, it’s one of the best usability books I’ve read in terms of giving quick, easy-to-read, usable information. From my review on

“Good or bad, there aren’t many books that I can use for my job that I go through quickly. There’s just something about a limit to my absorption of information from these books that makes me take my time to get through them. However, that was not a problem with this book. Chock full of good information, Wroblewski manages to make it a quick, easy and yet informative read that only took me 2 days cover-to-cover.”

You can see more on Luke’s website, to get a feel for the book.