Posts by Scott
Progressive Enhancement in Practice
It's one thing to talk about the importance of progressive enhancement. But how does it actually work in practice? Let's take a look at a specific example from a recently launched site, the Adaptation Learning Mechanism.
One of the key functions of this site is to find content by geographic location. The content is related to locations primarily with the location module. A hierarchical taxonomy also places content in broader regions, associated with location's broadest specificity, country, via taxonomy synonyms and a little nodeapi glue.
Views exposes all of this as filters on a list of content and we have a functional search by location.
Two things to note about this: 1) it works and 2) it could be improved. That's exactly how an HTML-only interface should be. People who aren't using JavaScript don't expect a JavaScript-like experience, but they do expect the site to work. So we're in good shape, but there's still a lot of room for improvement here. Both the region and the country lists are rather long, so it can be difficult to find the right place. Also, it's possible to enter impossible locations, e.g. a country of Cuba and a region of Western Asia. Finally, it would be nice if we could see the search results without a full page refresh.
We can improve all of this with JavaScript. For the first issue, we add a text field and set it to autocomplete with region and country names. Whenever a country or region name is typed, the corresponding
optionis automatically selected. Though they're still driving the search results, we hide both the region and countryselects, because we've replaced them with an easier method of accomplishing the same thing.Alas, we've actually removed a bit of functionality here; though auto-completes are better than
selects for searching, they're actually worse for browsing. So we improve the browse-ability with a link to a page with a browse-able list of regions and countries.For the second issue, we update the region
selectwhenever the countryselectchanges, selecting the associated region. So if you have Western Asia selected and you choose Cuba, it will change Western Asia to Caribbean. The last issue, speeding up the response time, is actually the easiest to address. We simply turn on AJAX in Views.
We've now enhanced the experience with JavaScript, but it could still be even better. People don't always think of locations as names; we often think of them as points on a map. Though Morocco is next to Egypt in an alphabetic list of countries in Northern Africa, the two are spatially very far apart. Flash would be useful here. After checking for Flash availability, we replace the link to browse locations with a Flash map for doing that. When areas of the map are selected, we select the same areas on the hidden
selects. And from there, it works just as if someone had used theselects directly.We also change the auto-complete field to update the Flash map in addition to the
selects, so the map always reflects the actual location selection.
That's all easier said than done, of course, but that's the basic approach.
- HTML-only: works
- HTML+JavaScript: works better
- HTML+JavaScript+Flash: works best
The key step here is incredibly simple: HTML-only works. That's not hard to do, but it requires a consistent focus. It's really easy to start thinking about the really cool Flash map and completely forget about the HTML-only alternative. So this is a friendly reminder: don't do that.
A Brief History of HTML
HTML5 is scheduled to enter Last Call Working Draft status in October 2009, next month. While that's not really the end of work, it's close enough that introductory articles on HTML5 are popping up all over the place. Beyond the technical details, these articles have a different tone than the previous round of introductory markup articles, when XHTML was the new hot. Where XHTML was presented as new with a reason (extensibility), HTML5 is offered as just new.
Part of that difference is just that the web has matured beyond the necessity to sell web standards; it's a given now. But another part of the difference in HTML5's pitch (or lack thereof) is the backstory, the process by which HTML5 came to be. It's an interesting story, and should be useful in understanding the why of HTML5.
Genesis
In the beginning (1993), there was Tim Berners-Lee, and Tim created HTML. He said "this is good" and proposed a draft to the Internet Engineering Task Force (IETF), a standards organization. IETF drafts require implementations, so the HTML draft referenced Mosaic, a web browser that later became Netscape, which later became Firefox. Mosaic, of course, would have been rather worthless without HTML. So a symbiosis between browsers and web standards drove the web from the beginning.
You may not have heard of XHTML 2, because browser vendors fell all over themselves in their rush to not implement it...When the original HTML draft expired in 1994, the IETF created the first HTML Working Group (HTMLWG), who created HTML 2. Also in 1994, Tim created the World Wide Web Consortium (W3C), with a mission
To lead the World Wide Web to its full potential by developing protocols and guidelines that ensure long-term growth for the Web.
If that sounds like what the HTMLWG was doing, that's because it was. The two standards bodies didn't run in parallel for long.In 1996, after a series of additions to HTML 2, the IETF HTMLWG was closed and further work on HTML moved to the W3C. The W3C published HTML 3.2 and HTML 4, both in 1997. In December of 1999, HTML 4.01 was published. In the seventh year (1998), the W3C rested. And rested. And rested. HTML hasn't changed since. In summary: one man created HTML, two standards organizations worked on it for 5 years, and then it wasn't changed for 10 years.
Exodus
And then there was XHTML. Backing up a bit, HTML was always based on SGML, the Standard Generalized Markup Language, an International Standards Organization (ISO) standard. Without getting into the details of SGML, many saw it as too ambiguous for a reliable web. Enter XML. The eXtensible Markup Language, is a stricter subset of SGML, with the goal of clearer communication. It's also, as the name implies, designed for extensibility.
The W3C didn't actually rest in 1998; it only rested on HTML, while publishing XHTML 1.0. XHTML 1.0 is essentially HTML 4 as XML instead of SGML. Many web publishers saw XHTML as the future and started publishing it instead of HTML 4. Some didn't see much value in XML syntax and decided not to switch, or to move back after trying it. One of the latter group was a guy named Ian Hickson, who in 2002 published a much-cited article on one key problem with XHTML: Microsoft's Internet Explorer browser didn't (still doesn't) handle XHTML well. Sometimes the symbiosis between browsers and standards doesn't work out so well. More on Ian later.
Judges
Like HTML, nothing has really changed with XHTML since 1999. But not for lack of trying. In 2002, the W3C first published XHTML 2, a complete rethinking of XHTML, this time less (not at all) focused on mirroring HTML 4 and more (completely) focused on extensibility. You may not have heard of XHTML 2, because browser vendors fell all over themselves in their rush to not implement it, so it never went anywhere. Symbiosis.
Many attribute XHTML 2's failure to the lack of compatibility with HTML 4 or XHTML 1. Of those who even noticed XHTML 2, one of the most scathing critiques of it came from Mark Pilgrim who summarized a 2003 article with
Standards are bullshit. XHTML is a crock. The W3C is irrelevant.
Ouch. More on Mark later.2 Kings
Amidst the lack of progress on XHTML 2, there was a workshop in 2004 in which some people decided the W3C was no longer managing the web very well. So they created their own organization, the Web Hypertext Application Technology Working Group (WHATWG). To quote the WHATWG's FAQ:
The WHATWG was founded by individuals of Apple, the Mozilla Foundation, and Opera Software in 2004, after a W3C workshop. Apple, Mozilla and Opera were becoming increasingly concerned about the W3C’s direction with XHTML, lack of interest in HTML and apparent disregard for the needs of real-world authors. So, in response, these organizations set out with a mission to address these concerns and the Web Hypertext Application Technology Working Group was born.
So now you can answer questions about HTML5 without even looking at the draft, which is handy, because the draft is 400+ pages long.Notably absent from that list of browser vendors is Microsoft, vendor of the most
popularwidely-used browser. The WHATWG nonetheless pressed forward working on what they called Web Apps 1.0, an incremental improvement to HTML, which people quickly started referring to as HTML5. "But wait," you say, "I thought the W3C was in charge of HTML." Well yeah, so did they. So did the IETF. Remember that?The initial WHATWG announcement was written by Ian Hickson, who careful readers will remember from his earlier criticism of XHTML, specifically Microsoft's handling of it. You can follow the WHATWG's own take on their work at their blog, which is updated by Mark Pilgrim, who is also writing a book about HTML5. You may remember him as the guy who said
XHTML is a crock. The W3C is irrelevant.
I did say this would be an interesting story.1 Kings
Perhaps hoping the kids would get bored and disperse on their own, the W3C didn't directly tell the WHATWG to get off their lawn. They didn't much react to HTML5 at all until 2006, when W3C director Tim Berners-Lee (remember him? guy who started all this?) wrote Reinventing HTML in which he said
Some things are clearer with hindsight of several years. It is necessary to evolve HTML incrementally.
That was a significant (complete) change from the W3C's previous position, essentially that HTML was dead, to be non-incrementally replaced by XHTML 2. If the new HTMLWG sounded a lot like the WHATWG, that's because it was.But unlike the previous transition from the IETF to the W3C, the WHATWG didn't simply hand control of HTML back to the W3C. Even after the WHATWG's HTML 5 draft was formally adopted by the HTMLWG in 2007, the WHATWG continued working on it. They did start working more closely with the W3C, but the relationship is ... what do the kids call it on Facebook now? ... it's complicated.
If you look at the HTML5 draft at the W3C, you'll find this explanation of who is creating the spec:
The W3C HTML Working Group is the W3C working group responsible for this specification's progress along the W3C Recommendation track... This specification is also being produced by the WHATWG.
The HTML Working Group is chaired by Sam Ruby of IBM and Chris Wilson of, wait for it... Microsoft. Ian is the editor of the HTML5 draft, and a member of both the HTMLWG and the WHATWG. Chris once said
I would hope in the eventuality of time the WHAT-WG would simply dissolve because it’s no longer necessary... in my opinion HTML is not in the hands of the WHAT-WG and never has been, despite calling a spec or set of specs 'HTML 5'; it belongs to the W3C.
Whereas Ian once said
The HTML5 work isn’t using the traditional W3C approach, and will never use a consensus approach so long as I am editor.
In summary: HTML5 has been developed within the WHATWG, intentionally avoiding W3C processes, but with plans to have it be adopted by the W3C, via a group chaired by someone who thinks the WHATWG never had any authority to work on it in the first place.Meanwhile, the W3C announced 2 months ago that XHTML 2 was officially dead. HTML5 does include XHTML5, an XML syntax of the language. So XHTML in general isn't dead, but it's not exactly healthy. Given the lack of any other path forward, it does seem very likely that somehow HTML5 will become the recommended web publishing format of both the WHATWG and the W3C's HTMLWG. Somehow.
Revelations
One thing that is entirely clear about the future of HTML5 is that browsers will support it. That's clear because they already support it. One aspect of it, canvas, has been supported, and heavily relied upon, for years. If nothing else, HTML5 has already reversed the relationship between browsers and standards bodies. Instead of the W3C all but begging browsers to implement standards, browsers are now impatiently waiting for the W3C to recommend the standards they've already implemented.
So now you can answer questions about HTML5 without even looking at the draft, which is handy, because the draft is 400+ pages long. Why is there a new <video> tag in HTML5? Because some browser vendor (maybe the one that also owns a large video site) wanted it. Why are there so many scriptable interface elements in HTML5? Because some browser vendor (maybe the one selling phones without Flash support) wants them. Why is there no support for RDFa in HTML5? Apparently no browser vendor wanted it.
Beyond understanding how HTML5 got here, this background should also give you a good idea of where it's headed. Until the next major shift, browser vendors will be driving standards. If you want a <pegacorn> tag in HTML 6, get it supported by a browser or two.
Nice like Elevators with Progressive Enhancement
Elevators are nice. They get you up and down tall buildings quickly with almost no effort. They're accessible and even somewhat social. Yet stairs remain the standard method for traveling from one floor to another. Even buildings with elevators have stairs. Being nice isn't the same as being standard. And being standard is very important.
Stairs are the standard because stairs are reliable. Stairs don't stop working when the building is on fire nor when a power line goes down. And stairs are completely intuitive; you simply walk up when you want to go up; there's no pressing the wrong button.
Starways are even better than stairways.
Photo by Carrington VanstonMany web technologies are nice like elevators. Most commonly, Flash and JavaScript are used to make a user experience quick and easy. But neither Flash nor JavaScript are universally supported on the web. The iPhone, for example, has a popular browser with no Flash support. And Google's search engine doesn't handle JavaScript. While these are exceptions to the norm, it would be a big mistake to ignore them.
Fortunately, HTML is reliable like stairs. And just like buildings with both stairs and elevators, we can use HTML everywhere on the web, and apply conveniences like Flash and JavaScript in addition to HTML. This strategy is known as "progressive enhancement." A term first coined 6 years ago (approximately 1800 years ago in web time), progressive enhancement is by now unremarkable for many web designers and developers.
The idea is that you put everything in HTML first (stairs), make sure that works, and then go on to make it work better with tools like JavaScript and Flash (elevators). This seems pretty obvious, and in many places it is. Drupal's JavaScript principles, for example, start with "All pages should be perfectly functional without scripts."
Yet progressive enhancement isn't yet ubiquitous. Plenty of sites are still missing stairs. Google finds millions of sites telling their visitors "flash required" or "javascript required". And these are the sites that have at least considered the possibility that Flash or JavaScript may be missing. Many more just silently fail.
Beyond practical concerns, a lack of progressive enhancement should raise a huge red flag. While the phrase was coined in 2003, "progressive enhancement" describes how the web has always worked. HTML embodies progressive enhancement; even if you don't have a browser, you can open HTML documents in any text editor and get the content out. At a lower level, the network protocols are built around the assumption that any given packet of data may not reach its destination.
A site built without progressive enhancement should raise questions about all other aspects of the site. Do security precautions on this site similarly ignore all but the most obvious scenarios? Will the site be as incompatible with future technology (e.g. HTML 5) as it is with older technology?
Elevators are nice. But if an architect designed a multi-story building with no stairs, he'd be immediately identified as unprofessional and probably fired. Even if it's an otherwise great building, you just don't leave out the stairs; that would be ridiculous. Some day the web will similarly dismiss websites without progressive enhancement. Even if it's an otherwise great site, you just don't leave basic content or navigation out of the HTML; that would be ridiculous.
Source Control and Frisbee
I’ve been working at Aten for a couple weeks now, long enough to reflect a bit. Since I started here, I’ve had a chance to work on a few interesting sites, attended DrupalCamp Colorado, and discovered some good restaurants near the office.
A large part of my initial interest in working at Aten was reading about their (now our) process. In theory, the process sounded great, but it’s easy to talk about a good process; more difficult to actually carry it out. I’ve been pleased to discover we really do take process seriously.
When friends and family have asked me about the new job, I’ve talked about playing frisbee over lunch, to give them a sense of why I like my job. But when colleagues ask me about it, I tell them about a different kind of lunch, the one we spent using salt and pepper shakers to represent working copies and repositories, to work out better processes around source control.
Few people know what source control is. Fewer still care enough about it to spend an entire lunch on the subject. And most people have no reason to know or care. Done well, source control speeds up development and safeguards against mistakes, but it’s one of those behind-the-scenes details that no one ever sees.
It’s easy to overlook minor process issues no one ever sees. But people who really care about what they’re doing will sweat the details. It’s really nice to see my new coworkers at Aten care about what they’re doing. And frisbee is nice too.
