The Real-Time Web

By Brian Lesser
November 8, 2008 | Comments: 10

During the last year-and-a-half my days have been dominated by two large projects: the implementation of Ryerson University's new HR system and an upgrade of our Student Administration System (SAS). Now that the HR project has gone live and the SAS upgrade is in progress, I've started to reflect on how we managed both projects and how my time was "spent" in them. A lot of it went into meetings.

Throughout both projects I became fascinated with the importance of those meetings and I started to wonder why, with all the software tools available for collaboration and project management, we needed to schedule so many people into the same room so often.

Some of the most interesting meetings I attended were the early HR fit-gap sessions where we looked at what the software we purchased does out-of-the-box and compared that to our current business practices. In those meetings the key software tool was Visio. We sat around the table talking about workflows while one person, who was assigned the task of updating each Visio flow chart, would listen to the conversation and update the screen everyone was looking at. One of the consultants on the project was especially good at it. If one person suggested a change she would make it with a minimum of fuss. If the change had to be undone in favor of a different change she did it quickly. Despite her talents, and admirable patience with us, some things felt strange about the process. It wasn't real time and it wasn't on the Web.

Meetings are supposed to happen in real time. You can see the expression on anyone's face and hear what they say immediately. If you brought paper and pen to the room you can work out ideas and jot down notes as things occur and refer back to them as you go. If there are big enough white boards in the room more than one person can annotate a drawing or sketch competing ideas at the same time.

The way we used Visio changed that. Our focus was mostly on the big screen and the often complex flow chart we were looking at. None of us could change the flowchart ourselves. We all had to wait for one person to update it. It was easy to see frustration with that as anyone who wanted to make a significant change would have to wait while the drawing evolved, often in what they considered to be the wrong direction, before they could ask that it be changed. By the time they got to say something the drawing might have little resemblance to where they wanted to go. So, despite the talents of our consultant, it took time to deconstruct and then reconstruct each flowchart. Visio is really a desktop drawing tool, and I assume, wasn't initially designed to help people collaborate in meetings. I couldn't help wondering what our meetings would have been like if everyone could have easily manipulated those Visio drawings from their laptops and saved different versions of the drawing for reference instead of waiting. I think it would have made our meetings more "real time" and so helped make all of us more active and productive participants.

When we finished each meeting the drawings were never quite complete. It took some time to clean them up and post the final version of each of them to our document repository. By the end of the project our repository contained a large collection of documents in a hierarchy of folders. Finding the document I needed in the hierarchy still reminds me of how hard it used to be to find something using gopher instead of the Web. Without seeing pages as you go, and being able to take short cuts via links, it is quite painful to hunt for things in folders, download big documents before being able to see if it's the document you want, and then move on until you finally find what you are looking for. I've heard many people blame the document repository itself for how difficult it is to find anything in it. We've been spoiled by the Web.

Since the early 1990s, the Web has seen an amazing evolution from a simple hypertext document presentation system based on three simple standards (HTTP, HTML, and URL) to a richer and more participatory and collaborative system. To get where we are today we needed more than HTTP, HTML, and URL. We needed other technologies like JavaScript, Cascading Style Sheets (CSS), XMLHttpRequest (XHR), and Flash, at a minimum, for audio, video, and vector graphics. Of course having the technology here or there wasn't enough. The technologies had to mature, browser standards had to develop, and near-standard implementations had to become nearly ubiquitous. It has taken years and a lot of false starts. If you remember the excitement about Java Applets in 1995/96 or Netscape's introduction of the Layer tag in 1998 you'll remember there have been many promising ideas that never achieved ubiquity.

With the exception of the two large projects I mentioned earlier, all the other projects I work on at Ryerson are documented in Confluence, Atlassian's Enterprise Wiki. If we change a document during a meeting there is no delay while it is polished up, passed from person to person for review, and then finally posted somewhere in a repository. We can contribute to it before, during, and immediately after a meeting. And, we can track how it has evolved over time. As Peldi notes, it is a superior way to work. For all that, it isn't real time: more than one person can edit the same page but you don't see their work in real-time as they are editing. When different people who have been working on the same document at the same time save their work, you have to look through the document's history to see everything.

Google Docs is interesting in that Google has built in a more advanced simultaneous editing feature into their system. They merge changes into a single document when multiple users, who are editing the document at the same time, save their edits. But it still isn't real time. There is no real-time indication what other people are doing/editing, and you don't see changes until they are saved.

Of course there are real-time collaboration tools available. Some, like Adobe's Acrobat Connect, Meebo, and Octopz work within a Web browser while many others require installing standalone applications.

But none of the in-browser solutions I've seen fully solve the real-time collaboration problem of being able to simultaneously work on things like a work flow drawing with the same level of awareness as you get when a few people sketch something out on a white board. Not even application sharing, where one person takes over another's computer really accomplishes this. What we need is something new. We need to be able to simultaneously contribute to and see the evolution of documents, graphics, and other media in real-time coupled with a recorded history of their evolution, including false starts, abandoned ideas and so forth.

To accomplish this I think we need to get away from the idea that we should share and synchronize files or application windows and look at real-time sharing of data models within the browser.

I mentioned earlier that I am fascinated with how important meetings are - especially when trying to keep a large complex project on the rails. There are good reasons to get team leads into a room so everyone can synch up, renew or redevelop a common vision, and leave with a coordinated and logical set of tasks that move the project forward. But it is also expensive to regularly get people into a room and to keep them waiting while they are there. I think taking the Web real time has the potential to change how we work for the better. If the Real-Time Web means groups can evolve complex data models while they all work on them, see those changes in real time, and track how those models evolved, then I think the way we work will change for the better. The Real-Time Web also is about providing some of the other things you get in meetings or hallway conversations. You can see the expression on everyone's face and hear what they are saying and how they say it.

If I can find the time, I'd like to make the Real-Time Web and shared models a theme of this blog. There are a lot of things to write about. There are competing and complimentary protocols like XMPP, SIP, HTTP extensions (Comet), and RTMP. There are different technologies for screen sharing and video conferencing. There are different ways to update and share models in real time and lots of ways to present those models to users. And there are no guarantees that the choices we make today will look like the solutions that are widely available in the next decade.

Some References

Real-Time group editing of complex data models/documents is a non-trivial undertaking and the work to date makes for interesting reading.

Operational Transformation
http://en.wikipedia.org/wiki/Operational_transformation

Operational Transformation in Real-Time Group Editors: Issues, Algorithms, and Achievements
http://citeseer.ist.psu.edu/cache/papers/cs/3938/http:zSzzSzwww.cit.gu.edu.auzSz~sczzSzpaperszSzcscw98.pdf/sun98operational.pdf/

Achieving Convergence, Causality-preservation, and Intention-preservation in Real-time Cooperative Editing Systems
http://citeseer.ist.psu.edu/cache/papers/cs/3938/http:zSzzSzwww.cit.gu.edu.auzSz~sczzSzpaperszSztochi98.pdf/sun98achieving.pdf/

Achieving Convergence with Operational Transformation in Distributed Groupware Systems
ftp://ftp.inria.fr/INRIA/publication/publi-pdf/RR/RR-5188.pdf


Tree-based model algorithm for maintaining consistency in real-time collaborative editing systems
http://www.globis.ethz.ch/script/publication/download?docid=279


Xerox Parc had an interesting project named Jupiter:
http://www.fastcompany.com/magazine/01/soccomp.html
http://mirrors.ccs.neu.edu/MOO/papers/JupiterWin.ps

Freetext: A Consistency Algorithm For Group Text Editors
http://www.computer.org/portal/cms_docs_dsonline/dsonline/topics/collaborative/events/iwces-6/Humbad.pdf

Products and Services?

If you know of browser-based products that provide multi-user real-time (a.k.a. visibly simultaneous) editing of documents or media please let me know by commenting below with a link to the service.


You might also be interested in:

10 Comments

Cobalt Project Page
http://www.duke.edu/~julian/Cobalt/Home.html

See the Video of the Virtual 3d Whiteboard
http://www.youtube.com/watch?v=uVFsxev-2sk

We've recently launched scribblar (http://www.scribblar.com), a free online whiteboard and collaboration tool. Please check it out.

Have a look at Digital Samba OnSync (http://www.digitalsamba.com), a flash based web conferencing platform.

I try to define Objects with a way to 'own' them. I have used flash remoting to build collaboration tools, such as real-time editing of KML data loaded in 'google earth'. Data sharing is how a conglomerate of input sources can generate a single output. In my case, sharing data between land developers. To me its simply deciding what and how users can or cannot change shared data values and viewing them as soon as possible.

spreed (http://spreed.com) supports collaborative whiteboards and (soon) mindmaps in their flash based conferencing system. You can give it a try for free.

Etherpad looks interesting, It is a real-time text editor that works inside browsers:

http://etherpad.com/

The beta is private so I couldn't try it.

Those are very good tools, however, in many cases those platforms are used just to allow the team to review the same document together in real-time and "be on the same page".
The recently launched free site http://www.showdocument.com does exactly that, quickly show documents to friends and colleagues.
It allows web meeting and co-browsing on any document, user uploads a document and invites friends to view it with him live
All the participants in the session see each others' drawing, highlights, etc.

Josh

I finally had a look at Show Document. It reminded me of Octopz where editing is restricted to turn taking.

Also I read today that AppJet, the developers of EtherPad, have been acquired by Google and will work on Wave:

http://etherpad.com/ep/blog/posts/google-acquires-appjet

several real time web companies were covered today at business insider.

http://www.businessinsider.com/the-five-real-time-startups-you-need-to-know-2009-12

http://sency.com was one company covered

Team apart's collaboration service includes a simple real-time notepad:

http://teamapart.com/

News Topics

Recommended for You

Got a Question?