Embedding Interface Sketches in Code

James Simpson and Michael Terry

User interface design is often practiced through the use of hand-drawn sketches, whiteboard diagrams, and mock ups using computer graphics/illustration packages. However, the implementation of user interfaces is commonly embodied in source code, an inherently textual design medium.

The Problem

Since the design and implementation of user interfaces are not embodied in a common medium, numerous challenges face developers who implement and maintain applications containing user interfaces. The graphical nature of their implemented user interface is not directly documented in the source code, meaning they may have to run their application and navigate to the particular state of the application to see what the interface looks like.. Developers often have to use specialized tools (i.e. separate applications) to see the design they want to implement. Some design mediums (i.e. hand drawn sketches), do not lend well to the online world and require additional steps (i.e. scanning the document in) in order for the document to become a digital embodiment. This may limit the ability for others to view and comment on user interface design.

The Textual Medium

To address these issues, we introduce a new type of medium for user interface design which bridges the gap between the design and implementation of user interfaces. Our design medium uses a combination of characters to express components of a UI design (i.e. ASCII art). This allows designs expressed in this medium to be embedded in source code as comments. In addition, since interface design is represented in a strictly textual medium it can be easily embedded inline into a variety of communication methods such as e-mail, newsgroups, and chat.

*       Login to System       *
* Username : [______________] *
*                             *
* Password : [______________] *
*                             *
*                             *
*   ----------    ----------  *
*   |Cancel  |    |Okay    |  *
*   ----------    ----------  *
*                             *

Login System Created Using Textual Medium

The Medium's Challenge

However, it can be tedious and time consuming to create and edit user interface designs using conventional tools (i.e. text editors), as they treat the designs as simply a string of characters rather than a combination of user interface objects. Making the smallest change in one design often requires careful realignment and adjust of rows of text that were affected by the change. In addition, it can be difficult for the designer to find a recognizable string of characters to represent the user interface components they want to express.

The GUIIO Toolset

We have attempted to tackle these challenges by creating two user interface builders which render the interface in ASCII art. These tools automatically handle the adjustment and manipulation of UI components without the need for user intervention. Since these applications are implemented as user interface builders, they automatically have preset renderings for each of the common components a designer may use.

The Stand-alone GUIIO Tool

The first tool, called GUIIO is implemented as a standalone application. Designers simply select the UI component they want rendered from a toolbox, and the component is rendered on an ASCII backed form. Designers can manipulate individual UI components by selecting the component from the form. Components are resized by dragging on a corner of the ASCII rendered component. GUIIO automatically re-renders the UI component's ASCII art representation as the designer is resizing. Components can also be moved around by dragging the component around the form. Finally, individual properties of a component such as caption are adjusted by double clicking on the component. Once the designer is satisfied with the design they created they can save the design to a file or copy the design to a clipboard. Since the design is always rendered in text, the designer has the freedom to use the design in a variety of applications (i.e. e-mail, or IM client).

The GUIIO Plugin for Visual Studio

Another tool, called The GUIIO plugin for Visual Studio, is implemented as a plugin for Visual Studio 2010, a popular developer tool. This plugin extends the Visual Studio source code editor by allowing interfaces to be easily designed and manipulated inside the source code editor. The resulting design is expressed as a comment embedded inside the source code. The plugin is invoked by pressing a shortcut key (Ctrl+Alt+G) inside the source code editor. Once invoked, the designer is presented with a list of supported UI components the tool supports. When the designer has selected the component they want, the component is inserted as a comment located at the position where their caret is located in the source code editor. The designer can then manipulate the position of the component on the form by positioning the caret of the source code editor to the immediate left of the component they want to position and pressing the key corresponding to the movement they want (i.e. space shifts the component to the right). Similarly, properties of each component are adjusted by positioning the caret to the aspect of the component commonly associated with the property and pressing keys which closely map to the property the designer wants adjusted. As before, the plugin automatically re-renders the affected component(s) as the designer is manipulating the UI component, freeing the designer from manual intervention.

UIST 2011 Video of GUIIO Plug-In in Action


Paper on GUIIO Stand-alone Application
Extended Abstract of GUIIO Plug-in for Visual Studio (UIST 2011)
GUIIO Stand-alone Application Project Page
GUIIO Project Page on Sourceforge