XML User Interfaces provide an effective and in= novative means to separate presentation from logic and data. XML User Interfaces provide faster= and more efficient methods to change and maintain code. Traditionally, a Graphical User Interface (GUI) modification involves editing the source code, compiling, a= nd executing the application. Yet with an XML User Interface Language a develo= per simply edits the XML file and restarts the GUI application. Examples of this can be seen in applications that produce interchangeable GUIs or skins.

The Multiple Device User Interface project is b= ased on XML User Interface Language (XUL). One goal is to take a XML description and with that create a framewo= rk based on rules that specify navigation, layout, and components. Yet another goal of this project is create a generalized framework for Graphical User Interface development and to define the rule specification. There are several devices and each= is different from the other with regard to its memory, display capabilities and internal representation of the data. Although a general representation can be used for all devices, this is sometimes thought to be too simplistic, thus the direction the User Interface framework will follow will produce a different rule set for different devices or perhaps produce different rule = sets for the same device.

Introduction=

Traditional methods for Graphical User Interface (GUI) programming can be tedious, cumbersome, and error prone. Newer methods use Graphical = User Interface Markup Languages that spur new ideas and challenges for software engineers and User Interface designers.&nb= sp; Both User Interface development and XML have become interesting rese= arch topics. Examples of the ongoi= ng projects include: LUXOR, UIML, VOICEXML, and THINLETS. Some interfaces are extremely complicated with varying widgets and functionality (i.e. SWING, SWT, C#), while others are simple and straightforward (J2ME).<= span style=3D'mso-spacerun:yes'> This project will develop a = new approach to User Interface design by developing a general scheme to compose Graphical User Interfaces, Multiple Device Markup Language (MDML), and also produce a generalized framework to generate code based on a User Interface toolkit.

One goal of this project is to provide a method= to separate the GUI from structure and behavior. In order to make this separation a framework must be built to close the gap between sophisticated User Interfa= ces and simple User Interfaces by generating a Graphical User Interface based u= pon a rule specification. The rule specification provides specific pieces of information like widget and attri= bute mappings, composition mapping, imports, accessibility functions, and functionality. MDML is a gene= ral markup language designed for generalizing both sophisticated graphical User Interface toolkits and simple graphical User Interface toolkits. The results of this research are important for other frameworks and projects that pursue a clear distinction between presentation, structure and behavior (i.e. ZOOM Project). The design of one interface for mu= ltiple platforms could revolutionize the way User Interfaces are built in future applications.

This framework provides a mechanism for develop= ers to develop GUI applications in a rapid and fast paced environment. Since defining a process and build= ing objects from scratch in every instance requires a large amount of time and resources, not to mention the issue of cost, this framework will provide a simple way for developers to change, maintain and update their Graphical Us= er Interfaces. The framework is developed using Java technologies and uses the ZOOM Project’s Code Generator.

Problem

There is a problem however. Can a User Interfac= e be designed for multiple platforms independent of any structure or behavior? As technology improves so do end-user demands. The Industry requires fast deploym= ent to multiple target platforms without the maintenance and development overhead = of storing and developing every possible implementation (4). There is a growing and urgent need= for systems that can accomplish this task.&nbs= p; Some projects that are related to this problem are the Alternate Abstract Interface Markup Language (AAIML), Abstract User Interface markup Language (AUIML), Extensible Interface Markup Language (XIML), Extensible U= ser Interface Language (XUL), Microsoft Extensible Application Markup Language (XAML), User Interface Markup Language (UIML), and W3C XForms (4). All of these projects are based up= on XML.

T= he Multiple Device Markup Language (MDML), the language introduced by this project, is based on XUL. MDML along with the framework does not re-implement XUL nor does it provide a on= e to one mapping for any toolkit similar to the LUXOR project. The parser is a general parser unl= ike the LUXOR parsers, which are tightly coupled to a toolkit. This project requires the user to = define mappings between MDML tags and GUI toolkit components. This mapping is called the rule specification.

Architecture

The initial architecture was based on a series = of XML transformations to produce the final language file for input to the code generator. The architecture h= ad two major problems. Firstly, the XSL transformations became complicated requiri= ng a lot of templates, and second, composition of advanced widgets became increasingly difficult to generalize. The later is the reason the architect= ure changed since it often required special case scenarios in the rule engine.<= span style=3D'mso-spacerun:yes'> In general if special case scenari= os are coded then coupling becomes tighter between a MDML tag and a GUI toolkit widget, which leads to difficulty for other devices. So, the decision was made to alter= the initial architecture thereby using java instead of XSL transformations.

There are four parts to the final UI architectu= re rule engine, display engine, handler, and code generation. The rule engine will parse the rule file. The display engine will= parse the MDML file and handle the layout. The handler will generate a XML file based on the generalized class – class.dtd. The code generation will generate an executable language file. All three parts are Java based applications. The properties = file contains specific pieces of information needed by the rule and display engi= ne see Appendix C for details.

Initial Architecture Diagram

Final Architecture Diagram

Rule Engin= e

Rules provide a mechanism to prevent errors and enhance User Interface design. The rule engine is responsible for reading the rule file -- i.e. rule.XML. The rule file has three sections <profile>, <tag>, and <event>. The <profile> tag may contain these tags <import>, <toolkit>, <accessor> and <topfunction>. The <import> tag is used for any imports or includes that must be specifi= ed at the beginning of any language file.&nbs= p; The <toolkit> tag will specify a GUI toolkit which is required= by the framework to be based on a OO language i.e. Swing, SWT, or C#. The <accessor> tag will prod= uce a method that will allow a back end source to access any specific component.<= span style=3D'mso-spacerun:yes'> The <topfunction> is the fun= ction used display the initial window or frame.&= nbsp; See Appendix A.

&= nbsp; The <event> section has six tags <action>, <focus>, <window>, <mouse>, <mousemotion>, and <selection>.<= span style=3D'mso-spacerun:yes'> The <action> tag is for any = mouse click, keyboard pressed or released, or menu selection. The <focus> tag is for a com= ponent when it gains or loses the focus. The <window> tag is used when a window event occurs, i.e. wind= ow closes. The <mouse> tag= is used when a user presses the mouse button over a component. The <mousemotion> tag is use= d when a user moves the mouse over a component.&n= bsp; Last the <selection> tag is used when an element is selected inside a component. This is a= ll that is required for any desktop profile.&= nbsp; Other profiles can build from these mapping i.e. PDA can substitute = can use a pen event for the <mouse> tag.

The <tag> section is compromised of all t= he MDML tags that need to mapped or ignored in some cases. Each <tag> section will cont= ain multiple <rule> tags. E= ach <rule> tag will contain a MDML tag i.e. <TextBox>. The MDML tag section will contain = four tags <widget>, <attribute>, <define>, and <composition>. The <widget> tag must specify the GUI toolkit mapping and optionally whet= her or not if a constructor is required. The <attribute> tag will specify how the MDML attribute is map= ped to the GUI toolkit attribute. The <define> tag is used to specify any required definitions before or af= ter the initial widget is defined. All methods under the <define> tag have generated names. The <composition> tag is use= d to specify how a particular component or element is added to its <widget> tag. See Appendix A for furth= er details.

In general to specify a method or constructor u= nder the <widget>, <attribute> and <composition> tags use the <method> and <constructor> tag. A <method> is used in the &l= t;attribute>, <define>, and <composition> tags to specify the method used in = each case. The <constructor>= tag is only used in the <widget> and <composition> case to specify = that normal definition is not appropriate but use constructor definition instead. Each <method> = and <constructor> can have any number of <Parameter> tags. Common variables used with <parameter> tags are $Parent, $TopFunction, $Default, $(MDML Tag),&n= bsp; and $Counter. See Appendix A for further details= .

Rule Examples

Example 1

The Swing toolkit provides a JTREE component for displaying hierarchical components; provide a rule for this component.

Step 1

Find existing rule file (i.e. rule.XML) or create ne= w rule file. If this is a new rule f= ile remember to update ui.properties file with the rule filename. Edit rule file with a text editor = or XML specific editor.

Step 2<= /o:p>

Add the new rule wit= h the <rule> tag and specify an MDML tag association.

&= nbsp; &nbs= p; <T= ree>

&= nbsp; &nbs= p; </= Tree>

&= nbsp; </rule>

Step 3<= /o:p>

Add the widget mappi= ng using the <widget> tag. In Sw= ing it is tempting to write this…

<rule>

&= nbsp; &nbs= p; <T= ree>

&= nbsp; &= nbsp; &nbs= p; <w= idget> JTree </widget>

&= nbsp; &nbs= p; </= Tree>

&= nbsp; </rule>

but in Swing one can not add elements to a JTREE. Elements are not added directly to= a JTREE they are added to a DefaultMutableTreeNode. So in this case the correct rule s= hould look like

<rule>

&= nbsp; &nbs= p; <T= ree>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; <widget> DefualtMutableTreeNode </widget>

&= nbsp; &nbs= p; </= Tree>

&= nbsp; </rule>

Step 4<= /o:p>

Add the attribute information using the <attribut= e> tag. Each attribute section contains 1 or more <Method> tags.&nb= sp; The Method in this case needs two attributes name and type. Name is the name that the rule file can use to associate it with the MDML attribute (attribute mapping). Type is required since the type will determine how the attribute from the MDML tag is formed. I.e. = String will contain quotes “”, text and numbers will contain no quotes= .

<rule>

&= nbsp; &nbs= p; <T= ree>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; <widget> JTree </widget>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; <attribute>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; <Method name=3D"text" type=3D"String">setText</Method>=

&= nbsp; &nbs= p; &= nbsp; &nbs= p; </attribute>

&= nbsp; &nbs= p; </= Tree>

&= nbsp; </rule>

Step 5

Define the JTree Component. Remember we have just defined the DefaultMutableTreeNode to so elements can be added but we still need to def= ine the JTree component because only the JTree can be added to a container in Swing. Use the <define>= tag to define other functionality that needs to be associated with a MDML tag TREE.

<rule>

&= nbsp; &nbs= p; <T= ree>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; <widget> JTree </widget>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; <attribute>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; <Method name=3D"text" type=3D"String">setText</Method>=

&= nbsp; &nbs= p; &= nbsp; &nbs= p; </attribute>

= ; &n= bsp; = ; &n= bsp; <define>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; <Method name=3D"myTree" order=3D"post" type=3D"JTree"= >

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; &nbs= p; <Parameter>= $Tree</Parameter>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; </Method>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; </define>

&= nbsp; &nbs= p; </= Tree>

&= nbsp; </rule>

Step 6<= /o:p>

Define the composition mapping. How do we add anything to our TREE= component? In MDML a tree is composed of NODE= (s). So we must tell the framework how = to attach these NODE(s). In our = case use the <METHOD> tag again and specify a method by using the name attribute.

<rule>

&= nbsp; &nbs= p; <T= ree>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; <widget> JTree </widget>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; <attribute>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; <Method name=3D"text" type=3D"String">setText</Method>=

&= nbsp; &nbs= p; &= nbsp; &nbs= p; </attribute>

= ; &n= bsp; = ; &n= bsp; <define>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; <Method name=3D"myTree" order=3D"post" type=3D"JTree"= >

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; &nbs= p; <Parameter>= $Tree</Parameter>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; </Method>

= &nb= sp; = &nb= sp; </define><= /p>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; <Node>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; &nbs= p; <Method name=3D"add"/>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; &= nbsp; </Node>

&= nbsp; &nbs= p; &= nbsp; &nbs= p; </composition>

&= nbsp; &nbs= p; </= Tree>

&= nbsp; </rule>

&n= bsp;

Example 2

Provide the rule for a MDML Node using Swing.

&nb= sp;

Step 1

Create a rule with a MDML tag.

<rule>

<Node>

</Node>

</rule><= /o:p>

Step2

Provide the MDML tag association with a swing component. In Swing A DefaultMutableTreeNode can be created using a constructor – DefaultMutableTreeNode node =3D new DefaultMutalbeTreeNode(“String= 221;). The rules provide a convenient way= to do this using a <Constructor> tag similar to the <Method> tag. Here the $text is a lets the frame= work know, a variable must exist named text.&nb= sp; In this case the MDML file contains a text attribute so it will map = to whatever the user sets the text to.

<rule>

<Node>

= <widget>DefaultMutableTreeNode

= <Constructor><= /o:p>

= &nb= sp; <Parameter type=3D"String">$text</Parameter>

= </Constructor>=

= </widget>

</Node>

</rule><= /o:p>

Step3

Provide the composition. Composition = exists because a node can contain other nodes.<= /p>

<rule>

<Node>

<widget>DefaultMutabl= eTreeNode

= <Constructor><= /o:p>

= &nb= sp; <Parameter type=3D"String">$text</Parameter>

= </Constructor>=

= </widget>

= <composition>

= <Node>

= &nb= sp; <Method name=3D"add"/>

= </Node><= /span>

= </composition>

</Node>

</rule><= /o:p>

Display Engine

The display engine is responsible for reading t= he initial MDML file into memory and changing layout as desired by the rules.<= span style=3D'mso-spacerun:yes'> The display engine uses Document O= bject Model (DOM) to initially capture the MDML file into memory. Every tag is translated into a User Interface Node (UINode). All display engine objects, component(s), bargroup(s), container(s), and layout= (s), will inherit from the UINode creating a UINode Tree.

The display engine contains two data structures= . One is a tree that mimics the DOM = with extended functionality, and the other is a HashTable that provides fast loo= kup by id for any UINode object. The DOM is not sufficient to use in producing the code since there a= re a number of things that are needed to be automatically generated and added to= the tree i.e. id number. The Hash= Table is useful when composing a component on the container since reading the who= le tree again is redundant, use a hashing mechanism to perform fast lookups for any UINode.

The input is a MDML file that is a valid XML fi= le and is based on the UISchema.xsd. The objective of MDML is to define a complete general set of tags for different= and varying platforms and provide simple and convenient documentation and usage examples. See Appendix D for explanation and details of each MDML component. There are two approaches for defin= ing a complete set of tags for any GUI toolkit.&= nbsp; One approach includes the minimum set of functionality provided by t= he toolkit and the other includes the maximum set of functionality. The minimum set is a subset of the maximum set.

One advantage in using the minimum set of functionality is the mapping of tags to the larger set is trivial since the minimum set is a subset of the larger set.= A disadvantage to this approach is the majority of maximum set of fu= nctionality remains useless; rendering non-fancy GUIs on a desktop. A disadvantage to using the maximu= m set of functionality is that the minimum set may not map directly to the maximum set, thus some work is required to enhance the mapping. But, this is justified, as the com= plete set gains the advantages of the maximum set of functionality while still be= ing able to transform to the minimum set. So, if a tag does not exist in the minimum set a useable tag(s) must= be mapped or omitted to that individual tag i.e. <toolbar> is omitted. Thus, this project will use the ma= ximum set of functionality to describe a complete set of tags for the MDML description. Below is a sample calculator written in with MDML.

Calculator Application Example MDML

<?XML version=3D"1.0" encoding=3D"UTF-8"?>

= &= lt;Window id=3D"String" name=3D"Calculator" show=3D"true&quo= t;>

= &nb= sp; = <MenuBar name=3D"menuBar1">

= &nb= sp; = &nb= sp; <Menu text=3D"Edit">

= &nb= sp; = &nb= sp; = <MenuItem mnenomic=3D"c" text=3D"Copy"/>

= &nb= sp; = &nb= sp; = <MenuItem mnenomic=3D"p" text=3D"Paste"/>

= &nb= sp; = &nb= sp; </Menu>

= &nb= sp; = &nb= sp; <Menu text=3D"View">

= &nb= sp; = &nb= sp; = <MenuItem mnenomic=3D"n" text=3D"Standard"/>=

= &nb= sp; = &nb= sp; = <MenuItem mnenomic=3D"u" text=3D"Scientific"/>

= &nb= sp; = &nb= sp; = <MenuItem mnenomic=3D"c" text=3D"Digital Grouping"/>

= &nb= sp; = &nb= sp; </Menu>

= &nb= sp; = &nb= sp; <Menu text=3D"Help">

= &nb= sp; = &nb= sp; = <MenuItem mnenomic=3D"h" text=3D"Help Topics"/>

= &nb= sp; = &nb= sp; = <MenuItem mnenomic=3D"a" text=3D"About Calculator"/>

= &nb= sp; = &nb= sp; </Menu>

= &nb= sp; = </MenuBar>

= &nb= sp; = <Panel>

= &nb= sp; = &nb= sp; <BoxLayout axis=3D"1"/>

= &nb= sp; = &nb= sp; <Panel>

= &nb= sp; = &nb= sp; = <FlowLayou= t/>

= &nb= sp; = &nb= sp; = <TextBox type=3D"text" editable=3D"false" columns=3D"40&quo= t;/>

= &nb= sp; = &nb= sp; </Panel>

= &nb= sp; = &nb= sp; <Panel>

= &nb= sp; = &nb= sp; = <GridLayout rows=3D"1" columns=3D"4" icolumns=3D"4"/><= o:p>

= &nb= sp; = &nb= sp; = <Button text=3D""/>

= &nb= sp; = &nb= sp; = <Button text=3D"Backspace"/>

= &nb= sp; = &nb= sp; = <Button text=3D"CE"/>

= &nb= sp; = &nb= sp; = <Button text=3D"C"/>

= &nb= sp; = &nb= sp; </Panel>

= &nb= sp; = &nb= sp; <Panel id=3D"Calculator Panel" name=3D"JCalcPanel">

= &nb= sp; = &nb= sp; = <GridLayout rows=3D"4" columns=3D"6" icolumns=3D"6"/><= o:p>

= &nb= sp; = &nb= sp; = <Button text=3D"MC"/>

= &nb= sp; = &nb= sp; = <Button text=3D"7"/>

= &nb= sp; = &nb= sp; = <Button text=3D"8"/>

= &nb= sp; = &nb= sp; = <Button text=3D"9"/>

= &nb= sp; = &nb= sp; = <Button text=3D"/"/>

= &nb= sp; = &nb= sp; = <Button text=3D"sqrt"/>

= &nb= sp; = &nb= sp; = <Button text=3D"MR"/>

= &nb= sp; = &nb= sp; = <Button text=3D"4"/>

= &nb= sp; = &nb= sp; = <Button text=3D"5"/>

= &nb= sp; = &nb= sp; = <Button text=3D"6"/>

= &nb= sp; = &nb= sp; = <Button text=3D"*"/>

= &nb= sp; = &nb= sp; = <Button text=3D"%"/>

= &nb= sp; = &nb= sp; = <Button text=3D"MS"/>

= &nb= sp; = &nb= sp; = <Button text=3D"1"/>

= &nb= sp; = &nb= sp; = <Button text=3D"2"/>

= &nb= sp; = &nb= sp; = <Button text=3D"3"/>

= &nb= sp; = &nb= sp; = <Button text=3D"-"/>

= &nb= sp; = &nb= sp; = <Button text=3D"1/x"/>

= &nb= sp; = &nb= sp; = <Button text=3D"M+"/>

= &nb= sp; = &nb= sp; = <Button text=3D"0"/>

= &nb= sp; = &nb= sp; = <Button text=3D"+/-"/>

= &nb= sp; = &nb= sp; = <Button text=3D"."/>

= &nb= sp; = &nb= sp; = <Button text=3D"+"/>

= &nb= sp; = &nb= sp; = <Button text=3D"=3D"/>

= &nb= sp; = &nb= sp; </Panel>

= &nb= sp; = </Panel>

= &= lt;/Window>

</Start>

Handler/Code Generation

The handler is general interface that contains = access to the rule set and the MDML display object. The classHandler is a handler that inherits from the handler class. Its responsibility is to generate a language specific XML file based= on the class.dtd. This file prod= uced is called Langauge.XML. The Language.XML file is passed to the code generator. To produce executable code this pr= oject uses the ZOOM Code Generator. The ZOOM Code Generator can generate executab= le code from a valid class.dtd or class-swing.dtd. This project uses the class.dtd si= nce the toolkit is not known before the profile is read. The class.dtd is general schema fo= r any Java class.

Language.XML/Swing GUI Code

Profile

The framework will establish a profile for each target platform. The profiles= are desktop, web, mobile, and voice. The desktop profile contains the maximum set of functionality with powerful, and rich toolkits Java Swing, Java SWT and .NET (C#, C++, VB). These APIs may be used to develop = very sophisticated graphical User Interfaces i.e. Microsoft Outlook and Eclipse. They have access to the complete s= et of widgets. Space and memo= ry are not limitations in these environments for implementation purposes; these is= sues however do have performance impacts. Screen length and width are usually 15” to 21” (diameter= ) so visual layout is limitless – within scope. This project will focus on t= he DESKTOP profile and the rule specification. See Future Work for further inform= ation.

The second profile is the WEB profile which is a subset of the first profile because the WEB has some constraints. The web toolkits HTML, XHTML, JSP = can delivery fancy interfaces although with certain limitations. Widgets used on a desktop applicat= ion might cause performance issues on WEB presentations since the presentation = is tied to a particular browser rendering capability. Other widgets might look awkward or non-functional i.e. a toolbar on a WEB page, this looks great on a DESKTOP application but awkward on a WEB page since the browsers already has a toolbar. Space and memory are issues since the WEB applications cannot store enormous amounts of informat= ion due to session limitations. A= lso, constant database hits could cause the WEB applications to be slow and usel= ess. Screen length and width depend on = the browser; but usually this is not an issue for display.

T= he third profile is the MOBILE profile. This profile can be separated into= two separate profiles those with less than 2MB memory and those with 2MB or gre= ater memory. Both profiles have li= mited user input devices; keyboard and/or mouse are typically not available (2).<= span style=3D'mso-spacerun:yes'> Space and memory are limited due t= o size of the device and available power source on the device (2). Since the CPU has limited processi= ng any instructions are expensive so should only be done if there is a strong justification (2). “A wireless device has limited bandwidth, compared to a wired device, for transmitting and receiving data” (2). Also, intermittent connections are typically not reliable since a persistent point-to-point connection is difficult.

The last profile is the VOICE profile. This profile has gained much atten= tion in the past year with the introduction of voiceXML. “Its major goal is to bring = the advantages of web-based development and content delivery to interactive voi= ce response applications” (3). &nb= sp;

Dimensions

There are three independent dimensions to any p= rofile layout, language, and widgets. The layout dimension is concerned with space, look, and feel. The language dimension handles dif= ferent choices of object oriented (OO) languages used to display the User Interface. The widget dimensi= on involves mapping MDML tags to actual User Interface toolkit specific compon= ents or methods.

The layout dimension is compromised of four dif= ferent layout managers desktop, web, mobile and voice. The desktop layout manager produce= s the exact same layout as described MDML and is the superset. All other layouts are subsets. There is a 1:1 mapping between the= MDML and the layout generation for the desktop platform. The web layout manager = is similar but there are some restrictions on layout compared to desktop. Toolbars are generally not common = in web applications so they are omitted. Menu bars are generally not at the top of web applications but they cannot be omitted because the user needs to navigate through the application. Menu bars need t= o be created but placed on the left side of the page’s content. Another major difference is = web applications use tables for managing layout whereas desktop applications use layout managers i.e. flow, box, or grid.&n= bsp; If the table tag is used then there is 1:1 mapping if the <box> tag is used then a conversion needs to happen between the <box> tag a= nd the HTML table. If the <layout> tags are used they should be converted in some way to HTML’s particular table layout. This is difficult because dynamic content is not known at runtime so= the size of the table must be assumed or displayed column-by-column or cell-by-cell.

&= nbsp; The mobile layout manager is the most restrictive layout manager. There are two issues that involve = layout for mobile units display and input. Limited display or space is a challenge to any mobile device since t= he display is small compared to any desktop monitor. The power of mobile technology is = in part due to its compactness, but application development is not the main go= al of these devices. With the J2= ME MIDP, usually full scale applications do not exist but rather smaller appli= cations called MIDlets, which require limited input because entering text is awkwar= d; there is usually no keyboard only a numeric pad. Mobile User Interface programming = is difficult, but not impossible because technology is changing rapidly.

&= nbsp; The voice layout manager is simple; there is no layout in terms of display. But there is a layout in terms of navigation. A user should onl= y be able to do things in some hierarchical order. For example, if a user does not gi= ve the correct username and password then they should not be allowed to proceed any further, not only for security purposes but for application integrity and f= low.

&= nbsp; The desktop will focus on the Java language.&n= bsp; In theory, our description is generic and could choose other languag= es that include C#. This project will concentrate on the Java language because Java provides a cross platform mechanism and easy to learn GUI toolkits. Java has three main GUI toolkits A= WT, Swing, and SWT. This paper wi= ll provide examples for Swing and SWT toolkits. Web application use HTML for static contents but for dynamic content there are a multitude of chooses JSP (Java= ), ASP (.NET), or DHTML. Mobile applications commonly use C, Java (J2ME), or OS specific languages for development.

&= nbsp; All languages with GUI toolkits have a set of widgets. Widgets are components or controls= used in GUI applications. The MDML defines general tags based on XUL (XML User Interface Language) transformed into some GUI toolkit specific widget or component. In the next chart shown there is a mapping from general MDML tag to Java based GUI toolkit component.

*Tags*	*Swing=*	*SWT*	*J2ME<= /span>*
*<checkbox>*	J= CheckBox	B= utton.SWT.CHECK	L= ist
*<textbox>*	J= TextField J= TextArea JEditorPane \|JTextPane	T= ext T= ext SWT.MULTI StyledText	T= extField
*<button>*	J= Button\| J= ToggleButton	B= utton.SWT.PUSH Button.SWT.TOGGLE	C= ommand
*<group>*	B= uttonGroup	G= roup	*=
*<label>*	J= label	L= abel	S= tringItem
*<menu>*	J= menu	M= enuItem	C= ommand/List
*<menubar>*	J= MenuBar	M= enu	L= ist
*<menuitem>*	J= MenuItem JCheckBoxMenuItem JRadioMen= uItem	M= enuItem M= enuItem SWT.CHECK MenuItem SWT.RADIO	C= ommand
*<panel>*	J= Panel J= ScrollPane	C= omposite ScrolledComposite	I= GNORED
*<image>*	I= mageIcon	I= mage	I= mageItem
*<list>*	J= List	L= ist	L= ist
*<combobox>*	J= ComboBox	C= ombo	L= ist
*<boxlayout>*	B= oxLayout	F= illLayout	I= GNORED
*<flowlayout>*	F= lowLayout	R= owLayout	I= GNORED
*<gridlayout>*	G= ridLayout	G= ridLayout GridBagLayout	I= GNORED
*<toolbar>*	J= ToolBar	T= oolItem SWT.RADIO SWT.CHECK SWT.PUSH SWT.SEPARATOR	I= GNORED
*<progressbar>*	J= ProgressBar	P= rogressBar	*=
*<separator>*	J= Separator	L= abel SWT.SEPARATOR	I= GNORED
*<slider>*	J= Slider	S= lider	G= auge
*<tabbedpane>*	J= TabbedPanel	T= abFolder	*=
*<table>*	J= Table	T= able, TableColumn, TableItem	C= ustomItem
*<window>*	J= Frame	S= hell	F= orm
*<frame>*	J= InternalFrame	*=	*=
*<dialog>*	J= Dialog	S= hell	F= orm \| Alert
*<splitpane>*	J= SplitPane	S= ash	I= GNORED
*<tree>*	J= Tree	T= ree	*=
*<customecomponent>*	*=	*=	*=

Event Mode= l

<= /span>The event model is general and shares the Swing naming convention. In this framework there exist six basic event types – action, focus, selectio= n, mouse, mouse-motion, and window. The user of the framework is responsible for implementing the functionality or classes for these events.= The framework requires the name of these classes to be specified in = the rule section. Each event type= tag contains a <Field> tag. The <Field> tag contains two attributes name and method. Name specifies the variable name a= nd method specifies the method to use with the components. Under the <Field> tag the us= er must specify the type. The <type> tag specifies the class that the backend functionality must implement. Also, if the functionality is in a different package or subsystem, that package or subsy= stem must be specified in the <import> section of the profile.<= /span>

Example 1

Define rules for Swing Events. The <Field> is used here with two attributes name and method. Name is the variable name and meth= od is the method used with the component. The <type> in this case is the class that the user must define outside the framework to handle any events.

<Event>

&= nbsp; <Action>

&= nbsp; <Field name=3D"a= ctionListener" method=3D"addActionListener"><= /p>

&= nbsp; &nbs= p; <Type>ActionEventHandler= </Type>

&= nbsp; </Field>

&= nbsp; </Action>

&= nbsp; <Focus>

&= nbsp; <Field name=3D"f= ocusListener" method=3D"addFocusListener"><= /p>

&= nbsp; &nbs= p; <Type>FocusEventHandler<= span style=3D'font-size:7.0pt;font-family:Arial;color:blue;background:white; mso-highlight:white'></Type>

&= nbsp; </Field>

&= nbsp; </Focus>

&= nbsp; <Window>

&= nbsp; <Field name=3D"w= indowListener" method=3D"addWindowListener"><= /p>

&= nbsp; &nbs= p; <Type>WindowEventHandler= </Type>

&= nbsp; </Field>

&= nbsp; </Window>

&= nbsp; <Selection>

&= nbsp; <Field name=3D"s= electionListener" method=3D"addListSelectionLis= tener"><= /p>

&= nbsp; &nbs= p; <Type>ListSelectionEventHandler= </Type>

&= nbsp; </Field>

&= nbsp; </Selection>

&= nbsp; <Mouse>

&= nbsp; <Field name=3D"m= ouseListener" method=3D"addMouseListener"><= /p>

&= nbsp; &nbs= p; <Type>MouseEventHandler<= span style=3D'font-size:7.0pt;font-family:Arial;color:blue;background:white; mso-highlight:white'></Type>

&= nbsp; </Field>

&= nbsp; </Mouse>

&= nbsp; <MouseMotion>

&= nbsp; <Field name=3D"m= ouseMotionListener" method=3D"addMouseMotionListe= ner"><= /p>

&= nbsp; &nbs= p; <Type>MouseMotionEventHandler</Type>

&= nbsp; </Field>

&= nbsp; </MouseMotion>

</Event>

Conclusion=

Many projects approach bu= ilding an XML User Interface by tightly coupling XML tags to components or methods (i.e. THINLETS, LUXOR). This project is loosely coupled si= nce the rule specification provides the mapping between XML tags and components. For example, what= does it mean to have a <MENUBAR> in our framework? With relation to a desktop profile= it means a menu bar will be attached to the given window or dialog, yet in a mobile environment menu bar do not normally exist. So in this framework because= the tags are loosely coupled it may mean a menu bar is mapped to a tree compone= nt or list component in order to provide some form of navigation to the end us= er displaying possible applications. This framework will allow the end user to change the component mappi= ng through XML rules without modifying the implementation.

&n= bsp; Another benefit of this framework is that a developer is not tied to one code generator. Although, this pro= ject uses the ZOOM code generator, one could develop an entirely new Handler from the Handler interface. The new Handler could produce the desired inp= ut for another generalized code generator or use a XSL Transformation to trans= form the Language.xml into a valid i= nput for a new code generator. Las= tly, with this framework, the ZOOM model of separation between User Interface, behavior, and structural is proven for the User Interface. The UI model can= be generated independent of any behavior or structure.

= Using this framework requires thinking about design first and code later since the code is automatically generated. Unlike the traditional approach code and run the design is extremely import in this framework since the idea is it can be used on multiple platf= orms and devices with varying degrees of richess and limitations. Also, to the novice, it might appear that this framework requires a lot of overhead, rule specification a= nd xml description to complete an application. But, once the rule specification is written generally it doesn’t have to be changed unless the application rendered doesn’t conform to the initial design.

Future Wor= k

Mobile Profile

As stated earlier the goal of the project is to produce a GUI for multiple platforms and devices not only to high-end desktop devices (Desktop Profile) but also support the low-end devices like MIDP-Mobile Information Device Profile. This section discuss= es extending the framework to support low-end devices using the MIDP Profile.<= o:p>

To support the mobile profile (J2ME) the display eng= ine must be enhanced to produce an alternate layout from the original MDML file. There are two options, = let the user produce an alternate MDML file for the mobile profile, or by adding new rules dynamically produce an alternate layout. In other frameworks like UIML the = first option is used to generate GUI’s across profiles. In this framework, the latter is desired, since the rule specification contains the toolkit characteristics.=

There are two approaches to dynamically produce convert MDML file into an applica= tion for low-end devices. The first approach uses direct widget mapping technique. All widgets and layouts that can be defined in the MDML are mapped to the widgets of the MIDP API or any other API that is to be used. So for example if the MDML defines a button then the corresponding behavior could be mapped onto the Action button on the mobile devices. Similarly text box definition in the MDML file could be mapped to the corresponding element of the MIDP API.

PROBLEM:

The first approach has a severe drawback. Remember MDML is required to map betw= een sophisticated GUI (Desktop) and non-sophisticated GUI (Mobile). The MIDP API does not support = most of a these widgets in a sophisticated GUI.= In order to map all the widgets defined in the MDML schema, several = MIDP widgets and layouts have to map to a single or no widgets in the MIDP profi= le. The following diagram shows a MDML tag to MIDP Widget mapping.

The number of widgets in the MDML clearly out number the number of widgets in t= he MIDP API and there is no one to one mapping; instead there is a group of widgets in the MDML that map to the same widgets in the MIDP set. This indicates that one or more functionalities in the MIDP will be delivered us= ing the same MDML Tag. So the fir= st approach mapping is naïve, the number of widgets that is required to display the GUI on the low-end device is not equal to the number of widgets that is defined in the GUI definition using MDML.

Another drawback is the limited screen size of the l= ow-end device. It is absolutely impossible to display a sophisticated desktop GUI = on a single page/screen of the mobile devices. So it is obvious that the single = page desktop application must be divided into multiple pages on the mobile device. The MDML schema can p= rovide a way to tell the rule engine about the page separations but there is no ex= act way to dynamically create these page separations. This could lead to a complete disa= ster.

Consider the following calculator application from t= he MDML that is displayed on the desktop.

<= /o:p>

This application is a simple desktop application. Ho= wever it is impossible to map the complete application on the display to a small mobile device. This applicati= on could show six to eight widgets on a single screen as opposed to the 41 wid= gets that shows up on the desktop application. Thus it is clear that this application needs to be separated into multiple pages. If the rule engine is allowed to perform a naïve break down, then it will show a few buttons= on one page and a few buttons on the other pages thus partitioning the other buttons in the same manner. <= /p>

Thus naïve m= apping does not break the application logically, thus making it difficult to use.<= span style=3D'mso-spacerun:yes'>

Though the applic= ation only demonstrates the problems with widgets, the same applies to layouts and other unsupported widgets like the menu bar. Also the application developer must develop the application recalling that eventually it will be required to be mapped to more than one profile, in this case a low-end device.

The above examples demonstrate major difficulties that must be addressed in ord= er to make a sensible and usable transition from a desktop application to a mo= bile application. It clearly indicates that the rule engine below could be naïve and rendering a complete useless application so the framework mu= st incorporate reasonable amount of intelligence.

Solution:

The above examples are summarized below:

Number of widge= ts required in the MIDP is equal to the number of widgets in the MDML definition.
Logical element= s are not grouped together

The solution lies in shrinking the desktop application. Instead of mappi= ng the widgets naively, an attempt must be made to map the functionality as compared to the display in a way such that it reduces the number of widgets that are required for display. In this approach, the intelligence is incorporated inside the rule engine to produce the desired output. T= here are a number of intricate details about the device that the rule engine pro= file must know. Some of them= are listed below:

Size of the MIDP display.
The widget list= that it supports.
Constraints und= er which the rule engine should work.
Other details s= uch as the support for color and other GUI applications.

The process that is described below should be used iteratively until the application could be easily displayed on the mobile device or the applicati= on could not go any further reduction.

Below is the list of steps that the application should iterate through:

= 1. &n= bsp; Chunking

= 2. &n= bsp; Reduction<= o:p>

= 3. &n= bsp; Smart Mapp= ing

= 4. &n= bsp; Eliminatio= n

Each of these steps is described below using an example. Consider the below desk= top application that is required to be shrunk into a mobile application.

3D"Text = <= /span>

0

= 1

= 2

= 3

= +

= -

= 4

= 5

6

= 7

= 8

= 9

<= /span>

Heading = with some Graphical Effects

&= nbsp; &nbs= p; &= nbsp; &nbs= p;

Below is the a description of the above steps to shrink the application

Chunking

In the first step the application needs to be separated into multiple logical pages.&nbs= p; The application has four logically distinct and separate parts, the title part, two text articles and the small calculator application. So the chunking process should mark the application into four logically separate parts.

Reduction

The text articles are way too long to be transp= orted to the mobile application. It is surely possible to reduce that one. There = are various automatic text reduction algorithms that could work around the prob= lem based on various things and reduce the text to some key words that would me= rely give the gist of the article. A good way is to take the help of the publisher of the article and reduce it.= It is certainly not possible to shrink it to only key words, so some of them c= ould be a link to a detailed description optionally that could be directed to but only if the reader wishes to.

Smart Mapping

This is the most interesting and important thin= g that should be understood very well. As mentioned earlier it might not be possib= le to display all the forty-one widgets in the calculator application on the s= ame page. It is now required to reduce the number of widgets. Given a thought t= he number widgets could be mapped to a single widget – the text box. Also the four operation widgets could be mapped with just one widget – the command button thus transforming the application into a three widget application from forty-one that could be easily displayed on a single page.= It is not always possible to reduc= e an application so drastically, but it is always possible to map a large number= of widgets to a fewer one which helps in reducing the number of widgets to be displayed.

Elimination

If the image is not giving any information like= a logo than it could be eliminated, as it would just prove to be an overhead = in the mobile application. Also the title could be displayed without any graph= ical effects as a trade off for efficiency and space.

This process could be iteratively used to shrin= k the application. Also there are specialized web browsers that could aid in displaying these images. These browsers also have capabilities to display l= arge images providing a magnifying widget that rolls over the small image on the horizontal plane.

Below is the final look at the application after iterative process.

3D"Text <= img width=3D66 height=3D132 src=3D"FINAL_files/image016.gif" alt=3D"Text Box: = Part 2" v:shapes=3D"_x0000_s1110" class=3Dshape v:dpi=3D"96"><= img width=3D78 height=3D150 src=3D"FINAL_files/image017.gif" alt=3D"Text Box: = Part 3" v:shapes=3D"_x0000_s1111" class=3Dshape v:dpi=3D"96">

The first approach that was used to shrink the application is a software engineering technique. However there is a second approach= that merges software engineering techniques with the field of Human Computer Interaction (HCI). In order to accomplish this task, this project will branch off into several different projects to complete the work for the mobile profile. These projects will concentrate on solving the above problems using methods and theories from HCI.<= /span>

&n= bsp;

Desktop Profile

Desktop Profile Itemized Future Updates:

1. &nbs= p; MDML/RuleEngine – Support Enhanced Layout Manager i.e. BorderLayout, FrameLayout=

2.&n= bsp; = ; Event Handli= ng – Support events within each rule.&n= bsp;

A user might not want to code for general event cases. Allow for specific listeners or ev= ent handlers.

Project Schedule

Initial Groundwork (Summer)

Initial Presentation (June)
Language Decision – Swing, SWT (Jun= e)
Research components, attributes, composit= ion, etc (June, July, August)

Architecture (Summer)
1. Architecture Design (August)

<= span style=3D'font-family:Arial;mso-fareast-font-family:Arial'> &nbs= p; &= nbsp; &nbs= p; &= nbsp; &nbs= p; i. &nb= sp; XSLT<= /o:p>

<= span style=3D'font-family:Arial;mso-fareast-font-family:Arial'> &nbs= p; &= nbsp; &nbs= p; &= nbsp; &nbs= p; ii. &n= bsp; Java =

Initial Architecture Complete (September)=
Intermediate Presentation (Fall)

Software Design (Fall)
1. Diagrams (October)
2. Coding (October, November, December)=

Architecture Re-work (Fall)

Architecture redesigned (December / 1 Wee= k)
Final Architecture Complete (December 12<= sup>th)

Code (Fall)

Final Implementation (December 2 weeks, J= anuary 2 weeks)
Complete (January 10^th)

Final Deliverable (Winter)

Testing examples (January)
Final Paper (February 6^th)
Final Presentation (February 6^th)

The initial project was scheduled for six months. Although, one might questions does= a software engineering project ever end; this project deliverable were extended two months.

References

= 1. = Daconta, Michael C. and Bullard, Vaughn and Smith, Kevin T. Essential XUL Programming. <= /span>New York: John W= iley & Sons, Inc., 2001.

= 2. = B’Far, Reza and Richards, Roger and Ditlinger, Stephen. Designing Effective User Interfaces For Wireless Devices. DevX.

= 3. = VoiceXML Reference. 2000. Systinet. http://www.zvon.= org/xxl/VoiceXMLReference/Output/.

= 4. = Abrams, Marc., Phanouriou, C., Batongbacal A. L., Williams, S. and Shuster, J. UIML: An Appliance-Independent XML User Interface Language, http:/= /www8.org/w8-papers/5b-hypertext-media/uiml/uiml.html.

Appendix A – Swing Rule Specification

&nb= sp;

<?XML version=3D"1.0" encoding=3D"UTF-8"?>

&nbs= p; <Profile><= /span>

&nbs= p; <Import name=3D"Import"><= /span>

&nbs= p; &= nbsp; <Statement>javax.swing.*</<= span style=3D'mso-bookmark:_Toc25684720'>Statement><= /span>

&nbs= p; &= nbsp; <Statement>javax.swing.event.*= </<= span style=3D'mso-bookmark:_Toc25684720'>Statement><= /span>

&nbs= p; &= nbsp; <Statement>javax.swing.border.= *</<= span style=3D'mso-bookmark:_Toc25684720'>Statement><= /span>

&nbs= p; &= nbsp; <Statement>java.awt.*</<= span style=3D'mso-bookmark:_Toc25684720'>Statement><= /span>

&nbs= p; &= nbsp; <Statement>java.awt.event.*</<= span style=3D'mso-bookmark:_Toc25684720'>Statement><= /span>

&nbs= p; &= nbsp; <Statement>javax.swing.tree.*<= /span></<= span style=3D'mso-bookmark:_Toc25684720'>Statement><= /span>

&nbs= p; </<= span style=3D'mso-bookmark:_Toc25684720'>Import><= /span>

&nbs= p; <DisplayMethod>show<= span style=3D'mso-bookmark:_Toc25684720'></<= span style=3D'mso-bookmark:_Toc25684720'>DisplayMethod><= /span>

&nbs= p; <ToolKit>SWING= </<= span style=3D'mso-bookmark:_Toc25684720'>ToolKit><= /span>

&nbs= p; <Accessor><= /span>

&nbs= p; &= nbsp; <Method name=3D"getComponent= "= returnType=3D"Object"><= /span>

&nbs= p; &= nbsp; <= /span><Parameter type=3D"String">name<= span style=3D'mso-bookmark:_Toc25684720'></<= span style=3D'mso-bookmark:_Toc25684720'>Parameter><= /span>

&nbs= p; &= nbsp; </<= span style=3D'mso-bookmark:_Toc25684720'>Method><= /span>

&nbs= p; </<= span style=3D'mso-bookmark:_Toc25684720'>Accessor><= /span>

&nbs= p; </<= span style=3D'mso-bookmark:_Toc25684720'>Profile><= /span>

&nbs= p; <Event><= /span>

&nbs= p; <Action><= /span>

&nbs= p; &= nbsp; <Field name=3D"actionListener"= method= =3D"addActionListener"><= /span>

&nbs= p; &= nbsp; <= /span><Type= >ActionEventHandler<= /span></<= span style=3D'mso-bookmark:_Toc25684720'>Type= ><= /span>