UI Widgets

This week you are going to build off of what you did last week with the scene graph. As you saw last week, the FigmaTree is pretty good. But there are still some things it cannot do. Some UI elements – like ninepatches, sliders, and progress bars – have to be done by hand. So this week you are going to have to get used to editting JSON files.

While working on the previous activity, you may have realized that the scene graph JSON format has two problems. First, it can get very large, especially as you start to design more complex UIs. And second, it is very redundant as you often repeat UI elements with minor differences to them. This is going to be a problem when you edit JSONs by hand, as it is easy to get lost in them and make a mistake.

This is the purpose of widgets. A widget is a separate JSON file that used to store a reusable UI element. Just like basic JSON format, these are covered in the CUGL Scene Graph Tutorial. By breaking up your UI elements into separate widgets, you can greatly simplify your design process.

This assignment will require that you build off what you completed in the previous activity. If you did not finish that lab, contact a TA so that they can get you started. When you finish this activity, you will see the image below.

In addition, in the final task we will challenge you to make a scene of your own.

As a reminder, this assignment is graded in one of two ways. If you are a 4152 student, this is graded entirely on effort (did you legitimately make an attempt), and that effort will go towards you participation grade. If you are a 5152 student, you will get a more detailed numerical score.

Table of Contents

• Useful Resources
• Widget Overview
• Instructions
• Submission

Useful Resources

Most of these resources from last time are useful for this activity. But since you are going to be editting JSON entries by hand, we want to re-emphasize the following resources.

JSON Tutorial

If you do not know what a JSON file is, then the above tutorial is an excellent resource for how to use them. As a designer, you do not need to know how to use JSON files in Javascript (or C++ for that matter). But you do need to know how to write and modify a valid JSON file with no errors in it.

For those of you who have taken Python, you will notice that JSON files look pretty much like (nested) Python dictionaries. And this is a helpful way to think about them. However, in Python all strings must have double quotes, not single quotes. Furthermore, all booleans are lower case (so true instead of True).

JSON Validator

When you write your JSON files, this files will have errors in them. And 90% of those errors will be the result of a comma: either a comma that is missing or an extra unexpected comma. While CUGL does provide you with error messages when this happens, it can be hard to read these error messages if you are not used to programming.

We cannot stress how important it is to use this tool while working on this assignment. While sometimes your mistakes will cause the SceneTool to produce a useful error message, other times you will simply get the dreaded “Segmentation Fault” (which means an unexpected error occurred).

CUGL Scene Graph Tutorial

Once again, this is a complete documentation of the JSON format used to create scene graphs. While this lab will take you through most of the process, you might want to keep this document around as a reference.

The Scene Tool

The scene tool that you will beusing is the same as it was for the previous lab. There have been no changes made to it. But if you have misplaced it, here are the links for the various platforms:

• macOS: SceneToolMac.zip
• Intel macOS: SceneToolIntel.zip
• Windows: SceneToolWin.zip

In both cases you run the application by typing the name SceneTool.exe. On some platforms you may need to add an extra ./ like ./SceneTool.exe.

IMPORTANT: On macOS, remember to validate the application by typing the command

xattr -cr SceneTool.exe

Widget Overview

A widget is a separate JSON file that defines a template for a UI template. We talked about these in the scene graph lecture. A widget is a separate JSON file that is stored in the widgets folder (not the json folder). It looks a lot like a scene graph with a few differences. Consider the widget below:

{
    "variables" : {
        "scale"   : ["data", "scale"],
        "texture" : ["children", "up", "data", "texture"],
    },
   "contents"  : {
        "type"   : "Button",
        "data"   : {
            "upnode"   : "up",
            "pushable" : [160,280,190,370,280,400,370,370,400,
                          280,370,190,280,160,190,190],
            "visible"  : false,
            "pushsize" : true,
            "anchor"   : [0.5,0.5],
            "scale"    : 0.8
        },
        "children" : {
            "up"       : {
                "type"   : "Image",
                "data"   : {
                    "texture"  : "play"
                }
            }
        },
        "layout" : {
            "x_anchor" : "center",
            "y_anchor" : "middle",
            "y_offset" : -115,
            "absolute" : true
        }
    }
}

The looks sort of like a scene graph JSON. Indeed, the contents of the contents object are a scene graph (in this case a simple button). The only thing that looks unusual is the variables object that comes before it.

To use a widget, you have to do two things. First you load it. Suppose the above widget is in a file called mywidget.json. Then you would need to add the following to the assets.json file, just before scene2s:

"widgets": {
    "widgetname" : "widgets/mywidget.json",
},
"scene2s" : ...

To use this in a scene graph, you would write something like the JSON below:

{
    "scenename": {
        "type"      : "Node",
        "format"    : {
            "type" : "Anchored"
        },
        "children":  {
            "button": {
                "type": "Widget",
                "data"   : {
                    "key"     : "widgetname"
                }
            }
        }
    }
}

What does this do? The game engine will take the contents object of mywidget.json and make it the contents of the button child. Hence if you have a UI element you are using over and over again, this cuts down on the amount that you need to write.

So what do the variables do? This is a form of customization. Right now the button uses the play texture for its image. What if we wanted to use a different image, called stop? There is a variable called texture, which we can set as follows:

{
    "scenename": {
        "type"      : "Node",
        "format"    : {
            "type" : "Anchored"
        },
        "children":  {
            "button": {
                "type": "Widget",
                "data"   : {
                    "key"     : "widgetname"
                    "variables" : {
                        "texture" : "stop"
                    }
                }
            }
        }
    }
}

This tells CUGL to go into contents, find the value texture, which is inside of data, which is itself inside of up, which is all inside of children, and replace the play with stop. This allows you to do minor reconfigurations like change a color or some text or even the size of a UI element. This is the type of thing you will be doing in this activity.

Instructions

We assume that you have completed the previous activity. In particular, you should still have your Figma project from that activity. Your Figma project should look something like the following.

If you did not finish that lab, contact a TA so that they can get you started. You should not use your assets.json from that previous assignment. Instead, you will be using FigTree to help you make a new version of assets.json.

1. Create a Widget for the Scene

The file assets.json is going to contain a lot of information. As the name says, it is where you load all of your assets in the game. As such, it does not make sense to clutter it up with really complicated scenes. A better option is to take each scene that you design and make it a separate widget. That makes the amount of lines that you need to include in assets.json really small. It also means you can edit individual scenes without having to worry about messing too much up.

Fortunately, making a widget is really easy. The FigmaTree supports this as well. Take you project and go to developer mode. Look for the line that says code, with tree to the right. To the right of that, there should be a symbol with two vertical lines with knobs at different heights. Select that to get the menu below.

You want to change the output format from Scene Node to Widget. Once you do that the JSON under FUGL Scene will look something like this.

{
    "variables": {},
    "contents": {
        "type": "Node",
        "format": {
            "type": "Anchored"
        },
        "data": {
            "anchor": [
                0,
                0
            ],
            "size": [
                1280,
                720
        
  Show 727 more lines of code

See the "variables" and "contents"? That is how you know this is as Widget. While the variables are empty, that is okay. Widgets are not required to have variables, especially if you do not plan to customize them.

Go into the widgets folder of SceneTool (you did download a copy of the new SceneTool, right?) and make a new file called labscene.json. Copy all of this JSON into this file.

Next open up the new assets.json, which should look like this.

{
    "textures": {
    },
    "fonts": {
    },
    "widgets": {
    },
    "scene2s" : {
        "lab" : {
            "type"      : "Node",
            "comment"   : "This is the root node of the scene for the lab",
            "format"    : {
                "type" : "Anchored"
            }
        }
    }
}

Remember to update the "textures" and "fonts" entries just like you did in the previous activity. CUGL will not know how to find your images and fonts without this.

This time you also want to update the "widgets" entry to add your new widget. Add a reference to the file you just created, like this:

"widgets": {
    "labscene" : "widgets/labscene.json"
},

Finally, you need to turn the lab scene into a Widget. Change the "type" from "Node" to "Widget". Finally, right after "format", add the following.

"data" : {
    "key" : "labscene"
}

Remember to add a comma after the closing curly brace of "format" and before "data". If you do not do that, this will not be proper JSON, and it will crash.

Once you do all that, it is finally time to run the SceneTool. You should be able to see the familiar scene from the previous activity. In addition, all of the buttons should work the same as last time.

2. Create Button Widgets

See those three grey buttons labeled "First", "Second", "Third"? They are all very similar, which means that they have a lot of redundant JSON. We are going to factor them out as widgets.

Again, we are going to take advantage of the Figma plugin. Go to developer mode, and then move over to the layers panel on the left. Instead of selecting root, we want you to expand the layers and select button:button1. Now in the developer panel, you should see something like this.

{
    "variables": {},
    "contents": {
        "type": "Button",
        "format": {
            "type": "Anchored"
        },
        "data": {
        "anchor": [
            0,
            0
        ],
        "size": [
            240,
            60

  Show 110 more lines of code

In the previous activity, it never made sense to select a sublayer, because we always wanted the entire scene. But now that we are making widgets, it does.

Make a new file called textbutton.json in the widgets folder. Copy all of the JSON into this file. Then go to assets.json and add this entry to the "widgets" group. Remember to have a comma between entries in this group.

Finally, we want you to turn the first button into a widget. Search for "button1" in labscene.json. You are going to need to make some changes to this entry. As a general rule, you want to keep everything after "layout" as that information information comes from the parent, not the widget. But the entries "format", "data", and "children" can all be deleted (as that information is stored inside the widget). Instead, the "data" entry should be replaced with the following.

"data":  {
    "key": "textbutton",
},

Do that, and load up the SceneTool. You should still see your original scene unchanged.

Now, do the same for the other two buttons: "button2" and "button3". When you are done, you should get a scene that looks like this:

3. Add Button Variables

Before we had three different buttons. But now they all look the same. That is because they are using the same widget. But that is okay; we just need to add some customization features to the widget.

Replace the "variables" entry at the top of textbutton.json with

"variables" : {
    "text"   : ["children","up","children","text","data","text"],
}

How do we interpret this? Look at "contents". In that group there is an entry called "children". Inside that entry is another one called "up". That has some children, one of which is called "text". That node has some data, with the value "text". That "text" is currently assigned to "first".

We are telling CUGL that we want that value "first" to be replaceable. We are creating a variable called "text" and specifying how to reach it in the contents section. We did not have to call this variable "text". We could have called it "word" or something else if we wanted to.

Now go back to labscene.json. For the entry for "button1" change the "data" entry to the following.

"data":  {
    "key": "textbutton",
    "variables" : {
        "text" : "first",
    }
},

This tells the widget we want to use "first" as our text in the button. Do the same with "button2" and "button3" but make the text "second" and "third", respectively. Now when you run the SceneTool you should see your original scene with three different buttons.

Before moving on to the next step, we want you to add one more variable to textbutton.json. Add one for ["data", "size"], the size of the button itself. It can be named anything you want.

4. Add a Nine-Patch

When we made the buttons, we had a grey background that was exactly the right size of our button. But that will not always be the case. As we will see later in class, image files take up a lot of memory on the computer, so you want to minimize the number of them. You would like to have a single background that works for buttons of various size.

Go into assets.json and look at the "textures" entry. Replace greybutton.png with greynine.png. This is a much more compact image designed to be used for buttons. Launch the SceneTool again. You should see the following image.

Note that if your image does not look like this, it is possible that you never set the constraints on the button backgrounds to left-and-right/top-and-bottom like we told you to. That contraint is necessary to get an image to fill the entire node. To fix this, you can go into textbutton.json and find the "greybutton" entry. Find the "layout" values of this entry and change both "x_anchor" and "y_anchor" to "fill".

Even if it is working correctly for you, this obviously not want we want. The problem is that CUGL is stretching the image to fit and this is distorting the corners of the rounded rectangle. How do we fix this?

The solution is to use a nine-patch, which is an image that can be resized without distortion. Nine-patches work by breaking the image into nine quadrants. The corners do not stretch at all. The sides only stretch along one access. And the intertior stretches in all directions. To define a nine-patch, you need only take an image and define its interior. The other quadrants will be computed automatically from that. The file "greynine.png" in textures is a nine-patch image, and its interior is as shown by the dotted blue box below.

Creating a nine-patch is just like creating an image node. The only difference is that you have a data property for the interior, which is the box show above. The interior for this particular image is defined by

"interior" : [27,29,42,40]

The first two values are the x and y values of the bottom-left corner of the interior. The second two values are the width and height of the interior, respectively. All values are in pixels (because you are referring to a file and not a scene).

When you create a nine-patch you must also define the size. The only exception to this is if the "x_anchor" and "y_anchor" values are both "fill". In that case it expands to fill its entire parent. That is why it was important for you to set this constraint in Figma.

To turn the buttons into nine-patches, go into textbutton.json. Find the entry for "greybutton". Set the type to "Ninepatch". Finally, add the interior to the "data" group. When you are done, you will see your original image once again.

Now it is time to show off the power of nine-patches. In the previous step we asked you to add a "size" variable to your widget. Go into labscene.json and set this variable for each of the buttons. The first button should have size [220,60], the second [240,60], and the third [260,60]. When you are done, run the SceneTool. You should get the following image.

5. Add a Slider

It is now time to add a new UI element to your scene. This time we are going to add a slider with a knob. This is a common UI element, especially on mobile devices. This element is not supported by the Figma plugin, so it will require some manual JSON editting. However, it is possible to mock up the slider first, to minimize the JSON that you need to edit.

In the latest version of the SceneTool, there are two new images in the textures folder: knob.png and slot.png. Place both of these into your Figma scene (named knob and slot, respectively). They should be under the root frame, but not under any other layer. The slot image is going to represent the track of your slider. Frame that image and rename that frame slider. Put the knob image inside this frame as well. The position both of them below the menu, as shown below.

The constraints on the slider frame should be Center/Bottom, as we want to anchor it to the bottom center of the screen. The other constraints are less important, but Center/Center is alway a good idea if you are unsure.

It is time to add this slider to CUGL. However, you have spent a lot of effort modifying this scene, so it would be a pain to have to go through all those steps again. Once again, you are going to use widgets to solve the problem. Create a new widget slider.json. Select the frame slider in the layer panel and copy the widget code into the file.

When copying the widget to a file, you will need to make one minor change. The anchor of the slider (on line 9 is not quite right. Figma only sets anchors when a node is a child of a parent. In this case the slider is the root, so the anchor was set to [0,0]. We want the slider centered, so the anchor needs to be set to [0.5,0.5] (and indeed this would be the case if we selected root instead).

You now want to place the slider in the scene. Go into labscene.json and find the very first entry for "children", which are the immediate children of the entire scene. The first entry should be for "background", which is the background image. We always want that to the first child, as children are drawn back to front. But the slider can be the second child, immediately after it. Add this entry between "background" and "startmenu".

"slider" : {
    "type" : "Widget",
    "data" : {
        "key" : "slider"
    }
},

Also, remember to add the slider.json to the list of "widgets" at the start. Run the SceneTool and you should see the slider. It will be a static image and not do anything yet, but it will be there.

However, it will not play well with the changes in aspect ratio. That is because it is missing the layout information. Widgets do not have layout information, because that is assigned by the parent. To find the layout information, select the root in developer mode, as that is the parent of the slider. Copy the JSON code into another blank file so that you can search it. Look for "slider" in this file, and find the layout information. Copy "layout" and everything after it into assets.json in the correct place. Remember to check your commas before running SceneTool. Now the image will work correctly at different aspect ratios.

Of course, the slider is still a static image. We need to turn it into an actual slider. Once again, we will need to modify some JSON. Fortunately, this can all be done in slider.json, which is a much smaller file. As a first step, change the type of this widget to "Slider". But if you run the SceneTool now, it will crash. That is because the slider needs some extra information in "data" to work properly.

In particular, a slider must have the following entries:

• "bounds": The location of the track in the slider
• "knob": The knob image
• "path": The slot image

Of these, "bounds" is the most important. It specifies the line for the slider track. The knob will always stay on this line. The line is specified as a rectangle [x,y,width,height]. The line starts out at (x,y) in the slider coordinate space, and goes to the opposite corner of the rectangle. So, if you want a horizontal slider 100 units long starting at (20,50), you would specify [20,50,100,0]. Similarly, [20,50,0,100] would create a vertical slider from the same starting point.

We want these "bounds" to line up with the slot image that we placed. If you look at this image in a graphics program like photoshop, you will see that the left side of the slot interior is a rounded areay centered at (18,18) while the right side is centered at (382,18). This means that a bounds of [18,18,364,0] makes the most sense.

Add that value to "data" and run the SceneTool. You should see something like the following image.

What is going on here? You did not tell CUGL that you had images for the knob and path, so it created them for you. It created some crude images using the geometry tools in CUGL. Not particularly pretty, but they get the job done. With that said, this is useful as a first attempt, as it show you where the slider will actually be positioned once you hook up the images.

It is now time to add the images. Assuming you named the Figma layers knob and slot, add the following values to "data"

"knob" : "knob",
"path" : "slot"

Run the SceneTool again. Now you will see the following.

Touch the knob and move it. You now have a working slider!

Sliders have several other properties that are not visually apparent, but are important to your programmers. Whenever you slide the knob about, it produces a value. The value is where it is along the track. But this value is not a position. Instead, the slider has an associated "range" which is a min and a max value. If the slider is all the way to the left, it is the minimum value and if it is all the way to the right it is the maximum value. By default the range is [0,100], but you can change this in the JSON.

You can also change the initial value of the knob. By default, the knob always starts in the middle of the range. But if you assign a "value" in the JSON it will put the knob there initially. Set the value to 25 and watch what happens.

Sliders have many useful settings, including snapping to tick marks. All of these are all covered in the scene graph tutorial. You should explore these on your own time.

6. Create Your Own Scene

Up until now, you have been creating an scene that we designed for you. It is now time for you to show off everything you have learned. Make us a new scene! The scene should be a widget so that we can swap it out with labscene.json when grading. You should add this widget to the "widgets" section of assets.json.

The scene should use new images and a new font. Make sure that you have the appropriate rights to use the font (Helvetica is not okay for embedding into a mobile game!). The website dafont.com is a good source of free fonts. If you do not make your own images, make sure they are properly sourced as well.

This scene should have buttons and a textfield. Remember that you can make a textfield in Figma by tagging an text object with edit:. You will need to give the textfield a fixed width and height so that there is enough space for typing. See the discussion in the last activity for more information.

Beyond that, the scene is up to you. Impress us!

Submission

Due: Sun, Feb 23 at 11:59 PM

This activity is a little more work than the other two. That is because we had to build up your skills to get here. But this activity should be very doable for anyone who completed the previous one. Unlike that activity, this time you modified more files than just assets.json. So this time we want you a zip file containing the following:

• The file assets.json
• Everything in widgets
• The new image file(s)
• The new font file(s)
• A README.txt describing the purpose of your new scene

Submit this zip file as lab3design.zip to CMS