Networking

Many of you are interested in multiplayer games. But networking is a potentially difficult topic. Fortunately, CUGL provides a very simple to use networking interface that is designed for cooperative and competitive games between local mobile devices. This interface was inspired by the the success of Family Style and is based on the code base from SweetSpace, a game designed for this course in 2020. Unlike Family Style, this interface is based on the open source library Slikenet, which helps cut down on licensing and server costs.

In addition to working with the networking library, this lab is the first introduction that the programs will have to the scene graph tools. The UI elements in this game – text fields and button – are all defined by the JSON scene graph specification. You can find this specification in assets.json under scene2s. To simplify this activity, we have completed the UI elements (and the code that instructs them what to do) for you. You only have to add the networking code.

Once again, this assignment is graded in one of two ways. If you are a 4152 student, this is graded entirely on effort (did you legitimately make an attempt), and that effort will go towards you participation grade. If you are a 5152 student, you will get a more detailed numerical score.

Table of Contents

• Project Files
• The User Interface
• Application Overview
• Instructions
• Submission

Project Files

Download: NetLab.zip

You should download the project for this assignment. This is a self-contained project just like all of the ones available from the demos page. You should pick either XCode or Visual Studio as your primary development environment. That is because these IDEs have the best debugging tools. However, if you plan to use Android, you will need Android Studio installed as well.

Running the Network Lab

For once we have an application that runs and runs the same on all platforms. Yes, you can input text into a a text field on a mobile device like Android or iOS. In that case it will pull up the virtual keyboard. The only thing you need to do is to write the networking code, which is exactly the same across all platforms.

The problem is testing the application. For that you will need two devices running the project. One device should be host and one should be a client. You can do this with two laptops but we prefer to use a laptop and a mobile device. That is because you are constantly updating the code, and you would have to synchronize this across the two platforms (though GitHub could solve this problem). With a mobile device, you could plug it into your laptop and code for both at the same time.

Note, however, that XCode will only run the application on one device (laptop or phone) at a time. To solve this problem, I first compile and run the application on my iOS device. I then stop the application and disconnect the iOS device from the laptop. The code is still installed on the iOS device, so now you can run it normally (you just will not see any debugging print statements). Now run the application on your laptop and connect the two

The User Interface

The code for NetLab is very different from the previous labs. Look at MenuScene, which is the first scene launched after the loading screen is finished. There is no update or render method. What is going on? We are inheriting these methods from Scene2. This class is used to handle scene graphs, particularly scene graphs with UI elements automatically. The only thing we need to do is to define the scene graph and hook up all the listeners/callback functions that respond to those UI elements.

So where do we define the scene graphs? This is actually done in assets.json. If you look at that file, you will see a JSON object called scene2s. This is a collection of scene graphs. The object menu is the scene graph for MenuScene. Each of the other scenes have a scene graph defined in this file.

Because this is a data asset, it is loaded with the asset loader. You can then access the elements by name. Names are separated by underscores ( _ ). So the menu button for the host scene is referenced by the name menu_host. We intended to refactor the separation to use periods (as this is more standard), but this was not completed in time for the semester.

You can see us accessing these nodes in the init method. We have to use std::dynamic_pointer_cast because the asset manager does not know what type of scene graph node each asset is. Once we have the object, we can add a listener. Note that listeners are specified with lambda functions as described in the second C++ tutorial. In this project we use this to specify our variable capture. This is typically the best approach in a class-based application as it means we capture all attributes and methods of the surrounding class and that we can use them in the lambda function.

There is one more thing. Just adding a listener to a button does not guarantee that a button works. You also have to activate it. An activated button accepts clicks while a deactivated button does not. The reason this is important is because an activated button will still receive clicks even when it is not visible. So if you switch from the menu scene to the host scene, and the menu scene buttons are still active, they will still register when you click on them.

To solve this problem we need to activate UI elements when a scene is active, and deactivate them when the scene is not. In addition, you also may need to reset the UI elements to a default state when the scene is made active. For example, if we click on the host button in menu scene, it takes us to the host scene. And if we click on the back button in the host scene, it takes us back the menu scene. If we do not reset the host button, it will still be down when we return and we will be in an infinite loop between the two scenes.

As a result, the primary methods for handling user interface elements in a scene are init and setActive. All of the scenes for this game are very similar in that regard. Look at them for inspiration on your own project.

Application Overview

The user interface is complete in the provided project. You do not need to add anything new to make it work. You can click on the buttons to go between the various scenes. You can even “play” the game, clicking on the colored buttons to change the color of the background. To start off you might want to just play around with how the scenes fit together.

The game itself is simply. You press a button and the screen turns that color. When the game is fully networked, pressing a button will change the color on your screen as well as that of any connected screen. So the only difficult part of this application is understanding the networking.

The Punchthrough Server

Every game must have an external server. This server acts as the “lobby” for your game. It is where players find each other, so that they can group themselves together. The server does not have any game specific code; it only groups players. The primary reason for this server is so that you can connect to another player hidden by a NAT. That is why we often call it the NAT punchthrough server.

We have provided you with a NAT punchthrough server. Its address and configuration is in the file server.json. This server will work for this lab. However, you may not use this server for your game. You are expected to create and host your own server. Fortunately, we have provided you the tools to make this easy.

Docker Container

Every game can use the same punchthrough server. There is no need for custom code. That is why we packaged a Docker container for you to use. Simply launch it and point your application to this server by changing the values in server.json.

Source Code

If you really think that you need to make changes to the punchthrough server (why?), you can do that. However, you are on your own. We ar enot going to teach you how this server works.

The Host

As with Family Style, one player must initiate a game. That player is designated the “host”. We will talk about this designation during a later course lecture, but the important thing to know about the host is that the host controls the play session. The host determines when the game starts and when the game ends. Beyond that, the host is not particularly special. Everything a client sends is broadcast to all players, not just the host.

Because the host is in control of the play session, if the host loses connection with the punchthrough server, the game is over. There is no support for host migration in CUGL. On the other hand, clients can rejoin if they drop out, as the punchthrough server will put them back in contact with the host.

When the host successfully contacts the punchthrough server, the server will give the host a room ID. This defines the game room. All players who join the same game room will play together. This allows you to support multiple concurrent play sessions that do not interfere with each other.

The room ID is a string. It can theoretically be a string of any length with any type of characters. However, the punchthrough server we have provided gives 5 character strings made up of numbers. This allows for up to 99,999 rooms. This should be enough unless you are Family Style. Just make sure to write your interface so that the size of the room string does not matter.

The host then waits for players to join. In our application there is a text label indicating the number of players so far. In your game you may want to convey more information, such as the names of all players. When the host believes that everyone has joined, the host starts the game.

The Client

While the host joins the punchthrough server immediately, the client must wait for a room ID. That is why this network interface is for local games. The host must communicate the room ID to the players outside of the game (typically verbally). The room ID is a password that allows the players to join that game.

In this application we have a text field. You activate the text field by touching with a finger or a mouse. You can then type text into it. On a mobile device, activating a text field will activate the virtual keyboard. The players type in the room ID and hit start. This will connect the players to the host but will not start the game. The game is only started when the host indicates that is started.

While the text field approach is simple and works for this lab, it is not the preferred approach for mobile devices. It is much easier on the player to ask them to press buttons. This is the approach that Family Style took, as seen below (from the host’s perspective).

Instructions

Through this assignment you will work on three files:

• NLHostScene.cpp
• NLClientScene.cpp
• NLGameScene.cpp

For once you should not need to modify the headers. Each of the files above has a section marked student methods. The methods in this section are either empty or incomplete. It is your job to complete them.

When you write your game, you normally factor out all network processing and controls into a NetworkController, just like all of the previous labs had an InputController. But in this lab, we have decided to separate the network code for each scene. This makes it very clear what the network controller is doing at each step of the process.

However, this separation can lead to some nasty race conditions, which we will discuss later in the instructions. But you should not worry about those. The purpose of the lab is just to get everything connected. We will worry about synchronization issues when we give a proper lecture on networking.

Task 1: The Host

The first thing to do is to set up the host. There are six methods in the student methods section.

• update, which receives the latest update from the server
• processData, which interprets the data from each update
• connect, which joins the passthrough server
• checkConnection, which checks the network status
• configureStartButton, which updates the user interface
• startGame, which starts the game

Despite being part of the student methods section, the first two are done for you. We completed update so that you had an idea for how to implement it in the other scenes. And processData is unnecessary because the clients don’t send any data to the host before the game begins. The later will likely not be true of your game, as most games like to send player names or something similar.

The key thing about update is that you must call receive every animation frame, even if you are not expecting any messages. This is how you update the connection state. And if you go to long without calling this method, the network interface will drop you for inactivity.

Connect to the Server

The first thing to do is to connect to the server. The host scene has a _config attribute that was defined in _init. Read the documentation for NetworkConnection use the alloc method to create a smart pointer for a host connection. Assign this to _network. You should also immediately call checkConnection().

Check the Connection Status

The function checkConnection should look at the status of the network connection. As the status of the network changes, the status of the scene should change to match. If the network status is Pending, the scene status should be WAIT. If the network status is Connected, then the scene status should either be IDLE or START, and it should be IDLE the first time connection it established. Finally, for any network status of RoomNotFound, ApiMismatch, GenericError or Disconnected it should disconnect the network connection and set the state back to WAIT. The latter case is also the only time this function should return false.

In addition, when this host is connected for the first time, you should get the room ID and assign the text of the _gameid to this room value (otherwise how else are the players to know where to go). Similarly, you should update the number of players when it changes.

When you have finished this step, you can actually test your application for the first time. Start the application and go the host scene. Do you seen a room number? If so, you are connecting successfully!

Configure the Start Button

Why do we need a function to configure the buttons? The button will be deactivated until the game has room ID (this is very much not the case now). In particular, until connection with the punchthrough server is made, the button will say “Waiting” and will be deactivated. It is only once that connection is made that it will be activated and changed to “Start Game” (which is what this method does right now).

Changing the text on a button is somewhat tricky. That is because a button does not have a text attribute. It only has two children: up (what it looks like when it is up) and down (what it looks like when it is down). And technically down is optional, as it can just darken up. But up does not have a text attribute either. It has two children: one for the text and one for the image. So changing the text requires that we set the text attribute in the child of child. To simplify this process we have provided the method updateText.

You should not need to access _network to implement this method. Everything should be clear from the scene state and the current properties of _startgame.

It might seem weird that we are having you implement this method. In a lab that is about networking. But look at where this method is called. It is called in update, because the buttons depend on what is happening on the network and not what the player is doing. This will be quite common in your application.

Start the Game

The host can start is own game simply by setting _status to START. The NetApp class will see this change and swap the scene. But that is not good enough. You want the clients to start as well. How do you do that? By sending a message to the clients that they should start.

The send method in NetworkConnection is very straight-forward. You send a vector of bytes. If you want to send another type of data, you need to first convert it to a vector of bytes using NetworkSerializer. However, that is not necessary in this assignment. You just need to send an array of bytes.

In fact, because clients know to start when they see the first message from the host, you only need to send one byte. We sent the byte 255. But you can choose whatever you want.

Task 2: The Client

It is now time to connect the client. This time there are five methods in the student methods section, all of which are unimplemented:

• update, which receives the latest update from the server
• processData, which interprets the data from each update
• connect, which connects to the host
• checkConnection, which checks the network status
• configureStartButton, which updates the user interface

Connect to the Host

The client does not connect to the passthrough server right away. The player has to enter the room number first. Fortunately, we have written the code to process the text field and get the room number from the user. And the connect method is called by the start button (or when you press enter in the text interface). So all you have to do is set up a connection.

Once again, use an alloc method to create a smart pointer for a client connection. Assign this to _network. You should also immediately call checkConnection().

Check the Connection Status

The function checkConnection is the same as it was for HostScene. Only the logic differs, because ClientScene has a different set of states. A client starts out as IDLE, waiting for the player to enter a room. Once the network connection has been created and the network status becomes Pending, the scene status is now JOIN, indicating it is read to join a room. If the network status changes to Connected then the scene status becomes WAIT unless the state is already START. This indicates we are waiting for the other players but ready to go at any time. Finally, for any network status of RoomNotFound, ApiMismatch, GenericError or Disconnected it should disconnect the network connection and set the state back to IDLE. The latter case is also the only time this function should return false.

You should remember to update the number of players whenever it changes. This important to see if the network connection is working (or when it is dropped).

Update the Network State

We did not implement update for you this time. But it is pretty much the same as it was for HostScene. Do not do anything unless a network connection is established. If one is established, process the data, check the connection state, and configure the start button (we will talk about the latter one later).

Once you do this, it should be possible to test your connection. Launch an application with your working host on another device, be it laptop or mobile. Then try to connect to it with this one. If both devices note that there are two players then you can be confident in your connection.

Configure the Start Button

This one seems really weird. We understand that we no longer want the start buttons to say “Start Game” after we press it, but why is this function called in update. There are two reasons for this.

First, we want the button to say three different things, depending on the scene status. That means it needs to change whenever update changes the status. At the start, when the scene is IDLE, the button should be active and should say “Start Game” When the scene has status JOIN, we want to deactivate the start button and have it say “Connecting”. This text should change to “Waiting” when the scene status is WAIT.

The second reason for method is very subtle. When we press the start button we immediately switch to JOIN. But we cannot deactivate the start button each time. Why not? It is unsafe for a button listener to process code that deactivates it. This is the same as unsafely deleting from a list that you loop over. Or for those of you in the introductory course, it is the same as deleting a box2d object in the collision handler. We cannot do this. So we let update handle it for us.

Process the Network Data

The last thing to do is to process messages from the network. Since we are a client, we are waiting for a message - a message to start the game! The function processData receives a byte vector from the network. So as soon as we see our first message we know to start the game. You can check that it is a correct message (one element, with a value of 255) if you want to, but this is not necessary. Once you receive the message, set the status to START. The class NetApp will take care of the rest.

Why is this a race condition. While update is only called once an animation frame, there is no guarantee that processData is called only once. When you invoke recieve, the callback function provided is executed for each message sent that frame. So there could be more than one. In this case, the host could send a start message and quickly chose a color soon after. Both of these messages would arrive to the client at the same time.

Why is this a problem? Because the code to handle a color change is in GameScene (and we have not yet written it). As there is no such code in ClientScene, this message will be dropped. Indeed, there is no way to fix this problem, so do not try. This is exactly why we recommend that you have a single network controller, instead of spreading out the code across multiple scenes like we have done here.

Task 3: The Game

You are almost there. It is now time to play the game. The class GameScene has four methods in the student methods section, all of which are unimplemented:

• update, which receives the latest update from the server
• processData, which interprets the data from each update
• checkConnection, which checks the network status
• transmitColor, which sends the color change to the other players

In the game, the players change the background color. There are multiple ways to do this, but we chose to change the clear color in the application. This is the the color that the application uses to “erases” the screen before starting again.

Update the Network State

You should know how to do this one by now. You do not need to update a start button, but the other elements are the same.

Check the Connection Status

There is a lot less to do here, since we are not trying to connect to the a machine. We are just trying to make sure that our current connection remains live. Remember to update the number of players as they change. If you ever receive a network status of RoomNotFound, ApiMismatch, GenericError or Disconnected you should disconnect the network connection and quit (so that NetApp takes us back to the menu screen).

Oh, and if you do quit, it is very important that you change the background color back to its normal color, which is the web color "#c0c0c0". See the documentation for how Color4 objects work.

Transmit a Color

When you transmit a color, you should first set the clear color. That is because you never receive your own messages, so you cannot rely on processData to do all the work. Then transmit the color at a vector of 4 bytes. As each attribute of Color4 is a byte, this should be easy. The order (r,g,b,a or a,b,g,r) does not matter as long as you are consistent.

Process the Network Data

To complete the game, you need to write code to process an incoming message.
Whenever you receive a 4 element byte vector, turn it back into a color and set the clear color to that value. When you do this, you are done with the lab. Congratulations.

What is the problem? The network interface guarantees that messages sent from a single sender are sent in order. But there is no guarantee on the order of messages from two different senders. Suppose two players both press a color at exactly the same time. They both set their devices to the color they pressed and transmit this out. Since no device receives its own message, the next message they each receive is the color of the device. So they swap colors and are now in inconsistent states.

You will not be able to fix this problem with the way this lab is organized. Do not try. We will talk about how to address this problem during lecture.

Submission

Due: Fri, Feb 18 at 11:59 PM

This assignment is a breather after the previous one. The code is most straightforward and we have done a lot of the heavy lifting for you. When submitting the assignment do not submit the entire project. That is too large and CMS will crash under the load. Instead we want you create a zip file containing the files:

• NLHostScene.cpp
• NLClientScene.h
• NLGameScene.cpp
• readme.txt

The file readme.txt should tell us if you worked in Visual Studio or in XCode for the desktop version. You should also tell us how you tested your network connection (two laptops, two mobile devices, or one of each).

Submit this zip file as lab3code.zip to CMS