Arduino to Unity Project Part 9: Adding to the scene + jumping

Adding to the scene

I had now added two analog thumbsticks as input and could could use them to control a FPS Camera in Unity. However the very simple Unity scene I had created where making it difficult to se how well this worked. To be able to improve and adjust the details of the interaction I needed a better scene. I went to the Unity Asset Store: https://www.assetstore.unity3d.com/

This store is great, you can buy assets and directly from the store import them into you Unity project. Since this was a school project, I chose a scene which someone made available on the store for free:
https://www.assetstore.unity3d.com/en/#!/content/6459.

As described in earlier post, adding a script to an object is as easy as dragging it on to it. So after opening the scene in Unity, all I had to do was drag my C# script on to the first person camera included in the scene and then I could start using it with my controller just as in my previous much simpler scene. Great!

However, this scene had terrain. The ground was not at the same level everywhere but the where hills and valleys. As long as I was moving to lower levels everything was fine. But when I tried to go up a hill, to a higher levelled terrain my character would fall through the terrain and of the whole plane to an endless fall. As I have mentioned before, I am a novice in Unity. I started investigating if I had to add some kind of physics component to my FPS Camera object. There is something called "Rigid Body" and something else called "Collider". I was a little bit confused by this because whatever I added, I still fell right through. I tested to control the FPS Camera with the mouse and keyboard again and I did not fall through terrain, so it had to have something to do with my script.

I compared the movement of the FPS Camera Object using the mouse & keyboard vs my controller to see how they differed and then understood something that might be very obvious to someone with more experience with Unity: The FPS Camera asset object that comes with Unity really consist of two sub objects called Graphics and Main Camera. When moving the mouse left to right that rotates the whole FPS Camera object. However when moving the mouse forward/backwards only the Main Camera sub-object is rotated. So to replicate that action with my controller I needed to access the camera sub-object. You can do this by the function GameObject.Find() and then use "/" to move down in the hierarchy of you game object. So in my case it was :
cam = GameObject.Find ("First Person Controller/Main Camera");

This worked!

Jumping

However, as I was investigating physics and terrain in Unity I found this checklist for people with problem with falling with terrain:
http://answers.unity3d.com/questions/147687/checklist-object-or-character-is-falling-through-t.html

It says that it is bad practice to use transform.Translate() in a script to move a character and it is better to use CharacterController.SimpleMove . It turns there are built-in functions to do what I was thinking up how to do in Unity(of course there are!) I ended up using CharactedController.Move instead and I could pretty much replicate the example script given on the Unity documentation page.

This example script also came with a jump button implementation. The CharacterController-class of Unity there is boolean .isGrounded to check if the character is on the ground or not. We check if the character is on the ground(you can't jump if you are not on the ground in this implementation) then we check if the button of the right analog stick is pressed. I think I mentioned this earlier, but the thumbsticks input sensors more than having the two analog potentiometers also have a digital button which switches between 0 and 1 when you press down on the analog stick. So if the character is on the ground and the button is pressed, we use the CharactedController.Move  to move the character along the y-axis resulting in a "jump". Then we move the character down again over time, which represents the end of the jump when gravity is returning the character to the ground.

Arduino to Unity Project Part 8: Adding a second analog stick

Once I knew how to get input from one controller, send it to Unity and use it in a script there, it was very easy to replicate. I just connected the second stick to the arduino just like I connected the first one. The sent data from both sticks over serial. In Unity I only had to make it so that position data from the right stick was not used to do tranform.Translate() in unity but instead transform.Rotate():
http://docs.unity3d.com/ScriptReference/Transform.Rotate.html

For moving the FPSCamera-objekt right or left, I sent a vector of (0, Righstick X-position, 0) to  rotate around the Y-axis. To get this right, I also had to state to the function to rotate in relation to the world(the scene) y-axis and not the object itself.

Then I repeated this function call but for rotation around the X-axis in order to move the FPSCamera-objekt up/down.


This was the result (sorry for bad filming, It starts to get really hard to film and control the two thumbsticks at the same time!)

Arduino to Unity: double thumbsticks-input from Evert Lagerberg on Vimeo.

Arduino to Unity Project Part 7: Something to hold on to

As I was testing my analog stick, since it was not attached to anything, I needed to hold the breakout board with one hand and steer the analog stick with the other hand. The next step in my project was to add a second analog stick at which point it would get impossible to hold everything still. I needed to build some kind of stand or board to fasten my analog sticks.

The aestetics and ergonomy of how the input sensors of a interactive controller are placed is of course very important for the user experience. If I choose to continue this project I would want to design a more embodied interaction. At this stage of the project though, I really just needed something that would allow me to place each of my thumbs on the left and right thumbsticks.

Since my arduino wires are quite short my best option was to somehow build in the arduino together with the thumbstick sensors and then have the much longer serial to usb-cable that connects a arduino to a computer, run out of the controller. I found a carton box of the right size. I placed the arduino inside of the box and cut holes in the roof of the box for cables so that I could have the breakout board and the thumbstick outside and on top of the box. I used blu-tack to secure everything. Not a very permanent solution but it worked for now.

From the inside: The arduino connected to the computer via a serial-to-usb cable running out of the box. The analog sticks are connected to the ports of the arduino. I also had to use a breadboard since I now wanted to connect both analog sticks to the 5V and Ground ports of the arduino.

From outside. This allowed me to hold the box with one hand on each side and resting each thumb och each thumbstick

Arduino to Unity Project Part 6: From zones to direction vectors

In my previous implementation of the Arduino directional input to Unity I divided the scales of the Y and X-axis ranges and gave each zone a number 0-8. This way I only needed to send that number over Serial to Unity. But that also meant that I only could input 8 directions(+ 0 for standing still). After doing some state of the art research I decided that this was not enough. I wanted to be able to input any direction(360 degrees).

To do this we can picture the the analog stick in a coordinate system where the rested position of the thumbstick is in the origo(x1, y1). If I then move the thumbstick in any direction that is going to change the potentiometers value so that I get an x2, y2 position.

v = x2-x1, y2-y1 then then gives me a vector which goes from origo to the new position of the controller. And if I normalize that vector I get a directional vector which is vector of length one that describes a direction. I could then use Unitys tranform.Translate()-function to move an object in that direction in real time, following the user input of X and Y values over the serial port.


Setting up a coordinate system.

First I had to reverse the range of the X-axis from the thumbsticks horizontal potentiometer. For some reason it was made so that the vertical scale was growing upwards but the horizontal scale was growing to the left. To get this to work as a conventional coordinate-system I needed to reverse it so that the X-axis was growing to the right.

For some reason the horizontal scale grows to the left?

Next I calculated the delta X and delta Y. Now the axis range from -510  to  510 with 0 in the middle.

Sending data over serial

Previously I only needed to continuously send number over the serial port to Unity. I could do that with Serial.write() in the Arduino IDE and read it with ReadByte() in Unity. So I was sending one byte with each package. However, 1 byte can only hold a value 0-255. My values where between -510 and 510 and both for the X and Y axis which meant it would no longer fit inside a byte. From reading online it seems possible to send multiple bytes with the Serial.write()-function, however I am not sure how I would receive that without knowing the exact number of bytes sent. I could also try to split the data up into several packages but that would require some algoritm to check when all the parts of a package has been received. Instead I used the Serial.print()-function which prints "Prints data to the serial port as human-readable ASCII text." as it says in the Arduino documentation. I sent the X and the Y value seperated by a "," and then I could use ReadLine() in Unity and then split and parse to get the values. This certainly does not seem like the most elegant method of sending numerical data(converting int to string, converting back to int) but I am yet to find a better way nor anyone else who have done it. Truth is that information on Arduino to Unity Serial communication is quite limited online.

I wrote a C#-script in Unity to read the serial data, split and parse it to X and Y valuebles. Then I created a Vector3(X,0, Y) and normalised it to get a directional vector to describe the direction that I wanted to move the character (in this case the Unity FPScamera). This scrips seemed to work fine. However I had some problems with the serial communication. It seemed that there was a sync problem between the arduino sending code and the unity receiving. My theory is that ReadLine() takes longer than ReadByte() which is why the serial connection are not as direct (write -> read). This causes a async problem. I had to play around with delay(ms) on the Arduino side and ReadTimeout() on the Unity side to make them sync a little bit better. However, if the delays where too high that would gravely affect frame rate in my Unity scene. I finally found out that 4ms delay in arduino and 5ms ReadTimeout() in Unity seemed to work OK. I also had problems with Unity sometimes not parsing correctly and I have a lot of different exceptions when running. I need to optimize the code. But this is the version I have at this point:

Arduino to Unity: Thumbstick X, Y coordinates to calculate exact direction vector from Evert Lagerberg on Vimeo.

It works OK. Framerate is about 60 FPS and acceptable. It is now possible to move in any direction instead of just 8.

Arduino to Unity Project Part 5: State of the art game controller conventions

As of part 4(http://evertlagerberg.blogspot.se/2015/06/arduino-to-unity-project-part-4.html) I have created a 8-directional joystick input from an Arduino to Unity. I have created a script to read the Arduino input over serial in Unity and added that script to control the movement of the FPS Camera.

The next natural step of my project according to the limit-section of my project specification(http://evertlagerberg.blogspot.se/2015/05/arduino-to-unity-project-part-1-project.html) would be to add a second joystick to control camera rotation of the FPS camera in Unity. However, playing around with my 8-directional joystick I started thinking about game controller conventions. How many directions should I be able to move in? Is 8 directions sufficient? This blog is a brief study on game controller conventions.


D-Pad:
The d-pad(short for directional pad) was an early game controller convention, used on home gaming consoles such as the NES, SNES and Playstation among others. The d-pad have four digital buttons which each gives input of one direction (UP, RIGHT, DOWN, LEFT). However in some implementations of the d-pad, the user can get diagonal directions by pressing two buttons. For example pressing both UP and RIGHT would result in a direction input of a diagonal direction between up and and right. Today these are mostly used in 2D-plattform games. The user input in most games a quite straight forward. The user presses the button in the direction he/she wants to move the character and the character moves in that direction.

Joysticks/Arcade sticks:
Joysticks or arcade sticks are a directional controller type that consists of a single stick which gives a digital input depending on its position. Joysticks are available in 2-way, 4-way and 8-way set ups, describing the amount of directions possible to move. Today these are mostly used for fighting games where you want to execute exact button combinations do perform in-game combo attacks.

Dual analog sticks:
Modern home gaming consoles often uses the convention of two dual sticks. The sticks placed on a controller held with both hand and each thumb is resting on one thumbstick. The convention is that the left thumbstick is controlling the character movement and the right thumbstick is controlling the camera rotation. For a first person perspective game this means that the user walks by pressing the left thumbstick in some direction and control the direction the character is turned towards with the right thumbstick. The user input of analog sticks enables user input in any direction (360 degrees) for each stick, meaning that the user can move/rotate in any possible direction. 

This user input requires some training to get accustomed to. Novice user sometimes think it difficult to control these two different movement systems at the same time (plus a dozen other digital buttons usually on these modern controllers). 

Keyboard(WASD) + Mouse:
On PC the controller convention is to control the movement of the player character with the keyboard and rotation of the camera with the mouse. For a first person game this mean the user walks with the help of four digitial buttons on the keyboard (W, A, S, D) and controll the direction the caracter is turning to with the mouse. This is interesting, because the keyboard + mouse controller conventions only have 4 directional input instead of 360 degrees as the dual analog stick. However often the same game are available over a range of platforms where both these conventions are present. I ended thinking about this a lot, until I decided to do a small comparison.

WASD vs Dual analog Comparison:

I tested controlling a first person perspective game first with a WASD + mouse and then with a dual analog stick controller. The "game" I tested with WASD was the Unity FPSCamera. With WASD you can only mouse in four directions. The convention is that you move with the same speed in all directions. Moving sideways by pressing A or S and holding the mouse relatively still makes you move sideways while not turned that way. In real life this movement would be called side-stepping but in games it is called "Strafing". This movement is unnatural since in real life a human can not side-step without decreasing movement speed. In game however, strafing is a common movement convention. However, the the mouse is used to rotate the camera. Moving the mouse forward/backwards results in a motion that in the real world would be moving the head up and down. Moving the mouse left or right represents a motion that by real world terms could be described as turning the whole body in that direction. This means that pressing the digital button W on the keyboard will move the camera forward, however, by turning the mouse to the left/right, changes the rotation of the camera and therefore the direction that is forward. In this way, the WASD + mouse conventions can still move in any possible direction (360 degrees)

The game I tested with a dual analog stick was Bioshock Infinite and I used a Playstation 3 controller.
In comparison to the WASD for movement control, the left thumbstick enables the user to move in any direction(360 degrees). I am not sure if the analog controller input is divided into zones or if it is really completely "any direction", but the user experience is that you can move in any direction. In other words, the control is sensible enough that I can move in a direction of 35 degrees or 45 degrees. However, playing the game I noticed that I am do not give very exact inputs down to a certain direction degree. Rather I tend to move forward and backwards and strafe left and right, occasionally moving diagonally, oftenthe 4 direction convention of the WASD. To do exact movement, instead I use the right stick to turn the direction of "forward", just as I would to with the mouse.  However, the right stick movement is not as quick and exact as using a mouse, so my experience was that I am compensating the slowliness of the right stick, by also using the left stick(more than I would WASD) to adjust movement direction. The combination of the the dual sticks then is a quite complex user input, both controlling the camera movement and the right stick also controlling camera rotation. It is not entirely obvios how to understand the coagency of both thumbs in this set up. I find it very complex and intersting in a way that I have not thought of when previously playing video games.

The camera rotation of the right stick can be made faster/slower depending on how far the analog stick is pushed in a certain direction. If I push the right thumbstick a litte to the right the camera will slowly rotate that way, however If I push it further the the camera rotation will be faster. The user experience is that the change of speed is not in steps but continues. I also noted that the maximum rotation speed UP/DOWN was set to much slower than rotation LEFT/RIGHT in the particular game I tested with(Bioshock Infinite).

Conclusion:
I will use the dual analog stick convention  and try to use the interaction and user expereience from the control of Bioshock infinite to shape my user input.

Arduino to Unity Project Part 4: Thumbsticks 8 directions input to Unity

I used the set up and Arduino sketch I had created for giving input from the thumbstick in form of a number of 0-8 to indicate a direction. I described this here: http://evertlagerberg.blogspot.se/2015/05/ardunion-to-unity-project-part-2.html

I rewrote the Unity C#-script I wrote when I tested with just one button, to act differently on 8 different states(+ 1 state for doing nothing) instead of just on/off that I had for the button. Each number indicates which direction vector is to be used in Unitys tranform.Translate()-function. When we move, we want to move along the x-axis and z-axis but be constant on the y-axis. So the vectors are:

I then added the script to the first person camera:


Arduino to Unity: thumbstick 8-directions from Evert Lagerberg on Vimeo.

Arduino to Unity Project Part 3: Components and soldering

I ordered some new components for my project:

2 x thumbstick joysticks
2 x break out board(for the joysticks)
1 x connecting headers

The thumbstick consist of two potentiometers, one for horizontal movement (X-axis) and one for Vertical(Y-axis). You can also press down on the thumbstick to press a button.

I soldered the thumbstick to the breakout board and then soldered on a 5-pin header to be able to connect cables to the break out board.

Then I connected the thumbstick to the arduino as described here: http://42bots.com/tutorials/arduino-joystick-module-example/

The same link also provided a example sketch to print out the values of the horisontal and vertical potentiometers. The sketch worked fine. I noticed that both potentiometers had a range of 1-1020. For example, pushing the thumbstick all the way forward showed 1020 on the X-axis and 510 on the Y-axis. I wanted to define 8 directions on the thumbstick input so I wrote a sketch that printed a number depending on within which range it was.

The result ended up as this:

Arduino to Unity: thumbstick to serial from Evert Lagerberg on Vimeo.