For my first semester I decided I wanted to gain first hand experience in the augmented reality industry. As a result I approached SPARK EMERGING TECHNOLOGY LAB who are the UK's leading augmented reality agency, and I have been a huge admirer of them over the last couple of years within the AR agency.
Brief
Being an intern in SPARK, my role was to undertake AR projects that the developers did not have time to build/ update. My third project in Spark was to create an Augmented Reality for the 3D minature model company, Funko Pops. My breif was to create an augmented reality application, which should contain a variety of models that the user can be able to select from, and manipulate in AR.
Funko Pop's is a toy based company with over 8,000 3D models are created based upon film, pop culture or significant individuals, source: How aggressively cute toys for adults became a $686 million business. These models are extremly popular in the collectible market, not only for fans but also rare and limted models are released as shown below with the rare Batman vinyl figure.
The application was intented to be a prototype for Funko as a potential AR application that they may utilise in the future for their own site. Some of the features they wanted were:
Abilty to select a range of models from the AR application
Functional usage such as rotation, scaling and true size
Abilty for more Funko Pops to be added
As Funko is a very important and large brand, it was imperative to make sure the key basics of the AR application was correctly built.
Next I started to draw out the wire frame of how the end result of UI on the application would look like.
Next I started to draw out the wire frame of how the end result of UI on the application would look like.
The Solution ✅ Development- Week Seven 🗓️
To begin with, the first challenge was to scan the Funko Pop called 'Neard Rage' and convert the model into a 3-D asset we could implement and import into unity. It was imperative that the model would be scanned into FBX file (Filmbox) to then be directly imported into Unity. .dae (Collada)
.dxf and .obj. file types are not part of the importchain that fbx is File formats when importing into unity. As a result, I started by testing three types of 3-D affect scanners:
The first scanner application that I decided to test was the Polycam LiDAR (Light Detection and Ranging) & 3D Scanner. This application despite using my iPhone Twelves advanced leader radar, did not produce a very accurate 3-D model of the nerd Funko pop. As you can see it is not able to distinguish the main face and a body at all despite putting the range at the lowest point.
I therefore decided that this application was not suitable for creating and scanning a model, and I looked at other alternative 3-D scanning applications.
Next I looked into a second LIDAR 3-D scanner application, 3D scanner from the appstore, that specialises in scanning smaller objects such as shoes and padlocks to make into 3-D models, that can be used for AR or VR.
On the left you can see the lights are trying to get a rough representation of the Funko pop 3-D model, with trhe purple represeting the LiDAR attempting to get detail from the model.
However upon rendering and previewing the texture, the model did not show any quality at all as can be seen on the right. I therefore decided to use photogrammetry to scan the model.
I came across an application called Meta Scan, and began to take photos of the model. It was imperative that I took multiple photos around model to get the highest quality. My first test as shown on the right, consisted of 23 photos which was not near enough to create a high quality 3D model, so I retook the photo consisting of over 150+ photos of all angles of the model.
After completing the final scan, the GIF on the left showcases a very accurate representation of the nerd Funko pop in 3-D.
I then exported the model to an FBX format and sent it across to unity.
I then scanned the Starwars Poe funko pop and the Nerd Rage Box and took over 120 photos from all angles of the model. This ensured the highest quality is attained.
Intergration to Unity
Next I imported the model into unity, and created a new unity build where I imported AR foundation to be the Augmented Reality framework of the application. Upon importing the model, I also decided to utilise URP (Universal Render Pipeline) as the chosen shader. However this led to the models going pink, but this was quickly corrected upon upgrading all of the models textures.
The two models imported into Unity.
Next I implemented the ARPlaceAtPoint script that Spark assisted me in writing with the Troll and Dragon project. However we altered the script, to allow for the transformation of each 3D asset inside the parent object. This is outlined in the next section below in augmentation tests.
I then added the Raycast manager and divided the screen by two using the vector 2 screen width and height. This enables the user to detect a flat surface for the AR to Augment from.
Next I attached the script to the button to place the Nerd rage. The function is within a script which then enables to place the nerd rage, which is derived from the ARPlacePointScript.
Next I implemented a scaling script which I located online from Youtube Pinch to Zoom. The script enables the 3-D Funko pops to be scaled up and down, depending on the users preference.This is a very important script to be able to utilise in an augmented reality project like this due to the added interactivity bonus that it shows. The Input.touchCount == 2 is in reference to the two fingers of the user scaling the object up and down.
Week 8 & 9- Adding rotation and augmentation tests
Next I implemented the rotaion script, enabling the assets to rotate from finger input. This script is also used on the troll and dragon project from Stack Overflow StackOverflow rotation script
Next i implemented basic UI buttons to enable the Nerd Rage and Nerd Rage Box to be placed.
Here is the hierarchy view of how the funko pops are structured inside unity.
Next I built out the application after importing the buttons into unity. Unfortunately upon testing there was an issue when the buttons were not placing the objects when placed.
This was due to them being placed in a parent object which wasn’t declared in the code. As we needed the parent to contain all of the assets for scaling, we needed them to be placed independently.
After altering the code as explanied below, the models were unable to be deployed appropriately as shown on the right.
We altered the code to take on all of the assets inside the parent: ‘foreach (Transform childTransform in parentObject)’.
if (childTransform.name != "Plane")
{
childTransform.gameObject.SetActive(false);
This part of the script means that the plane won’t be turned on every time the code runs as it is the main plane for the script.
Development of UI
Next I improved the UI and integrated a reset button and script as well to reset the scene for repositioning the Funko Pop models.
Next I changed the UI to include scroll view to allow for multiple characters to be added on the scroll rect. I set anchor positioning for all of the UI elements, to fix the postitioning of the UI, when the application is running.
Next I added more of the characters to flesh out the application, as well as integrating the scroll view into the hierarchy, and a scroll bar horizontal as well underneath the icon buttons. I also moved the reset button to the top right of the screen, as well placing the box view of the nerd rage icon to the top left.
I then began linking the icon buttons to an on click function within the button, where the AR plays point script would place the Freddy Funko 3-D object into the scene as shown on the left. Above I've been tested out each button and added each asset to the script to it's relative icon buttons, where it successfully showcased the application working.
Week 10- Feature Development- Box On/Off, Smoothdamping, Character tagging, Drop down menu intergration
As shown above, when each object is selected the nerd rage box on the top left does not disappear. However we decided that we wanted the nerd rage box to only appear when the nerd rage character was selected and no other character.
Therefore Buttonopen was created, to only appear when the Nerd rage is selected. IconOpener != null) showcases that it will return true if the funko pop is selected. if it is not/ are not equal, it will be false otherwise
Box off is the opposite, where the box is not present and turns off when the other icons are pressed.
Here I attached OffBox to Poe, as an Onclick function to the model.
I did the same to the LootLlama where I attach the box off.off box scripts to the on CLick button. I made sure to untick the show nerd rage box, to make sure the box does not show when the Lootllama icon is pressed.
Next I added these smooth damp script to enable the scaling of the models to be in a smoother fashion and make the application look more professional.
Here the modesls are scalling far smoother with the smooth damp applied.
Next we started coding the script for drop down menu, allowing each individual Funko Pop to be categorised based on where they came from.
In this case the we created boolean stament that checked to see if the isNameTheSame = content is equal to thefunkoItem.tag == (which compares thefunkoItem.tag's refrence to the name). If so, the gameobject is set as active.
I wrote an if else statment, that would set the tag as true or false depending if the items tag was equal to the name.
After writing the code I then started to create the drop-down categories for the Funko pops.
I then started to tag the icons based upon their origin, such as influential figures.
Despite writing the drop-down menu categories, however upon reassessment we discovered there was a more efficient option. Instead of writing in each target tag, we changed it to an int stament with a varible of selectionId, within a void of OnMenuChanged. Each string name is equal to a menu option text, of it's own selection Id. For each of the funko pop models, in the container, the models tag I assigned earlier, is equal to the name on the drop down. Therefore sorting the models based upon the menu options.
I then played the scene within unity to test out the scroll bar, and to see the order of the drop-down menu.
Week 11- Filtering demo and Reset scene
I then tested out the menu system and category system within Unity, where we found out that every model responded to it's own category.
In order to make sure the Funko pop's categories would reset upon the pressing of the reset button, I added a reset scene that would access the senior manager in order to get hold of the active scene. This would then rebuild the index upon pressing the reset button as shown above in the ResetScene code.
Vibrations and Haptic Feedback
I decided to add a 4D element to the application, upon pressing on one of the icons, the user would receive a vibration upon pressing the chosen item. This creates a more real feel feedback when selecting their chosen funko Pop. I discovered a site called Nice Vibrations and utilised their vibrations to be part of the project. The float varibles 0.9 and 0.6 showcase the haptic intensity and sharpness of the pressing of the icons.
Next I highlighted all of the buttons and dragged the Button Buzz gameobject across the screen into the OnClick function. I then selected the function, to be Press Button Vibration.
The next developement of the project was to intergrate haptic feedback into the models when they are scaling. As shown on the right, Metascan had their own AR version of the models when scaling up and down and would pause the size when it got to 100% which is it's real size. From there the user could scale up and down, however when it reached its original model size it would temporarily lock and then get larger if the user persisted to alter the size.
I began by entering the transform local scale within an if statment, where the two float varibles are create, with float varibles greater than (>=) 0.95f and less than (<=) 1.04f. Inside the float varibles are the instructions for the spawned object to keep to the vector 3 scale of 1,1,1 original size of the model.
This then links to the Haptic Feedback script with an if statment that takes an the bool operator !, whcih will fire a haptic feedback once if the 1,1,1, has been reached resulting in true.
Here I'm testing the scaling of the model. The model sucessfully scaled up to 1,1,1 and locked in place; however it was not possible to scale the model down below 1,1,1. This was a problem I had to address.
Each model has three scripts attached to the On Click button.
To address the model not scaling below the required float varible, I reduced the bracket for the two float varibles to >=0.97f and <=1.02f. This allows the model to then scale below and then lock to the 1,1,1 scale between the two float varibles.
Here is the model now scaling up and down, where it sucesfully pauses between the value of >=0.97f and <=1.02f to 1,1,1 and once the values had been overridden, the model then scaled lower or higher.
Week 12- Final Outcome
Funko Pop’s feedback-
Spark then later pitched the Funko pops augmented reality project to the company. The feedback was that they really liked it and were very impressed with the ability to interact with the models in augmented reality; however they did not see an immediate need to deploy or release the application, which is often the case when a company has never used AR before. They would also need more models to be added in the future with a range to up to 60 models.
Reflection and analysis
To reflect on this project, I believe I have developed many skills during my development time. Developing the augmented reality Funko pop AR application, has been by far my favourite and most enjoyable augmented reality project to undertake. My skills have been greatly enhanced as I have dealt with many Csharp scripts during my time on the project as well as many assets being integrated into one AR application. Also the ability to add 4D elements to the application such as the haptic feedback and vibrations, really developed my knowledge and develop experience in what an augmented reality can produce and how interactive it can be. Some of the challenges I faced were down to the writing of the scripts, and having to learn each individual part of what each script meant which was challenging due to my C sharp knowledge being very limited at the start of the project. However learning from the developers at Spark my interpretation and ability to code has dramatically improved whilst working on the project as well as the online resources. Overall I have really enjoyed working on this project and it will be a highlight of mine throughout my augmented reality developer career.
Time management
I structured my time management on a week by week basis, where I made sure that I took each stage into account. The Troll and Dragon project only consisted of three weeks, with a Lamborghini project consisting of four. This was down to the amount of development work required to build out the application. The Funko pops however stretched between week seven and week twelve, as they were far more in-depth and required many more C sharp scripts and development work to occur to them. Overall I was happy in the way I managed my workload and found it to be a success.
Upon handing in the augmented reality application, the relevant files were all stored on Spark labs computers. As Spark own the projects and the files are on their computers/ databases, they do not need handover documentation.
