The YOUAR Cloud SDK
Enabling Dynamic, Game-Changing Features
George BlikasFollow
Nov 26, 2019 · 7 min read
Part 2 of 3 on our introduction to the YOUAR Cloud SDK
Last time we covered what the YOUAR Cloud SDK enables. Today, we will cover why it is important for developers and the AR Cloud.
A teapot pushing the boundaries of innovation in computer graphics. Photo posted by Jessie Dunietz, cutesy of the Association of Computer Machinery.
This teapot has a long history. It has set the comparative standard for computer graphics, appeared in several Pixar films, and it has been said, “anyone with a new idea about rendering and lighting would announce it by first trying it out on a teapot”.
This shape provided an important base case for developers as it was mathematically simple, being smooth surfaces, but was complex enough for the research at the time because it had a hole between the body and handle and could cast shadows on itself. Much like this teapot, the YOUAR Cloud SDK provides new and essential features for AR re-localization SDKs.
ARCore, ARKit and Azure Spatial Anchors
The de facto industry standards, Google’s ARCore and Apple’s ARKit came to market in 2018 and 2017, respectively, and made high-end AR consumer experiences widely available. Between the two of them, Google and Apple arguably taught the world the term “Augmented Reality”. Recently, Microsoft joined the fray with their Azure Spatial Anchor platform, an amalgamation of Universal Windows Platform and cloud services. The feature lists they each provide for AR development have been growing and competing, neck-and-neck. Their most recent endeavors provide image tracking and human pose occlusion.
Although Google, Apple and Microsoft have the big money, big teams, and push useful features for AR developers, they leave essential elements to the wayside: a method and storage system in which people can browse and create an endless repository of events, utilities, and experiences in an accessible universal framework. Much like the internet builds upon HTML, CSS, and JS constructs with browsers, the AR Cloud builds upon 3D feature maps, or “Anchors” of physical locations with augmented reality SDKs.
Now, while Microsoft’s Azure Spatial Anchor, Google’s ARCore Cloud Anchor API, and Apple’s World Map provide ways to create Anchors and to store them, they do not allow the operability necessary for a viable network of developers, nor an AR Cloud. This is due to two main factors:
Users must stay entirely within a provider’s ecosystem to store anchors and verify locations.
Providers control how long anchors persist.
Anchor Storage and Verification of Location
Requiring end-users to store and verify anchors on a single ecosystem involves a separation of data, almost by necessity. This is akin to how Microsoft, Google and Apple all maintain different, broken attempts to maintain interoperability methods for the others’ APIs and services. The trio of Keynote, Google Slides and PowerPoint is a good example of this. Usually you can open a slideshow in any of the three programs, but it is never optimal: Data is missing, the device you present on requires a specific Google, Apple, or Microsoft account, or additional proprietary hardware, and certain features simply quit working. You must have a “verified account” to operate efficiently, with all the headaches that entails.
Anchor Lifetime
That the big players decide how long anchor data exists is even worse. Market position dictates what services are available and supported from AR SDK providers. Who would invest in creating a project that could suddenly get deleted, because of an arbitrarily enforced 24-hour lifespan, or company redirection? Carrying this paradigm through to the augmented reality industry would cause AR Cloud users to fight against a constant mudslide, with their work and experiences pulling away from them.
The YOUAR Cloud SDK, Developers, and Users
Photo by xkcd
At YOUAR, much of our thought process comes from the open source community. One of our cornerstones is to ensure developers only deal with walled gardens in ways that facilitate and promote their applications. We want to support the use of high-precision SDKs like ARCore, ARKit, and Azure Spatial Anchors, but with coding and user freedom at heart. A core tenet of our SDK is to allow 100% interchangeability, regardless of the host’s native computer vision.
From slideshows, to web infrastructure policies, to the Khronos Group, the reason to support a central standard (or API) for developers is clear; without one, many applications would have never made it to market, as any collaboration would be overly difficult. YOUAR achieves this for AR Cloud developers through 4 main features:
Low-Level Anchor Access
Multi-Anchor Networks
Persistence Framework
Cross-Platform Support
YOUAR and Low-Level, Multi-Anchor Networks
Anchors are the fundamental driving force of all AR interactivity. An API which allows contribution and easy use of them provides the AR landscape with the tool necessary to create an AR Cloud. Lack of such functionality leaves users with limited accessibility.
The early-access for Minecraft Earth, which uses Azure Spatial Anchors as its back-end framework for AR, is a great example of AR implemented in a limited framework. Ultimately owned by Microsoft, Minecraft Earth has probably used almost every function available via Azure Spatial Anchors. However, the game is still limited to the “one-way” functionality of static uploads and downloads supported by Azure Spatial Anchors.
While Minecraft Earth’s ToS might imply something more intelligent is going on behind the scenes with Spatial Anchors (some “meta-data” analysis), like many other AR SDKs, they fall short of providing the actual functionality necessary. AR developers and users don’t have answers to how they should retrieve data, nor how to use it effectively, and concerns of accuracy in re-localization and persistence remain unaddressed.
ToS from Minecraft Earth, indicating use of Azure Spatial Anchors. Source: Azure ToS.
Multi-Anchor Networks and Continuous Persistence
A critical area where common AR frameworks fall short is accuracy and re-localization in object persistence across a large, shifting environment. The area’s size, changes in environmental conditions, and other moving objects in the scene are necessary for developers to account for when designing a persistent AR experience. Minecraft Earth gets around this by providing a temporary solve. Minecraft Earth creates localized experiences; the current activity anchors to the real world, yet that information gets lost between runtimes. There is no continuity in the Anchor’s persistence. This style of persistence is a common development scheme used by most AR applications
— Pokemon Go’s AR mode is another example.
The reason many AR applications adopt this UX pattern is because the SDKs do not focus on the underlying infrastructure of an AR Cloud. Without an AR Cloud, most every AR experience will fail to this “temporary persistence”. We solve this by providing an adaptive Multi-Anchor Network system that works automatically in the background, changing and repairing itself to allow users and developers to have “continuous persistent” AR experiences.
The video below shows how our back-end system morphs to address a changing environment, allowing for uninterrupted AR. Each color represents a different map of physical space, and the collection is what we call an Anchor. Each description of the physical space contains a corresponding colored 3D box that shows the origin of creation. Using an adaptive system for these anchors means you can see that each box stays properly localized in 3D space. This foundational technology allows persistences to exist from runtime to runtime, regardless of shifts in the environment.
Video by YOUAR.
Low-Level Access
A serious shortfall of mainstream AR Anchor systems is the method in which users access and contribute to the AR Cloud owned by individual companies. For instance, Minecraft Earth has adopted the standard model of user-selected database entries to populate and use. While the ToS says that Azure collects additional “metadata”, there is no low-level access to the meta-characteristics of the spatial data, meaning that the only way to serve AR Anchors is through ID look-up.
The dynamic nature of AR means there is no perfect comparison, however, one can see how the real-life requirements of traversing the physical world require more granular APIs into the data being served and stored. Users might want new Anchors served to them based on time of day, or lighting conditions, network (public vs. private), etc.. Juxtaposed to this is the YOUAR Cloud SDK and it’s low-level access, and metadata access. With a granular level of control, users can access scans based on real-world features like WiFi, GPS, average luminosity, and compass settings (C#):
//
// define the container we will use for our meta request AnchorMetaRequest meta = AnchorMetaRequest();//
// request an anchor by network properties. This is
// great for large development teams, or security
meta.networkInfo = new AnchorMetaRequest.NetworkInfo(){
ssids,
bssids,
strengths,
};//
// a value indicating the average lighting of the scan.
meta.lux = AnchorMetaRequest.RangeF() {
max = <value>,
min = <value>
};//
// a typical use-case in today's ar landscape.
meta.gpsInfo = AnchorMetaRequest.GpsInfo(){
latitude = <value>,
longitude = <value>,
altitude = <value>,
};//
// get anchors based on how easy it was for others to
// re-localize to.
meta.successRatio = <value>;//
// finally, load all possible anchors available
// checking agains the current meta request
LoadUnloadByMetaRequest(meta, allAvailableAnchors);
There are many more options than those listed above, this is just a taste of the important control you get access to with the YOUAR Cloud SDK.
Next time…
A viable AR SDK hinges on putting developers first, by giving them access and control to implement new and revolutionary ideas. Through important features like Low-Level Access, and Multi-Anchor Adaptive Networks, the YOUAR Cloud SDK puts important tools into the hands of these developers. With these tools, the YOUAR Cloud SDK becomes an important link-in-the-chain to solve fundamental problems in how to retrieve data and how to use it.
YOUAR
YOUAR Augmented Reality Tools
YOUAR
Following
