ARKit: How Apple is quietly reinventing the smartphone

ARKit is the new iOS11 feature that could potentially revolutionize the platform. As of today, anyone owning an iPhone 6S or later will be able to run augmented reality apps. In one fell swoop, Apple will create the largest AR platform.

What makes ARKit so powerful?

In short: ARKit takes a lot of work away from the app developers. With ARKit, Apple has released a programming interface that allows developers to insert digital objects into real-world images from the camera. ARKit assumes the task, in order to supplement the depth information of the two-dimensional image of the camera. This is necessary, for example, so that a digital figure appears to actually stand on the street and not just float over an image. This kind of spatial recognition is complex and requires additional sensors – much like Google Tango. ARKit on the other hand, does not.

In addition to spatial environment, ARKit also detects lighting conditions. This is indispensable for realistic, seamless integration of digital objects. ARKit recognizes horizontal surfaces like floors or tables, as well as feature spots (such as objects that stand out).

IMG 0700 ARKit in action / © AndroidPIT

ARKit makes all of this information accessible to developers. With the help of Unity or other 3D engines, an app can then insert objects, and thus supplement reality – that is, create an augmented reality world. In order to minimize latency, your iPhone or iPad must be equipped with an Apple A9 or A10 chip or later. In other words, you’ll need an iPhone 6S or newer, a current generation iPad, or an iPad Pro.

wwdc ar2 ARKit live demo on the WWDC stage. / © Apple, Screenshot by AndroidPIT

ARCore is Google’s solution

ARKit and ARCore are technically quite similar. In the run-up to the ARKit launch, Google was already working on a similar technology – drawing on their previous work on Tango. This rarely uses time-of-flight cameras (i.e. depth sensors) within an AR session. When using a Tango-enabled app, typically only motion and acceleration sensors are used to determine exact positions. Thus, Google was able to quickly adapt the Tango technologies into ARCore. How will things progress with Tango? ARCore is probably the heir and Tango may disappear.

google arcore taco ARCore could be an ARKit competitor. / © Google

That’s why ARKit for augmented reality is also important to Android: key toolkits will support both ARKit as well as ARCore, so the porting effort for the developers will probably be minimal. And the most successful ARKit apps will also be available in 2018 on Android with ARCore.

Great demos with ARKit

Because ARKit is part of the beta version of iOS11, there are already lots of demos that show what ARKit can do. The first major demo event was held more than a week ago in the US. There, media outlets like TechCrunch were able to examine complex AR apps. IKEA and TV channel AMC presented AR apps there, for example. Even well-known children’s book The Very Hungry Caterpillar has made the leap into AR. According to TechCrunch, many of the complex demos took just two or three months to create. This shows how fast content can be provided for ARKit.

Below are some trailer videos of the ARKit apps, both for augmented reality as well as for virtual reality: words are good, but videos are better.

The Walking Dead: Our World

Video: Apple / IKEA (via Techcrunch)

The Very Hungry Caterpillar

Augmented reality changes our smartphone usage

ARKit is impressive but the technology is still in its infancy. However, it’s already clear that augmented reality will become an important part of smartphone features in the coming years.

Practically all of the applications shown here have been developed for iOS11 only. Android users should not despair. With ARCore, they’ll soon get an alternative. Successful apps will find their way onto Android quickly, thanks to the technical similarities. If ARCore proves to be a stark competitor, Google will also be able to make tens of millions of smartphones AR-capable within a short time. We can expect to hear a lot about ARKit apps in the next few months, but in half a year, Android will also receive the most successful apps.

Opinion by Hans-Georg Kluge

Augmented reality will advance smartphone development over the next few years

What do you think?

4 participants

These major changes are coming to both platforms. The camera will become a tool for perceiving our environment in a new way. Google has brought together machine learning and AR with Google Lens. Some AR apps will be fun, others will help us stay oriented in foreign cities. Others will open up new horizons for smartphones.

What features would you like to see with AR? Looking forwad to it? Leave your thoughts in the comments below.

Android News + App Reviews + Hardware Reviews – AndroidPIT

ARCore: Google's answer to ARKit is coming to the Pixel and Galaxy S8

Thanks to Apple’s ARKit, augmented reality has been causing a stir for the past few weeks. Now Google has presented the world with a preview of ARCore, which means AR for Android devices without any further hardware prerequisites.

I would definitely fit into the AndroidPIT team!

What do you think?

7126 participants

ARKit was the big highlight of the recent Apple keynote where iOS 11 was presented. With ARKit, augmented reality apps become possible without any additional hardware requirements, apart from the smartphone (in this case, an iPhone). Google has long been working on the AR technology Project Tango, but Tango needs additional sensors in the smartphone called time-of-flight cameras. Currently, only the Lenovo Phab 2 Pro and the ZenFone AR have this. ARCore is completely software-based and does away with this requirement.

Google’s answer to ARKit is called ARCore

ARCore is based on Tango technology. ARCore allows developers to develop augmented reality apps that run on a number of Android smartphones. The platform focuses on three main characteristics.

google arcore taco Google ARCore puts virtual objects into real scenes. / © Google

First, tracking the movements of the smartphones with the camera and sensors allows virtual objects to remain in one place. Second, the platform can understand environments, allowing ARCore to recognize tables or floors. Third is light analysis, which allows developers to illuminate virtual objects according to the real environment.

On the one hand the tracking of movements of the smartphone with the help of the camera and sensors, whereby virtual objects always remain in one place. Secondly, the platform allows the environment to understand. This ensures that ARCore can recognize tables or levels. The third feature is a light analysis, which allows developers to correctly illuminate virtual objects.

It remains to be seen whether ARCore can match up to Apple’s ARKit. ARCore is the next step in the AR space, so the platform will surely get more features. Google is looking into releasing browsers which bring ARCore into the mobile web, so websites could make use of ARCore without having their own app installed on a user’s device.

google arcore lion ARCore works without additional sensors. / © Google

Google will first publish the SDK of ARCore as a preview, compatible with the Google Pixel and Galaxy S8 smartphones. Google will require at least Android 7.0 Nougat for compatibility going forward. Google is working with Samsung, Huawei, LG, Asus and others to expand support for ARCore. It’s impossible to say whether all Android smartphones with Android 7.0 and up will be supported, as Google wants to ensure a good standard of quality and performance. When the preview comes to a close, Google hopes to bring ARCore to 100 million smartphones. 

Opinion by Hans-Georg Kluge

With ARCore, Google has found the right answer to Apple’s ARKit.

What do you think?

Be the first to vote

It’s unknown how long ARCore will be available as a preview for developers. It’s already possible to see some experiments made with ARCore. What do you think? Tell us in the comments.

Android News + App Reviews + Hardware Reviews – AndroidPIT

Google Tango vs ARKit: Why Google lost the first round in the AR battle

Google has finally released a second Tango-compatible phone on the market: The ZenFone AR. The apps are fun to try out, but they also show limitations. And then there is that persistent competitor, Apple, which changes the game at the drop of a hat. What future does Tango have? How impressive is ARKit really? We investigated details about the ZenFone AR.

What exactly is Tango?

Tango is Google’s platform for augmented reality (AR) apps. Essentially, it’s about placing digital objects in the real world. These objects would then be visible on the smartphone display. 

Tango uses 3D motion tracking and depth perception to allow the smartphone to know where it is within a room, so as you move around, these digital objects follow the movement of the phone and are thus seamlessly integrated into the environment.

AndroidPIT zenfone ar 0656 Augmented reality is only seen through the smartphone. / © AndroidPIT

Augmented reality can only be seen through a smartphone; some scenarios are still pipe dreams, but they may soon be real: Using an app, you can leave notes for your friends in a restaurant, or the servers could show you the dishes on the menu as a 3D animation. Naturally, digital shopping guides would also be a possibility with a smartphone app. Amazon may be planning to open AR furniture stores, for instance.

Now, Tango has made its second appearance in a consumer smartphone. The ZenFone AR shows the platform’s potential. We tried out some apps and games that are only available for Tango phones.

BMW i Visualiser: BMW comes to your living room

In a very brief period of time, BMW has developed a Tango app that places a BMW in your driveway, your living room or anywhere you please. The use case is not immediately clear at first, but it is quickly becoming apparent why augmented reality apps are so interesting. Because, when looking through the smartphone display, I suddenly see a BMW in front of me: either an i3 or i8. Stefan Biermann, Director of Sales Innovation, spoke to us about the BMW app’s background. His department has been working on the app for roughly a year.

AndroidPIT zenfone ar 0687 A BMW in the AndroidPIT office / © AndroidPIT

But what’s the point? Biermann told us that he demonstrated a prototype of the app at a sales manager event. The added value of such a presentation became immediately apparent to all who were present. Finally, a dealer can use the i Visualiser to immediately show his customer different automobile color combinations. And he doesn’t even have to be at the dealership – the rendered car looks incredibly sleek, particularly with the ZenFone AR.

bmw visualiser How can the BMW get on the city train platform? / © Screenshot: ANDROIDPIT

Wouldn’t virtual reality be a better alternative? Thanks to crafty tracking, a car can be integrated into any desired scenario. Biermann put a positive spin on it: Customers on sales floors would not feel good putting on VR glasses. On the other hand, an AR application is a whole other ball game: The smartphone here solely serves as a window into this imaginary world and the user—preferably the buyer in this case—always has control and has a feel for his environment. So, if BMW were to rely on VR platforms like the Vive, the manufacturer would probably have to provide an enclosed room for it. This makes the AR app more frugal; it only requires a square meter of free space.

However, an enormous amount of effort is required for the necessary 3D models. Even though BMW has the data, it must nevertheless be converted for the AR app and be implemented in the app. It currently requires a lot of manual labor, because the augmented reality development is still in the preliminary stages. Even the size of the three-dimensional models presents a problem. If BMW were to place the entire automotive range in the visualizer, it would quickly amount to several gigabytes.

BMW i Visualiser Install on Google Play

The American Museum of Natural History resurrects dinosaurs

How can AR be used for educational purposes? The American Museum of Natural History has come up with an idea. Dinosaurs can be placed in the environment using the app. Unfortunately, the three-dimensional models are quite simple and not texturized, which reflects badly on the app. Nonetheless, the dinosaurs are animated a bit and move around somewhat. It’s a shame that the developers have not even managed to create a brief audio track with information on the ancient animals, let alone an interactive demo.

amnh 1 Do not fear, the dino just wants to play. / © Screenshot: ANDROIDPIT Dinosaurs Among Us Install on Google Play

Tango trails: Through the sugar jungle

Do games always need to have a sophisticated story or even make sense at all? They certainly do not. Candy Trails is one such game. You start by first scanning the room, allowing Candy Trails to create the playing space. Done. Now the game is asking me to go up to a lollipop. OK. I look for the huge lollipop, go up to it and tap on the display. And then to the next lollipop. Well, the game seems to be quite boring. With the third lollipop, it finally makes sense to me: Because now I have to avoid walking into obstacles that suddenly appear as candy canes. The goal is to reach as many lollipops as possible. Candy Trails is definitely nothing more than a time waster, but a very interesting demonstration of the new gaming concepts that are possible with AR.

tango trails Tango Trail’s sugar labyrinth. / © Screenshot: ANDROIDPIT Tango Trails Install on Google Play

Domino World: The Domino Effect

Creating large lines of domino pieces is not only time-consuming but also tedious: Every single domino piece requires your full concentration; otherwise, you them tip over and all your effort is worth diddly squat. Of course, that does not happen with an AR game. The domino game lets you create your own lines on the floor, use assorted color domino pieces, insert obstacles, or integrate special actions in the line. Unfortunately, there is no option for integrating real obstacles into the line.

domino world Dominos fall, but only virtually / © Screenshot: ANDROIDPIT Domino World Install on Google Play

The competition touts impressive stuff – using no sensors at all

As impressive as some Tango apps are: the competition is not slacking off. Two to three years ago, it seemed clear that AR Apps required hardware support à la Tango, but Facebook, and mainly Apple, have proven otherwise. For instance, the recently presented ARKit by Apple can precisely analyze the smartphone camera’s image without using any additional hardware sensors at all. At the keynote, Apple presented an impressive spectacle on the table, which was visible with an ordinary iPad.

And the past several days has shown one thing: ARKit delivers on its promises. Many demos show impressive precision. And iOS 11 gives all iPhones 6S and higher, iPad Pro tablets and the latest iPad (fifth generation) full ARKit capabilities.

On the other hand, Google has most recently promoted the Tango platform, above all. It is very respectable from a technical standpoint, and yet, it faces a disadvantage: Smartphones have very thin margins, and each component costs extra. Will 50 percent of all Android smartphones really be equipped with Tango sensors in two years, as some people are imagining? I am inclined to doubt that due to razor-thin margins in Android smartphones.

Opinion by Robert Dunn

Apple’s ARKit is impressive.

What do you think?

5 participants

In testing, Tango has shown that while it does do a decent job indeed, it doesn’t do a fantastic job. Yeah, Tango works precisely, but it actually can’t place the objects three-dimensionally in the environment. The augmented reality layer in all the apps that I tested remains an overlay over the camera image.

So, AR overlays reality where the AR content needs to be overlaid, but that doesn’t need to be excessively important. Because anyone who simply wants to hang little AR pictures on their wall doesn’t need to rack his or her brain about it. But what’s really the problem is the following: Other AR platforms can achieve this without additional AR sensors. For instance, take the app 1600, which creates a history lesson out of a dollar bill. All that without a single sensor.

1600 Install on Google Play

Google Tango: The platform has a problem

Augmented reality in general is a very exciting field. Not everything is perfect yet, there mainly needs to be new usability concepts, because the more complex apps are not comfortable for the most part.

Google Tango in particular got off to a bad start. Facebook’s demonstrations at F8 came very close to Tango technologies and, by all accounts, Apple’s ARKit took it one step further. The problem: Apple has made augmented reality possible in software; on the other hand, Google only has a hardware solution. But hardware cannot be upgraded.

AndroidPIT zenfone ar 0755 Google Tango vs. ARKit: Tango’s problem is the cameras it requires. / © AndroidPIT

Augmented reality apps on Android are facing dark times. ARKit will already have an immense installed base, which is crucial for app developers. Tango is only available for a very exclusive circle of users. The prevalence alone will pull the rug out from under Tango’s feet in a very real way. Tango would have to provide significant added value in order to hold its own against a software solution.

The extent to which Google will continue developing its Tango platform remains to be seen. If Android managed to keep up with regard to augmented reality, then Google must develop an alternative to ARKit. I’m willing to bet that they are already diligently working on it at Google’s headquarters.

AndroidPIT lenovo phab 2 pro tof camera 4479 Tango is at a disadvantage compared to ARKit / © AndroidPIT

Software is Tango’s only hope

In any case, both ARKit and Tango selectively use the same sensor: ARKit knows the smartphone’s position in the room thanks to the position sensor and accelerometer, although Tango uses this, too. Google must also find a way to perform depth perception with the smartphone camera alone. It will be exciting to see how elaborate that actually is. A developer revealed to us that most Tango apps measure the room only once using the time-of-flight camera, but afterwards make do with the smartphone’s position sensors. In other words: Even Tango apps mostly work in a mode that resembles that of ARKit.

If Google manages to transform Tango into a software solution, then Google may have a chance to succeed. For the time being, however, the competition from Apple may leave it in the dust.

Google wants to expand Android by adding an AR platform, which is incidentally unwelcome news for the entire Android platform: Over the next several years, AR apps will be among the highlights of new releases. As of now, Android users will barely benefit from it, and Google is in a tight spot.

Have you seen some AR demos? What do you think of the topic? Let me know in the comments below!

Where to buy Lenovo Phab2 Pro

Best price

Android News + App Reviews + Hardware Reviews – AndroidPIT