ARKit: why i am giddy about ios 11

iOS 11 is around the corner, and along with it comes the release of something I am very excited about: ARKit. By now you may or may not have heard of it. Regardless, it likely does not mean too much to you yet. It was announced officially at WWDC a bit more than a month ago, and now with a couple weeks to digest, several samples have come out on the web.

AR (augmented reality) in general is not something that is “brand new”, and has already been done on iOS itself actually. Although the Niantic team must be scratching their heads over whether to continue using their own AR engine or switch to ARKit, Pokémon GO is a prime example of how ARKit will function: the feature where pokémons show up in your living room is exactly what ARKit will enable every iOS developer to do.

Just over the past weekend, an ARKit demo took off by @AndrewProjDent on Twitter, earning over 7,000 likes and getting serious press coverage. In this demo, and it’s predecessor video, the author shows off some really cool uses of ARKit to display arrows and directions overlaid over the real world, making you never take a wrong turn on your way to the nearest Starbucks.

This is just the beginning.

Over the weekend another developer released a demo app that simply drops furniture into a room, a precursor to what IKEA and many other major furniture retailers will likely be building quite soon.

There were several other demos I saw, and I imagine many more that I have yet to see.

I also wrote some code in the aim of using ARKit in the past couple days myself. I currently have a problem where my wireless network does not reach my bathroom very well, and it’s been nagging at me for a while now. When seeing the ARKit demos, I immediately had an idea: I could make an app that showed a heat map of WiFi signal strength. By leveraging ARKit for this, I would generate a virtual representation of data transfer rate. A stretch goal for this app would be to allow for routers to be moved within the AR environment to see the resulting changes in the heat map.

The hard part of such an idea is actually not ARKit, at least not yet. So far the bandwidth testing code has been trickier than expected, and I was unable to find a suitable open source library, but the fact that THAT is the hard part and I’m not worried about generating heat maps with ARKit is what excites me: Apple has made it blatantly simple to add augmented reality components to any application.

"The basic requirement for any AR experience—and the defining feature of ARKit— is the ability to create and track a correspondence between the real-world space the user inhabits and a virtual space where you can model visual content. When your app displays that content together with a live camera image, the user experiences augmented reality: the illusion that your virtual content is part of the real world."

• https://developer.apple.com/documentation/arkit/understanding_augmented_reality

Apple has built a framework on top of a simple concept: visual-inertial odometry. By combining iOS device motion sensors, such as the gyroscope and accelerometer, with the high resolution image from the device camera, ARKit is able to track the differences in the position of features around the room/space and the result is a “high-precision model of the device’s position and motion” within the 3D space.

World tracking, as Apple puts it, even allows for plane detection, allowing me as the developer to access which planes in the current scene are for example big enough for my friend to hang out on:

So why does this matter? Well, it matters because giving every iOS developer the power to plop items into the space surrounding the iPhone changes the game. For as long as I can remember and been working, one of the things I’ve spent the most time on is fitting as much quality info as possible into a small smartphone screen. If the screen real estate is now the space around the device too, though, I start thinking of the Marvel movies. Does ARKit alone let me build Jarvis the way it’s used by Robert Downey Jr. in Iron Man? No way, well not yet. It does provide technology, though, that will make us think and design software in ways closer to that than we have ever done before. That is truly exciting to me.

TL;DR: ARKit is really cool, read to find out why I think so :)

What about ARKit excites YOU most? Let me know in the comments!