You’ll be able to build structures and worlds on your table with friends… [credit: Microsoft
Microsoft may believe it has made augmented reality’s killer app: the just-announced Minecraft Earth for iOS and Android.
AR on mobile devices may carry tremendous potential, but it’s easy enough to argue that the mainstream value proposition hasn’t arrived yet. Pokémon Go is probably the most oft-cited “killer app” for AR, but it’s only barely a true AR app. And there are some neat shopping apps and educational tools (from Warby Parker and Ikea, for example) but none of them have made a big dent in the mainstream consciousness.
At first glance, Minecraft Earth seems a bit like Pokémon Go, given that it seems to be location aware in some ways. But there’s a bit more to it than that. Players will be able to construct builds on their living room tables either alone or in collaboration with others, then go and place them full-size in the outside world when they’re ready. You can collect new mobs (both familiar and new) and resources around you to incorporate in your build, then fight them in the life-size version of the build. Fundamentally, it appears to be the basic Minecraft experience translated to augmented reality with geolocation features.
The May 2019 update for the Xbox One’s system software is now rolling out, bringing some small refinements to the friends list, messaging, and game/app list.
Starting with the last one first, the app list will now ignore “a,” “an,” and “the” when sorting or grouping alphabetically. This is the kind of change that makes me amazed that they weren’t already doing this, as it almost always makes for easier-to-use listings. Video games don’t even have The The to contend with.
The Messaging change is rather inexplicable. There’s a sensible change: incoming messaging requests from your friends are now prioritized, with requests from non-friends put in a separate category. But for some reason, Microsoft is going to wipe all group messages as a result. You can save backups of the messages for a limited time at Xbox.com, and messages with individual users are safe, but the group messages are all going. There’s no obvious justification for this change, as even if there were some significant change being made to group messaging, one would expect Microsoft to handle migrating the messages from old to new.
Microsoft is investing in certification and training for a range of AI-related skills in partnership with education provider General Assembly, the companies announced this morning. The goal is to train some 15,000 people by 2022 in order to increase the pool of AI talent around the world. The training will focus on AI, machine learning, data science, cloud and data engineering and more.
In the new program’s first year, Microsoft will focus on training 2,000 workers to transition to an AI and machine learning role. And over the full three years, it will train an additional 13,000 workers with AI-related skills.
As part of this effort, Microsoft is joining General Assembly’s new AI Standards Board, along with other companies. Over the next six months, the Board will help to define AI skills standards, develop assessments, design a career framework and create credentials for AI skills.
The training developed will also focus on filling the AI jobs currently available where Microsoft technologies are involved. As Microsoft notes, many workers today are not skilled enough for roles involving the use of Azure in aerospace, manufacturing and elsewhere. The training, it says, will focus on serving the needs of its customers who are looking to employ AI talent.
This will also include the creation of an AI Talent Network that will source candidates for long-term employment as well as contract work. General Assembly will assist with this effort by connecting its 22 campuses and the broader Adecco ecosystem to this jobs pipeline. (GA sold to staffing firm Adecco last year for $413 million.)
Microsoft cited the potential for AI’s impact on job creation as a reason behind the program, noting that up to 133 million new roles may be created by 2022 as a result of the new technologies. Of course, it’s also very much about making sure its own software and cloud customers can find people who are capable of working with its products, like Azure.
“As a technology company committed to driving innovation, we have a responsibility to help workers access the AI training they need to ensure they thrive in the workplace of today and tomorrow,” said Jean-Philippe Courtois, executive vice president and president of Global Sales, Marketing and Operations at Microsoft, in a statement. “We are thrilled to combine our industry and technical expertise with General Assembly to help close the skills gap and ensure businesses can maximize their potential in our AI-driven economy.”
When your game tops a hundred million players, your thoughts naturally turn to doubling that number. That’s the case with the creators, or rather stewards, of Minecraft at Microsoft, where the game has become a product category unto itself. And now it is making its biggest leap yet — to a real-world augmented reality game in the vein of Pokemon GO, called Minecraft Earth.
Announced today but not playable until summer (on iOS and Android) or later, MCE (as I’ll call it) is full-on Minecraft, reimagined to be mobile and AR-first. So what is it? As executive producer Jesse Merriam put it succinctly: “Everywhere you go, you see Minecraft. And everywhere you go, you can play Minecraft.”
Yes, yes — but what is it? Less succinctly put, MCE is like other real-world based AR games in that it lets you travel around a virtual version of your area, collecting items and participating in mini-games. Where it’s unlike other such games is that it’s built on top of Minecraft: Bedrock Edition, meaning it’s not some offshoot or mobile cash-in; this is straight-up Minecraft, with all the blocks, monsters, and redstone switches you desire, but in AR format. You collect stuff so you can build with it and share your tiny, blocky worlds with friends.
That introduces some fun opportunities and a few non-trivial limitations. Let’s run down what MCE looks like — verbally, at least, since Microsoft is being exceedingly stingy with real in-game assets.
There’s a map, of course
Because it’s Minecraft Earth, you’ll inhabit a special Minecraftified version of the real world, just as Pokemon GO and Harry Potter: Wizards Unite put a layer atop existing streets and landmarks.
The look is blocky to be sure but not so far off the normal look that you won’t recognize it. It uses OpenStreetMaps data, including annotated and inferred information about districts, private property, safe and unsafe places, and so on — which will be important later.
The fantasy map is filled with things to tap on, unsurprisingly called tappables. These can be a number of things: resources in the form of treasure chests, mobs, and adventures.
Chests are filled with blocks, naturally, adding to your reserves of cobblestone, brick, and so on, all the different varieties appearing with appropriate rarity.
Mobs are animals like those you might normally run across in the Minecraft wilderness: pigs, chickens, squid, and so on. You snag them like items, and they too have rarities, and not just cosmetic ones. The team highlighted a favorite of theirs, the muddy pig, which when placed down will stop at nothing to get to mud and never wants to leave, or a cave chicken that lays mushrooms instead of eggs. Yes, you can breed them.
Last are adventures, which are tiny AR instances that let you collect a resource, fight some monsters, and so on. For example you might find a crack in the ground that, when mined, vomits forth a volume of lava you’ll have to get away from, and then inside the resulting cave are some skeletons guarding a treasure chest. The team said they’re designing a huge number of these encounters.
Importantly, all these things, chests, mobs, and encounters, are shared between friends. If I see a chest, you see a chest — and the chest will have the same items. And in an AR encounter, all nearby players are brought in, and can contribute and collect the reward in shared fashion.
And it’s in these AR experiences and the “build plates” you’re doing it all for that the game really shines.
The AR part
“If you want to play Minecraft Earth without AR, you have to turn it off,” said Torfi Olafsson, the game’s director. This is not AR-optional, as with Niantic’s games. This is AR-native, and for good and ill the only way you can really play is by using your phone as a window into another world. Fortunately it works really well.
First, though, let me explain the whole build plate thing. You may have been wondering how these collectibles and mini-games amount to Minecraft. They don’t — they’re just the raw materials for it.
Whenever you feel like it, you can bring out what the team calls a build plate, which is a special item, a flat square that you virtually put down somewhere in the real world — on a surface like the table or floor, for instance — and it transforms into a small, but totally functional, Minecraft world.
In this little world you can build whatever you want, or dig into the ground, build an inverted palace for your cave chickens or create a paradise for your mud-loving pigs — whatever you want. Like Minecraft itself, each build plate is completely open-ended. Well, perhaps that’s the wrong phrase — they’re actually quite closely bounded, since the world only exists out to the edge of the plate. But they’re certainly yours to play with however you want.
Notably all the usual Minecraft rules are present — this isn’t Minecraft Lite, just a small game world. Water and lava flow how they should, blocks have all the qualities they should, and mobs all act as they normally would.
The magic part comes when you find that you can instantly convert your build plate from miniature to life-size. Now the castle you’ve been building on the table is three stories tall in the park. Your pigs regard you silently as you walk through the halls and admire the care and attention to detail with which you no doubt assembled them. It really is a trip.
It doesn’t really look like this but you get the idea.
In the demo, I played with a few other members of the press, we got to experience a couple build plates and adventures at life-size (technically actually 3/4 life size — the 1 block to 1 meter scale turned out to be a little daunting in testing). It was absolute chaos, really, everyone placing blocks and destroying them and flooding the area and putting down chickens. But it totally worked.
The system uses Microsoft’s new Azure Spatial Anchor system, which quickly and continuously fixed our locations in virtual space. It updated remarkably quickly, with no lag, showing the location and orientation of the other players in real time. Meanwhile the game world itself was rock-solid in space, smooth to enter and explore, and rarely bugging out (and that only in understandable circumstances). That’s great news considering how heavily the game leans on the multiplayer experience.
The team said they’d tested up to 10 players at once in an AR instance, and while there’s technically no limit, there’s sort of a physical limit in how many people can fit in the small space allocated to an adventure or around a tabletop. Don’t expect any giant 64-player raids, but do expect to take down hordes of spiders with three or four friends.
Pick(ax)ing their battles
In choosing to make the game the way they’ve made it, the team naturally created certain limitations and risks. You Wouldn’t want, for example, an adventure icon to pop up in the middle of the highway.
For exactly that reason the team spent a lot of work making the map metadata extremely robust. Adventures won’t spawn in areas like private residences or yards, though of course simple collectibles might. But because you’re able to reach things up to 70 meters away, it’s unlikely you’ll have to knock on someone’s door and say there’s a cave chicken in their pool and you’d like to touch it, please.
Furthermore adventures will not spawn in areas like streets or difficult to reach areas. The team said they worked very hard making it possible for the engine to recognize places that are not only publicly accessible, but safe and easy to access. Think sidewalks and parks.
Another limitation is that, as an AR game, you move around the real world. But in Minecraft verticality is an important part of the gameplay. Unfortunately the simple truth is that in the real world you can’t climb virtual stairs or descend into a virtual cave. You as a player exist on a 2D plane, and can interact with but not visit places above and below that plane. (An exception of course is on a build plate, where in miniature you can fly around it freely by moving your phone).
That’s a shame for people who can’t move around easily, though you can pick up and rotate the build plate to access different sides. Weapons and tools also have infinite range, eliminating a potential barrier to fun and accessibility.
What will keep people playing?
In Pokemon GO, there’s the drive to catch ’em all. In Wizards Unite, you’ll want to advance the story and your skills. What’s the draw with Minecraft Earth? Well, what’s the draw in Minecraft? You can build stuff. And now you can build stuff in AR on your phone.
The game isn’t narrative-driven, and although there is some (unspecified) character progression, for the most part the focus is on just having fun doing and making stuff in Minecraft. Like a set of LEGO blocks, a build plate and your persistent inventory simply make for a lively sandbox.
Admittedly that doesn’t sound like it carries the same addictive draw of Pokemon, but the truth is Minecraft kind of breaks the rules like that. Millions of people play this game all the time just to make stuff and show that stuff to other people. Although you’ll be limited in how you can share to start, there will surely be ways to explore popular builds in the future.
And how will it make money? The team basically punted on that question — they’re fortunately in a position where they don’t have to worry about that yet. Minecraft is one of the biggest games of all time and a big money-maker — it’s probably worth the cost just to keep people engaged with the world and community.
MCE seems to me like a delightful thing but one that must be appreciated on its own merits. A lack of screenshots and gameplay video isn’t doing a lot to help you here, I admit. Trust me when I say it looks great, plays well, and seems fundamentally like a good time for all ages.
A few other stray facts I picked up:
Regions will roll out gradually but it will be available in all the same languages as Vanilla at launch
Yes, there will be skins (and they’ll carry over from your existing account)
There will be different sizes and types of build plates
There’s crafting, but no 3×3 crafting grid (?!)
You can report griefers and so on, but the way the game is structured it should be an issue
The AR engine creates and uses a point cloud but doesn’t like take pictures of your bedroom
Content is added to the map dynamically, and there will be hot spots but emptier areas will fill up if you’re there
It leverages AR Core and AR Kit, naturally
The Hololens version of Minecraft we saw a while back is a predecessor “more spiritually than technically”
Adventures that could be scary to kids have a special sign
“Friends” can steal blocks from your build plate if you’re playing together (or donate them)
The news for mspaint.exe aficionados is just getting better and better. Microsoft’s original plan was to deprecate Paint and end its development. It would still be installable from the Store but would no longer be included with Windows or receive any updates.
Last month, the company relented and said that the app would continue to be included in Windows. And now things have gone a step further: the program has been updated to include some surprising new features.
Paint has been updated to include keyboard support. More explicitly, Paint can now be controlled through the keyboard exclusively. The cursor can be moved with the cursor keys while the space bar is used to activate tools. There are keyboard bindings to control selections, switch between resize handles/control points, and generally do all the things that currently use the mouse.
In a rare move, console rivals Microsoft and Sony announced a major collaboration on Thursday to join forces on a potentially huge new gaming sector: the cloud. The companies announced today that they have entered into a “memorandum of understanding” to “explore joint development of future cloud solutions in Microsoft Azure to support their respective game and content-streaming services.”
The surprise move is the closest sign of collaboration between two fierce competitors in the console-gaming space, but it is probably not a sign that they will stop being competitors any time soon.
As part of the agreement, Sony will still use Microsoft’s Azure servers and data centers for its own game and content-streaming services. That presumably includes PlayStation Now—the Sony game-streaming service launched in 2014 after Sony’s 2012 acquisition of streaming company Gaikai—and PlayStation Vue, the company’s Internet-based cable TV alternative.
For the last two decades, Sony and Microsoft’s gaming divisions have been locked in all-out war against one another: on price, on hardware, on franchises, on exclusives… you name it. But it seems they’ve set their enmity aside temporarily that they might better prevent that filthy casual, Google, from joining the fray.
The official team-up, documented in a memorandum of understanding, was announced today, though details are few. But this is clear enough:
The two companies will explore joint development of future cloud solutions in Microsoft Azure to support their respective game and content-streaming services. In addition, the two companies will explore the use of current Microsoft Azure datacenter-based solutions for Sony’s game and content-streaming services.
Of course there is no doubt that Sony could have gone with a number of other cloud services for its gaming on demand services. It already runs one, Playstation Now, but the market is expected to expand over the next few years much like cord cutters have driven traditional TV and movie watchers to Netflix and other streaming services. Expansion would surely prove expensive and complicated.
The most salient challenger is likely Google and its new Stadia game straming service, which of course has a huge advantage in its global presence, brand recognition, and unique entry points: search and YouTube. The possibility of searching for a game and being able to play it literally five seconds later is an amazing one, and really only something Google can pull off right now.
That makes Google a threat. And Microsoft and Sony have enough threats already, what with the two of them making every exclusive and chip partnership count, the resurgence of Nintendo with the immensely popular Switch, and the complex new PC-and-mobile-focused gaming market making consoles look outdated. Apple Arcade exists, too, but I don’t know that anyone is worried about it, exactly.
Perhaps there was a call made on the special direct line each has to the other, where they just said “truce… until we reduce Google Stadia to rubble and salt the earth. Also Nvidia maybe.”
We don’t actually have to imagine, though. As Sony President and CEO Kenichiro Yoshida noted in the announcement: “For many years, Microsoft has been a key business partner for us, though of course the two companies have also been competing in some areas. I believe that our joint development of future cloud solutions will contribute greatly to the advancement of interactive content.”
Sony doesn’t lack technical chops, or the software necessary to pull off a streaming service — but it may simply make more sense to deploy via Microsoft’s Azure than bring its own distribution systems up to par. No doubt Microsoft is happy to welcome a customer as large as Sony to its stable, and any awkwardness from the two competing elsewhere is secondary to that. Google is a more existential competitor in many ways, so it makes sense that Microsoft would favor partnering with a partial rival against it.
Sony has long been in this boat itself. Its image sensors and camera technology can be found in phones and DSLRs that compete with its own products — but the revenue and feedback it has built up as a result have let it maintain its dominance.
Speaking of which, the two companies also plan to collaborate on imaging, combining Sony’s sensor tech with Microsoft’s AI work. This is bound to find its way to applications in robotics and autonomous vehicles, though competition is fierce there and neither company has a real branded presence. Perhaps they aim to change that… together.
Microsoft has selected seven lucky startups to receive grants from its AI for Accessibility program. The growing companies aim to empower people with disabilities to take part in tech and the internet economy, from improving job searches to predicting seizures.
Each of the seven companies receives professional-level Azure AI resources and support, cash to cover the cost of data collection and handling, and access to Microsoft’s experts in AI, project management, and accessibility.
Companies apply online and a team of accessibility and market experts at Microsoft evaluate them on their potential impact, data policies, feasibility, and so on. The five-year, $25 million program started in 2018, and evaluation is a rolling process with grants coming out multiple times per year. This one happens to be on Global Accessibility Awareness Day. So be aware!
Among this round’s grantees is Our Ability, a company started by John Robinson, who was born without complete limbs, and all his life has faced serious challenges getting and keeping a job. The unemployment rate for people with disabilities is twice that of people without, and some disabilities nearly preclude full-time employment altogether.
Yet there are still opportunities for such people, who are just as likely to have a head for project management or a knack for coding as anyone else — but they can be difficult to find. Robinson is working on a site that connects companies with jobs suited to disabled applicants to likely candidates.
“Our goal is to empower employers to understand and leverage the increasingly valuable employment population of people with disabilities, proven to lower job turnover rates and boost morale and productivity – because a commitment to an inclusive workplace culture begins within,” Robinson wrote in an email to TechCrunch. “Employers have previously been at a disadvantage to accelerate in this regard, because many job-seeking tools are not designed with people with disabilities in mind.”
John Robinson of Our Ability.
The plan that attracted Microsoft is Robinson’s idea to make a chatbot to help collect critical data from disabled applicants. And before you say “chatbot? What year is this?” remember that while chatbots may be passé for those of us able to navigate forms and websites easily, that’s not the case with people who can’t do so. A chat-based interface is simple and accessible, requiring little on the user’s end except basic text input.
Pison is another grantee whose technology would come in handy here. People with physical disabilities often can’t use a mouse or trackpad the way others do. Founder Dexter Ang saw this happen in person as his late mother’s physical abilities deteriorated under the effects of ALS.
His solution is to use a electromyography armband (you might be familiar with Myo) to detect the limited movements of someone with that type of affliction and convert those into mouse movements. The company was started a couple years ago and has been undergoing ongoing development and testing among ALS patients, who have shown that they can use the tech successfully after just a few minutes.
Voiceitt is a speech recognition engine that focuses on people with nonstandard voices. Disabilities and events like strokes can make a person’s voice difficult for friends and family to understand, let alone the comparatively picky processes of speech recognition.
Here are the remainder of the grantees (descriptions are from Microsoft):
University of Sydney (Australia) researchers are developing a wearable sensory warning system to help the 75 million people living with epilepsy better predict and manage seizures to live more independently.
Birmingham City University (United Kingdom) researchers are developing a system that enables people with limited mobility to control digital platforms using voice commands and the movement of their eyes.
Massachusetts Eye and Ear (Boston, MA) researchers are working on a vision assistance mobile app that offers enhanced location and navigation services for people who are blind or have low vision.
University of California Berkley (Berkley, CA) researchers are creating a mobile app for individuals who are blind or have low-vision to provide captions and audio descriptions of their surroundings.
The image up top, by the way, is from iTherapy’s InnerVoice, an app that provides AI-powered descriptions of images taken by kids who have trouble communicating. It’s a great example of cutting-edge tech being applied in a niche that helps a small population a lot rather than a large population a little.
Microsoft has been a good steward of accessibility for years and it seems to be leaning into that, as well it should. President Brad Smith had a lot to say about it in a blog post last year, and the commitment seems strong going into this one.