Around 2021, it has been increasing to see and hear the word "meta spring".

In the world, Meta, Microsoft, Qualcomm, NVIDIA, and Japan have announced that Panasonic, Sony, and Softbank will enter the metabers market, and all are looking for new ways to connect online.

Metaverse can be accessed from smartphones and PCs, but immersive experiences with VR headsets and A R glasses are considered to further develop the meta spring.

This article is an article that you want to read for those who are trying to incorporate VR and A R into business due to the excitement of the metaberth.

table of contents

What is Metaverse?

Metaverse is a generic term for virtual spaces built on the Internet and is a coined word that combines "Meta (transcendent)" and "Universe".

Users can operate an avatar in a virtual space to move freely in the virtual space, interact with other avatars, do shopping, and work.

Metaverse is an expansion of Internet technology, and the access point of Metaverse includes PCs and smartphones in addition to virtual world technology such as expansion reality (A R), complex reality, virtual reality (VR).

The word meta spring first appeared in 1992.It is said that this word was used in a novel "Snow Crash" by science fiction writer Neil Stevenson.

In 2003, "second life" became a hot topic as a virtual space where you can enjoy various experiences in the virtual world.

In the second life world, users were able to produce everything from avatars, landscapes, buildings, and fashion, and were able to transfer and sell them.It is said that more than 1 million users accessed more than once a month.

After that, in 2021, Facebook announced that it would change the company name to "Meta", the word meta -shaker spread at once.

In fact, the space where you can experience metaverse is "Decentraland".

In Decentoland, you can use the virtual currency "mana" to carry out economic activities, and buy and sell items and avatars on the land, virtual space.

However, not all companies agree with the immerseive meta -shaks with VR headsets.

META's Zuckerberg says, "Communication will be improved by devices that are always worn in the future," but Niantic, a startup company that develops Pokemon GO spinning out from Google, is restrained to VR headsets.The meta spring that is done is called the "Dystopia nightmare".

Niantic advocates the "real world metaverse," which is not a virtual world in a immersive digital environment that uses A R technology to combine the real world and the digital world and connect people directly.

Each company will make use of its strengths and demonstrate its power in a specialty field, but what kind of field will meta spring spread in the future?

IDC anticipates that the development of metaberth will affect the six major areas in the market of social connections, work, entertainment, shopping, education, and digital economy.

What is XR?

XR is a general term for VR, A R, and MR.

A paper published by Paul Milgrum of Toronto University in 1994 indicates the relationship between VR, A R and MR in the spectrum shown in the figure below.

In this paper, the real world and the VR environment are defined as a continuous change instead of a game relationship, and the different ratio is classified by the difference between the ratio of virtual and real.There is.

MR is a wide concept that combines A R and extension VR, and is a mixture of real and VR.In this paper, the scope of extension reality and extension VR, the part where the virtual and real is mixed is defined as A R/MR.

What is VR?

VR stands for "Virtual Reality", and is often translated as "virtual reality" in Japanese.

According to the definition of the Japan Virtual Reality Society, it is defined that "the male and shapes are not the original, but they are essential or as effective."It is a technology that creates a virtual space in a world or difficult situation in a world that is not real in reality by CG.

To experience VR, wear a headset such as goggles and head mounted displays (HMD).Covering all of the visibility increases immersive feeling.

Scenes where VR is used

VR uses a lot of entertainment such as games, but in business use, education and training use is leading the market.

By reproducing a situation where it is dangerous or costly to realize it in reality, you can experience it over and over again.

Classification of head mounted display (HMD)

VR devices are often roughly divided into three classifications as follows.It is necessary to consider which HMD to use according to the usage and the number of users.

PC接続型VR

Connect to a PC and use it.It is suitable for high -image quality and complex processing because it is processed on a PC.There are many relatively expensive devices.

オールインVR

You can experience VR only with HMD.Since there is no cable connection, it is possible to move freely.

スマホVR

You can experience VR by inserting a smartphone.It is suitable for many people to experience VR and experience what VR is for the time being because there is no need for expensive devices.It is said that the immersion is low.

Production of VR content

VR can experience three -dimensional images by installing HMD, and you can experience a real reality.It is devised to create three -dimensional content.

両眼視差の利用

The fact that the video distributed to HMD looks three -dimensional is that each side of the left and right eyes is shifted only for the parallax (the gap in the eye position).

広範囲の動画撮影・配信

When shooting VR videos, shooting is performed at a range of 180 ° and 360 °, which is sufficiently wider than the effective field of human views.

The video displayed in HMD is only in the human viewing angle range.Therefore, when the neck is shaken left and right, a video of the part that was not originally displayed is displayed.

Type of tracking

A technology to measure the position of HMD using a sensor or camera.Knowing where the HMD is in the real world can match the position and movement of the head in the real world and virtual space.

Tracking methods include inside out and outside -in methods.

DOF

DOF (Degree of Freedom) is a degree of freedom, and in XR, it means the number of directions that can be detected by HMD.VR HMDs usually have 3Dof and 6Dof.The 3dof detects the rotation and inclination of the head, and the 6dof detects the movements in front, back, left, right, right.

In recent years, 6dof HMD has become the mainstream.

What is A R

A R is an abbreviation of "Augmented Reality", and is called "extension reality" in Japanese.

It is a technology that edits (addition, deletion, emphasis, attenuation, etc.) of the sensory information created by the real environment with a calculator.In general, it is often treated as a technology that adds visual information to the appearance of reality through filters such as smartphones and A R glasses.

Interactions that transcend physical restrictions and visualization with spatial information can be performed.Unlike VR, it cannot be used anytime, anywhere, and is affected by the situation in the real world.

Area where A R is used

The following items include the fields where A R is used.

作業支援

Visual assistance can be presented by hands -free, such as visualizing the following tasks and visualizing invisible things.Work support to appeal to the user's intuition is possible.

訓練（教育、医療）

Until now, the training had to be complemented in the brain while alternately comparing the teaching materials and the real thing.By utilizing A R, information with three -dimensional and time flow can be described in the operation target itself.

ナビゲーション

In the real space, a sign that is in line with the individual can be produced temporarily according to the destination.

ショッピング・コスメティック

In addition to going to a physical store or ordering products, you can try it remotely, and you can convey information that cannot be conveyed in photos and videos.

エンターテイメント

By using characters and settings that do not exist in the real world, it is possible to realize an extraordinary activity in the daily life of the real world.

A R display classification

A R has various types.Here, we will introduce the classification on the display.

The display is roughly divided into three types.In the environmental installation type, projection mapping, which presents visual stimulation to the environment itself, corresponds to this.The handheld type uses a smartphone or tablet camera.The wearable type uses a dedicated device.

In addition, there are displays that add information to hearing and tactile sensation.

環境設置型

No special preparation is needed for experienced people.It requires a large setting to realize the experience.

ハンドヘルド型

There is no need to develop a new device.The immersion is low because the display size is limited to the terminal.

ウェアラブル型

Hands -free, directly works on individual users.A dedicated device is required.Many are still expensive.

A R and smart glass

A R glass and smart glass may be confused.Both are shaped like glasses, and their roles are similar in terms of giving additional information to visuals.The difference between the two is whether to recognize the space and overlap digital information.

スマートグラスとは

It often refers to glasses -type displays equipped with displays.In addition to the display, it is equipped with a camera and a microphone, so you can connect to photos, calls, and connect to the Internet to display various information.

A R and smart glassの違い

There is a difference between recognizing the space and overlapping digital information.It is often called a smart glass that only displays information, regardless of the surrounding environment.

About classification of MR and A R

MR is an abbreviation of "Mixed Reality", and is called "complex reality" in Japanese.If the virtual space and the real space are mixed with each other, there is no clear distinction between A R and MR.

What is called "MR device" in the market is an A R device (higher than existing), or the emphasis is on presenting the content that combines functionally and contextual with the real environment.It is said that there are many.

For example, there are those who wear wearable devices and work using both hands, or those who experience a common virtual content with multiple people.

The value brought by XR

XR features

VR creates the context of "looking at the sea in a tropical resort."A R expands "destination information" in the context of "at the scramble intersection in Shibuya".

Points to bring out the goodness of VR and A R

VR：距離的・時間的制約が解放されることによるインパクトが大きければ大きいほど価値が高い

For example, considering the experience of walking around Asakusa, foreigners who are expensive to go to Asakusa have more benefits.It is important for the user to save the cost depending on the VR.

On the other hand, considering the "experience of walking around Asakusa 10 years ago", it is not accessible to Asakusa 10 years ago.One of the values of VR is that you can experience something that is impossible in the first place, rather than cost.

A R：変化する利用者のニーズに応じて、必要な情報を遅延なくシームレスに届けられると価値が高い

For example, suppose there is a service that the person information is expanded with A R when you hold a smartphone over a business card.As a sales tool, it may be as good as the story, but it may not be used much.

On the other hand, I feel that the A R function of the Q4 e-tron announced by Audi is very convenient.

Q4 E-Tron is equipped with an A R function that displays information in the front screen in two parts on the front screen.At the top, navigation and lane deviation warnings are displayed about 10m forward about 10m forward, with images over the view of the windshield, and at the bottom is displayed about 3m ahead.

XR market trend

出典：IDC　A R & VR Headsets Market Shareを参考にアールジーンが作図

According to an IDC survey in 2021, the global shipment of A R and VR headsets increased by 348.4 % from 2020.All -in -VR headsets were shipped 89.8 % in the market, and Meta (formerly Facebook) was the top market share with a share of about 75 %.

The shipments of A R and VR headsets are expected to grow to 32.8 million units at an annual average growth rate (CAGR) of 45.9 % by 2025.The market size is Leaded by VR, but A R is expected to grow significantly from 2023 to 2025, and to gain a third share by the end of 2025.

Trends of XR solutions seen from facts (facts)

VR：教育・研修向けに浸透

Bank of America announced that VR training will be conducted on 45,000 employees.In addition, teaching materials are being developed on how to respond to customers who have emotional issues such as the death of relatives.

VR：ハードウェアの価格が低下

May 21, 2019 "Oculus Quest" was released for 45,273 yen, but "Oculus Quest2" was released for 33,800 yen on October 13, 2020.* All 64GB, excluding tax

VR：バーチャル店舗など新しい活用法

JR East and DOCOMO conducted demonstration experiments at JAPAN RAIL CAFE, adjacent to Tokyo Station.Here, you can travel around tourist attractions in Aomori Prefecture with a VR headset, and you can purchase local products while talking with sales staff in VR (products are settled in Café and received at stores).

A R/MR：教育・研修を中心に普及

Polytex Center Akita, which conducts vocational training for reemployment applicants, has introduced a system that can practice welding work using A R technology.The system is made in Spain and is 7.5 million yen.If there is a home power supply, it can be welded anywhere.

A R/MR：製造業向けの遠隔作業支援

PTC's "VUFORIA CHALK" provides apps, and workers can share local images with remote directors.The remote instructor can write on the video and give accurate instructions.Even if the camera is moved, the elements written do not shift.

A R/MR：Covid-19感染拡大

TeamViewer has announced the acquisition of UPSKILL, a pioneer in A R solution, which supports the manufacturing field work in the US market.Due to the new colonavirus epidemic, the remote work support initiatives accelerated.

VR device status

Look at some recent topics.

Oculus Quest2 is used not only for games but also for various purposes

Students of Saito Seminar of Chuo University Faculty of International Information have experienced VR English classes.

From students who actually experienced VR English classes, the effects of VR English classes are the reduction of emotional filters (psychological anxiety during language learning), the possibility of long -term memory retention, and motivation improvement.It was given.

On the other hand, the issues such as the expression of the other party and the visual information such as gestures, and the appropriate share of personal information to companies when using VR was also discussed.

In addition, at the JAPAN RAIL CAFE TOKYO, which is adjacent to JR Tokyo Station, an event utilizing the VR "Future Product Exhibition from Aomori" was held from March 17 to 28, 2021.

Speaking of "product exhibitions", it is an event where you can eat and buy delicious local ingredients and specialties, but now there are fewer opportunities to exhibit from rural areas and shopping now."Future Product Exhibition from Aomori" will realize such difficult product exhibitions with VR.

Panels with Aomori specialty products, alcohol, sweets, etc. are displayed, and you can enjoy shopping while interacting with local sales staff on the monitor.

Facebook's corporate VR service Oculus for Business

On May 21, 2020, Meta's affiliated Oculus VR announced that it has officially launched the Oculus for Business platform for business use of VR technology for business.

The VR headset used in the "Oculus for Business" is equipped with functions for general users such as game stores, and is equipped with functions to set up many devices, remote control, and security measures.Realizes convenience and safety.

Oculus for Businessを利用したJohnson & Johnsonの、VRトレーニングを実施した際に見られたパフォーマンスの向上は233％にのぼるという。

In addition, the annual cost reduction of 10 sales representatives per month by training 10 sales representatives per month is $ 100,000.

Things to notice by comparison between Quest and Quest2

Why is the price so cheap when the performance is clearly improved?The hint is a podcast interviewed by the US Media THE INFORMATION: Zuckerberg said:

Based on the above comments, I briefly illustrated Meta's business model.

It seems that META is going to provide high -performance devices as low as possible, while selling content in application stores.

VR technological innovation

フェイストラッキング

Face tracking is a three -dimensional position and size of each face, such as the inclination, size, and eyebrows, such as the inclination and size, as well as the inclination and size of the eyes, nose, mouth, and eyebrows.To detect.

On March 10, 2021, Taiwan HTC announced the VIVE FACIAL TRACKER device that can track human expressions by attaching it to the VR headset.Track 38 facial movements, such as chin, teeth, tongue, and cheeks, with a camera or infrared line.The price is 139 euros (about 18,000 yen).

Compatible with the company's VR headset "Vive Pro" and "Vive Pro Eye" and combined with Vive Pro Eye, which can track eyeballs and focus, capture the entire face movement.

触覚フィードバック（グローブ型・コントローラー型）

A "tactile feedback" function that stimulates the receptor on the human skin and gives a sense of tactile, power, and pressure.In VR, it is expected that the feeling of actually touching the 3DCG object in the VR space can be reproduced.

"HAPTX GLOVES DK2" has a significant upgrade, including the company's two years in development, reducing the size and weight, improving fit and ergonomics, realizing room scale VR support, and strengthening the faithful of touch.Is added.

In the outer skeleton, when the user grabs objects in VR or operates the steering wheel, the finger movement is stopped and the force touch feedback is returned.Applies to up to 40 pounds per glove (about 18 kg, 8 pounds per finger).In Japan, Asuku Co., Ltd. is a distributor.

Sony has made a design that naturally fits the hands of all sizes, taking advantage of the knowledge obtained from the controller released on the PlayStation platform for several decades so far.

Equipped with a haaptic feedback, it enables expressions that make you feel more powerful, or feel the feel and nuance of every feel in the game.The difference in the feel is clearly felt, such as the feeling of crossing the desert lying down, and the feeling of interacting with fists with a melee attack.

触覚フィードバック（スーツ型）

A "tactile feedback" function that stimulates the receptor on the human skin and gives a sense of tactile, power, and pressure.In VR, it is expected that the feeling of actually touching the 3DCG object in the VR space can be reproduced.

TESLASUIT (VR Electronics) is a VR/A R -compatible suit -type device that integrates tactile feedback, motion capture, and biometrics.

Inside, the mechanism of the Wi-Fi transmission and receiving function, 80 channels that create electrical stimuli, 10 internal motor sensors, and 8 to 10 hours of rechargeable life battery are incorporated.In Japan, Ask handles "Teslasuit" on August 4, 2020.

It provides sensory experiences in simulation that reproduces dangerous environments and effective education training with XR content.

ブレイン・コンピューター・インターフェース

BCI (Brain-Computer Interface) is attracting attention as a VR/A R device input method.If it is realized, there is no need for controllers or audio operations, and you can input just by considering what you need in your head.

In February 2021, it announced that it will release "Cognix One" that can be operated just by thinking by Canadian startup Cognix (current situation unknown).

The application at the time of the announcement is for those who have problems with communication such as cerebral palsy and muscle atrophy side sclerosis (ALS).

Instead of reading the user's thoughts itself, it is possible to judge the gaze and the direction to be pointed from the brain, select from the menu screen reflected on the head -up display, or type the message.

These initiatives have been taking place for a long time.In a paper published by Stanford University in 2017, the brain activity is installed in the brain of patients with paralysis of the body, and the brain activity is read and the cursor on the screen is operated.You can see the situation on YouTube below.

VR内歩行デバイス

Walking devices in VR can walk and run in the VR space without moving a wide range in the real world.

"KAT WALK C" is a room runner -style device assuming that it is used at home.By fixing the body to the harness on the back and "moving" in any direction, the movement is reflected in the VR.

The company announced that it has completed funding with the national funding.Reported $ 1.66 million (about 175 million yen).Price for general sales, about 260,000 yen.Compatible with HTC Vive, Oculus Rift (Rift S), PlayStation VR, etc.(Also supports QUEST)

Virtuix's OMNI series is a device that allows you to walk and run in the VR without moving a wide range in the real world.By wearing dedicated shoes and walking on the floor of the device, the movement is reflected in the VR.

The latest device "OMNI ONE" announced in October 2020.Lighter and smaller than past Omni.The company raises funds by Clound Funding.As of March 10, 2021, $ 11 million (about 1.2 billion yen).

テレイグジスタンス（遠隔存在）

Teleiggiji (remote existence) is a system that controls a mobile robot with a human eye and skill function from a remote area and transmits its presence and reality.

The market size is expected to grow at a rapid pace, and it is thought that it will grow significantly in the form of involving existing remote transmission systems, robotics, VR, SNS, and artificial intelligence (AI).

The latest movement of Gitai JAPAN, which develops humanoid remote control robots using VR, is as follows.

2019年2月　プロトタイプ開発

Developed a remote control robot prototype that replaces astronaut's work.Working cost is reduced to one -tenth.If you wear a VR device and a tactile glove, you can synchronize with a humanoid robot with a 360 -degree camera in another place and a wireless communication, and you can manipulate it like your own body.

2019年8月　4.4億円調達

Develop a robot and raise funds to launch a demonstration experimental machine to ISS at the end of 2020.Spiral Ventures JAPAN was the lead invaster, and DBJ Capital, J-Power, and 500 Startups Japan invested.

2021年3月　18億円調達

We carry out development costs and labor costs for out -of -space ships, out -of -ship technology demonstrations scheduled to be implemented in 2023, and financing for labor costs for expanding the US market.In the summer of 2021, GITAI robot demonstration of general -purpose work execution technology on the ISS ship.

A R glass status

Visual displays that realize A R can be classified into three types: "environmental installation", "handheld", and "wearable".

"Handheld" and "wearable" provide personalized experiences, but smartphones tend to be selected as a trade -off between costs and quality.I wonder if A R / MR glass will spread in the future, and what is the task at that time.

Main A R glass

HoloLens2(米Microsoft)

HoloLens2 is a standalone device that realizes a complex reality that allows you to combine the real world in front of you and CG.Equipped with Windows10.

Microsoft provides an application that is focused on industries and can be used immediately from the time you open the box.

HoloLens2 has doubled the viewing angle compared to the first model, making it possible to operate with hand tracking and eye tracking.Compatible with 6dof tracking.The operation does not require a controller, and can recognize the movement of the 10 fingers of both hands, so you can grasp the displayed 3D objects and operate them in detail.

Uses an inside out method that uses the sensor mounted on the object you want to measure to measure the position of the object.The weight is 566g.

Magic Leap 1(米Magic Leap)

MR headset developed by Magic Leap, USA.

It is equipped with a headset with a glasses shape, a controller, a processor and a unit with a battery, and is equipped with a unique OS.The news that succeeded in raising funds in units of 100 billion yen from major companies such as Google and Alibaba has attracted a lot of attention.

When compared to HoloLens, Magic Leap One is a device specializing in more creative experiences.In Japan, NTT DOCOMO is developing with the aim of spreading MR content combined with 5G.The weight is 325g.

Nreal Air(中国Nral Ltd.)

NTT DOCOMO and KDDI announced on February 14, 2022 that the smart glass "NREAL AIR" will start selling on March 4.

With a built-in small display inside, connecting with a smartphone and USB Type-C allows you to enjoy content such as videos as if a 130-inch virtual screen was projected 4m ahead.It is 79g lightweight for batteryless (power supply from smartphone).

While "HoloLens" and "Magic Leap 1" are high and high -performance, NREALLIGHT offers low prices for general users.

The function is limited compared to the other two models.The number of cameras for tracking the position is less than Hololens and Magic Leap 1, and the accuracy is inferior.The controller is also a simple one using a smartphone, and the range of operations using hands and fingers is limited.

NREAL AIR is used by connecting to a smartphone (only Android), but HoloLens works alone.Magic Leap 1 uses an external unit.

Introduction and consideration for corporations are progressing

HoloLens2がNASAの宇宙船Orionの開発を支援

Lockheed Martin, a former contractor of the spacecraft ORION, uses HoloLens 2 to assemble the various spaceships used in NASA's A RTEMIS II Mission, the first mission to carry Orion.

Million of the spacecraft production requires millions of work, and no error is allowed, such as the correct installation of the electric cable.Necessary information, such as animation of parts assembly method and bolt tightening torque, can be obtained from HoloLens2 and can eliminate assembly errors.

In the work of marking the screw position, the technician who received the holograph work was able to achieve 90%faster.

ニトリがのMagic Leap 1を活用したショールーム

A collaborative agreement on the study of docomo and Nitori XR services was concluded on Tuesday, February 23, 2021.

In the conventional showroom, depending on the size of the installation space, the combination of color variations and options that the experience could actually experience was limited.However, by using MR, you can see the 3D model customized by the experience from the huge variation installed on the digital.

In addition to just looking, you can experience the operation of the drawer opening and closing as if you were actually touching it.These experiences can be performed simultaneously with multiple people.

科学未来館で5G×NrealLight活用した、新たな体験

The Japan Science Museum, KDDI, and KDDI Research Institute have developed a digital exhibition "Hyper Landscape", which uses XR technology to the exhibition of the Future Museum to overlap digital content, and for 4 days from March 11, 2021.I went between 15:00 and 17:00.

When a visitor wears the A R glass "NREALLIGHT", "Virtual Human COH" appears and provides an exhibition guide.COH rendering is performed on the cloud side, and it is streaming to transfer A R glass in real time.It is always uploaded from the image from the A R glass, and by collecting it with a 3D map, it grasps the viewer.

Cases for consumers

In cases that are used for consumers, the use of smartphones that are easy for users to try are often used.

購入率や返品率改善に効果 ECへの3Dモデル・A R導入

The e -commerce platform SHOPIFY has provided a product confirmation function using 3D models and A R.

Gunner in the United States is a carry case for transporting dogs.The company's products have a strong strength, but they are heavy and costly.If returned goods are accumulated, transportation costs will be a big pain.

Therefore, Gunner utilizes SHOPIFY's 3D/A R function.The 3D model has been uploaded to the product page so that the user can check the carry case from the angle as they like.

In addition, the A R function is also adopted so that the carry case is displayed using A R using a mobile device so that it can be purchased after comparing the dog at home.

Compared to the previous quarter, the conversion rate for site visitors to put products in shopping carts is 3 %.The conversion that actually led to the purchase rose by 40 %.Returns also decreased by 5 %.

Champs Sports A R試着機能でユーザーのシェア数増

CHAMPS SPORTS (Champs Sports) in the United States has started an A R trying on Snaps Chat.

The A R Trion (A R Trying) is based on a function released by Snap.inc in June 2020.

Users can start the snap -chat app and hold the smartphone camera over their feet to see how they wear sneakers in A R.If you like the product, you can tap the "Shop Now" button to purchase it.

According to Champs, there are already many shares about A R trions.He also explains that it has a five -fold performance compared to the fitting of the store.

Attempts on A R devices

A Rコンタクトレンズ「Mojolens」

A R's emerging company MOJO VISION is developing contact lenses that can display information in the field of view simply by attaching them to the eyes.You will be able to see information from outside the world, such as temperature and weather, health data such as heart rate and body temperature, and messages from friends and family.

According to MOJO VISION, MOJO LENS is currently conducting clinical research under the approval of the US Public trial examination committee (IRB), and low vision such as retinal pigmentation, night blindness, etc. with real -time technology enhancement of lighting contrast.Planning to support people is ongoing.

Meta リストバンド型コントローラー

Facebook Reality Labs (FRL), Meta's A R / VR Technical R & D department, posted a blog post on the future of Human Computer Interaction (HCI).It describes a wristband -type input device that combines with AI that understands context.

FRL is considering the ideal input device type of A R glass.It is said that it has arrived in a wristband type because it can be worn comfortably all day, it is not very noticeable at the time of installation, and it is easy to take in hand movement information.

For example, audio is intuitive, but private places cannot be maintained in private and cannot be used with background noise.Devices like smartphones and game controllers have friction layers between you and your environment.

Software (tool for developers)

VR & A R: Microsoft Mesh (multiplayer framework)

Microsoft announced a new framework "MESH" using VR and A R at the company's event Ignite, which has been held since March 3, Japan time.

MESH is a multi -player framework provided by Microsoft for developers using Azure.Developers provide voice communication in space, cloud -based asset management, scenes preservation, and avatar system.

Compatible with other companies' VR headsets such as Oculus Quest 2, and PCs and smartphones.

There are four major issues that can hinder MR app developers in creating high immersion experiences.

MESH is trying to solve the above series of issues.

In addition, Microsoft has announced that it will release Mesh for Microsoft Teams in 2022.

Mesh for Microsoft Teams is a Microsoft Mesh MIXED REALITY (composite reality) function that allows people in physically different places to participate in collaboration through a common holographic experience.This is a fusion of TEAMS productive tools that enable collaboration above.

According to the company's blog, "Mesh for Microsoft Teams is the entrance to the digital world called" Metaverse ", a digital twin of people, places, and objects.Is announced.

VR & A R: Metahuman Creator (Virtual Human)

Epic Games, a popular online game "Fort Knight", has announced a new browser -based app "Metahuman Creator".Can be used for free.

With "Metahuman Creator", game developers and real -time content creators can reduce the time it takes to build a digital human from a few weeks to one hour.

You can start working simply by selecting a preset with a face you want to use for digital human from a wide range of samples.Corresponding to all kinds of faces of young and old, created digital human assets can be used immediately in Unreal Engine.

In April 2021, when financing $ 1 billion, Epic Games announced that the reason for procurement was "to support the long -term vision of metaverse."

VR & A R: Ready Player Me (avatar)

Ready Player Me, developed by 3D scan -based 3D scanning in Tallinn, Estonia, and Wolf3D, an avatar company, can create a 3D avatar from a user's selfie.

The feature is that the goal is not to rely on specific games.

WOLF3D is a blog, saying, "Metaverse is not a single app or game. Thousands of virtual world networks that people visit for people to play, work, and collaborate. Users have new avatars for each game or experience.It doesn't make sense to create it. Your avatars should be able to move with you across metavers. "

Ready Player Me also supports Mozilla's "Hubs" and "VRChat" recently, and the service currently supported is about 50.It is said that it is still working with many developing companies.

VR A R: RealityCapture

Epic Games, which provides game engine Unreal Engine 4 (UE4), has newly acquired Capture Reality.

Capture Reality is a Slovak company known for its photo glitter tool "RealityCapture".Photo grametry is a technology that creates a 3D model of a real space or object from multiple photos.

By using "RealityCapture", a 3D model of space and objects can be generated based on multiple photos.Through the acquisition, the Capture Reality team will continue to provide 3D scanning technology.

A R: Azure Remote Rendering (Remote Rendering)

Azure Remote Rendering officially released on March 3, 2021.

By using this Azure Remote Rendering, the drawing on HoloLens alone is approximately 100,000 polygons ( *), while Azure Remote Rendering can handle tens of millions of polygons to hundreds of millions of polygons. became.

"MIXPACE REMOTE RENDERING", which incorporates Azure Remote Rendering into the service on March 4, has begun to be provided by Holorab.

* Polygon is a polygon used in 3D graphics to express the curved surface of a three -dimensional object.In most cases, a triangle is used in the minimum unit that makes a curved surface.The larger the number of polygons, the more smooth the aspect, but the time required for calculation processing is longer.

A R: Arcore/Arkit, etc. (A R framework)

With the appearance of the A R framework (a commonly used common operation and processing package), the A R app can be easily developed.

The advantage is that the basic movements required for the A R app can be introduced and developers can use it for free.

A RKit（apple）

A R application development tool for iOS devices announced by Apple in June 2017.

The spatial structure can be recognized based on the sensors and image analysis data mounted on the iPhone without using a special hardware, and the A R experience can be realized.

Specifically, functions such as images (markers), planar recognition, and distance recognition can be implemented.After that, the version was upgraded, and now it is "A RKIT 4".

In the first quarter of 2022 (October to December 2022) in the company's first quarter of the company, Tim Cook, who was asked by analysts about Metaverse, said, "App Store has a 14,000 A RKIT app, and is currently now.We offer a great A R experience for millions of people. We have a great possibility in this field and we are investing in appropriate. "

A RCore（Google）

A R application development tool for Android devices announced by Google in August 2017.

Like A RKIT, A R content can be constructed only with smartphone cameras and motion sensors.Smartphones and tablets of various manufacturers, such as Google, Huawei, and Samsung, support Arcore.

In addition, there are also the official tool kit "Magicleap Toolkit" provided by Magicleap, and the cross -platform tool kit "MIXED REALITY TOOLKIT (MRTK)" for the development of VR and A R provided by Microsoft.。

A R: Real World Platform (A R Cloud)

A R Cloud is an A R content placed indoors or outdoor places "in that place" or "in that state", and always "multiple people can be shared at the same time".It refers to a technology that realizes a world that can be restored.

In order to realize a city -scale A R cloud, it is thought that it is necessary to create a digital copy of the entire city (for example, a format such as point data) and hold it on the server.

ナイアンティックのA Rクラウドの動き

Niantic, known for the development of Pokemon GO, acquired the American A R Startup 6D.AI.Niantic will promote the world's dynamic 3D mapping with the 6D.ai team to achieve a full -scale A R experience.

The 6D.ai, which was founded in 2017, spin -out from the Active Vision Lab at Oxford University, based on developed tools and computer visions to solve the basic issues of A R, such as three -dimensional reconstruction and the permanent of A R.We have developed the technology we used.

John Hanke, founder and CEO of Niantic, is a Metaverse in August 2021, and the Metaverse is a Distopia nightmare that wears VR headsets like "Lady Player 1".He says.

Niantic is said to use technology to snuggle up to the "reality" of expansion reality.

Later, in November 2021, he announced a vision and A R developer kit called "Metaverse in the Real World".This is to publish a tool for developing A R games, and anyone can use it for free if you have the basic knowledge of the game engine "Unity".

KDDIのA Rクラウドの動き

KDDI has concluded a strategic partnership with Sturfee, which is developing A R cloud -related technology.KDDI uses Sturfee technology to generate 3D maps from satellite photos, and jointly promote the planning and development of services that use VPS technology that can hold the smartphone and get information.

VPS (Visual Positioning Service) is a developed system of conventional GPS (global positioning system), and is mounted on a real world digital twin (3D mesh) and a smartphone or smart glass.It is a technique that matches images over the camera and identifies high -precision location information, including orientation and direction.

By combining with A R (extensive reality) that utilizes smartphones and smart glass camera functions, it creates various experiences such as digital signs and navigation linked to location information, as well as advertising, entertainment, art, and education.It is expected.

Sturfee has a technology that can generate 3D models used for VPS from satellite photos, and compared to 3D generation using images taken with conventional smartphones and cameras, more seamlessly, efficiency of the city and the entire space.It is possible to convert it to A R.

5G network

AWS WAVELENGTH

KDDI and AWS will launch AWS WAVELENGTH in Tokyo on December 16, 2020 in the au 5G network.

AWS WAVELENGTH is to operate various AWS services with affiliated telecommunications carriers and other facilities.By placing AWS computing services and storage services in the au 5G network and processing data, it is a 5G characteristic of an ultra -low delay.

There are many useful examples for 5G, but VR has a benefit.Now that there is a large amount of local processing, you have to attach a heavy headset, but if you can use AWS WAVELENGTH to transfer to the cloud, the headset can be lighter.

Law and virtual space

The rules for using the expansion reality (A R) and virtual reality (VR) for games and advertisements are delayed.This is because the current law does not assume copyright and ownership of ownership when reproducing reality in a virtual space.

The domestic market of A R and VR is expected to grow to over 1 trillion yen, 2.4 times compared to 2025, and businesses have entered one after another.It is necessary to discuss legal development as the technology is spread.

For example, installing advertisements in temples and general buildings in A R and VR space without permission, holding a smartphone over a bag to other companies' bags to emerge their logo with A R, and use the land owned by others.However, opening a live in the A R space is an example of using A R and VR that the current law does not assume.

In late October 2020, the "Virtual Shibuya", which was opened in a virtual space in late October 2020, was composed of the "Shibuya 5G Entertainment Project" composed of KDDI and others.

The highlight is that the scenery near the scramble intersection, such as Shibuya Station and the SHIBUYA109, is faithfully reproduced with VR, but if you look closely, the surrounding building signs and corporate logo are not real.According to KDDI, "we were particular about the streets and other streets, but avoided the risk of infringing other companies' trademarks."

summary

Meta spring

Some companies, such as META, depicting the future of metahares that are immersed in VR headsets, and like Niantic, using A R technology to fuse real worlds and digital worlds.Some companies are ideal about "real world metavers", which is not a virtual world in the immersive digital environment that directly connects.

Despite various companies in each company, the development of metavers will affect the six major fields of social connections, work, entertainment, shopping, education, and digital economy.

VR

The implementation of face tracking and tactile feedback has progressed, improving immersion.It may be applied to customer facial expressions, served customers, or to simulate surgery.

In order to generalize live events and sports watching in real time in real time, there are many issues that need to be overcome 5G, the price of devices, and the production cost of high quality content.

Expectations for remote existence are increasing, but it is necessary to grow while involving the technology of robotics and tactile feedback as well as 5G, and technology progress is awaited.

A R

The similar solution seems to be solid, as some companies have been acquired due to the expansion of solutions for remote work support due to the expansion of the corona infection.

For indoor use and corporate applications, high -definition 3DCG can be expressed in the combination of local 5G and remote rendering, and progress in the A R experience is likely to be seen.

A Rs are expected to blend into our daily lives, the weight, power consumption of A R glass, the spread of 5G, and the evolution of input systems, but it is still likely to take some time.

DX business support service that allows you to learn DX trends for corporations

Art Gean Co., Ltd., which is the operating body of IoTnews, provides services to support "collection of trend information" and "acquisition of actual know -how", which are essential for DX.

This article is a study session digest article, but in the actual study session, you can hear more stories from experts who are in charge of IoT and AI sites and ask questions directly.

Please check the link below for details.

"DX Trend Commentary" and "DX Practice Course" are a corner where the consultant of RG Gean Co., Ltd., which is the operating body of IOTNEWS, conveys his experience and knowledge.

Please contact us from the inquiry form for requests such as "I want you to hold a study session systematically", "I want the advice of planning", "I want you to do consulting".