Mixed reality pushes structural fabrication forward

Mixed reality is one of the most promising advances propelling the construction industry forward. It has the potential to simplify the design, build, and operate experience for every person touching a project. It takes constructible models and the data within them beyond the computer screen and into the physical world, opening the door for true collaboration and less waste.

Mixed Reality in the Fab Shop

Envision a project where all team members have access to mixed-reality headsets. In the shop, crews use this technology to coordinate and orchestrate fabrication. Out in the field, others access the same models to ensure accurate installation. Many of these use cases are already being implemented today.

Through their headsets, the fab shop team receives a notification that an assembly is ready for fabrication. They access the model, align it to the fabrication area, and view at full scale. Digital instructions included with the model provide guidance to gather the individual pieces and build the complete assembly.

Before starting fabrication, the team can verify the overall model against a 3D laser scan of the actual project site location to ensure accuracy. At the job site, installation crews view the model at full scale to visualize the construction sequence and plan their work to ensure safety and efficiency. If conditions require a change to the installation sequence, team members in the fabrication shop don headsets and are virtually “transported” to the job site to interact with the field team. The field team interacts with avatars of the fab shop team, and together—virtually—they find a solution. The models are then updated and new instructions are provided to all team members via their headsets.

The Basics of Mixed Reality

The terms “mixed,” “augmented,” and “virtual” reality are often used interchangeably, but they are discrete technologies. All three are different forms of extended reality. To understand the various ways they can benefit the industry, it’s important to know how they’re different.

Virtual reality is an immersive,100 percent digital experience. It removes the context of the physical space and replaces it with a 3D environment.

Augmented reality is a 2D approach that combines the physical and digital worlds to enable the user to view digital information in the context of the physical setting the user occupies. Mixed reality, as the name suggests, is a 3D hybrid of the digital and physical, where the two realities coexist and the user can interact with each.

The Time Is Right for Mixed Reality

The construction industry has made major strides in moving from 2D to 3D building information modeling (BIM). Most would agree that working from a fully developed, data-rich constructible model creates tremendous value. Mixed reality extends the value of that data and creates new opportunities to leverage BIM throughout the project.

Viewing the model overlaid on the physical environment allows people to perceive spatial relationships, distances, and sizes as they are meant to appear when the project is finished. Without mixed reality, the interpretation of digital content and its translation to real-world objects depend heavily on one’s spatial understanding. Not only does this way of working require highly skilled workers, which is complicated by the industry’s current labor shortage, it leaves room for errors that often lead to poor quality, cost overruns, and schedule delays.

Mixed reality brings the model off the screen so people can engage and interact with it on a one-to-one scale. Wearing a mixed-reality headset, a fabricator can visualize and interact with a data-rich 3D model in the shop, isolate and extract connections from the model to check for accuracy, and identify missing components or other deviations. During fabrication, mixed reality can ensure detailing aligns with the erection team’s preferred method of assembly, before manufacturing instructions are sent to production.

An Early Adopter

Mixed reality might seem like a far-out concept that hasn’t yet found its way, but early adopters have been experimenting with it for years. Consolis, one of Europe’s largest precast manufacturers, was one of the first to try mixed reality.

At the time Consolis was considering a laser-based system for quality checking of rebar, but the laser was cost-prohibitive and difficult to use. Curious about mixed reality’s potential to replace traditional work flows and optimize the production of rebar cages, Consolis explored a proof of concept pilot in 2016. Using Trimble Connect for HoloLens on the factory floor, a team of four manufactured a rebar cage without any prior knowledge of building rebar. They used the HoloLens for placement of the rebar and to complete nearly 50 steps in sequence, using a voice command to move from one step to the next until the cage was filled.

This made it possible for workers with no prior experience to manufacture the cage and deliver it to the site quickly. Through the pilot project, the Consolis team discovered a strong business case for mixed reality. Specifically, they could use it to avoid costly mistakes in production and accurately perform quality control.

Why Mixed Reality?

Mixed reality emulates real-life scenarios. The 3D experience provided by mixed reality illustrates size, dimensions, and proportions, so it’s far more reliable than 3D models on a flat screen.

Perhaps its biggest benefit is that information is retained, remaining in full 3D throughout the project. Because 3D information is unambiguous, misinterpretations and errors are rare.

Although the industry creates highly detailed 3D models, a great deal of time is still spent finalizing 2D drawings due to manual additions that are needed to make them fully understandable. Creating a 2D drawing in which information corresponds perfectly with a 3D model takes time and precision. Working in 3D from the beginning, using mixed reality to visualize the model as close to reality as possible, accelerates the design process and, ultimately, the entire project.

Mixed reality allows project stakeholders to iterate on design concepts in 3D, collaborate with others while immersed in the design, overlay designs on the real world for clash detection and visualization, and use models in the real world as an instruction guide during actual production. Testing in the field with mixed reality can spot errors before assembly.

Looking Forward

Mixed reality and constructible models address many of the major inefficiencies in design and construction today and will no doubt have a significant impact on the industry over the next few years.

Beyond mixed reality, potential applications of extended reality span architecture, engineering, construction, mining, and utilities. In civil engineering and utilities, companies are using augmented reality outdoors to visualize, explore, and place georeferenced 3D models above or below ground. Crews are using extended reality to locate buried utilities and visualize designs before starting the work.

In mining, workers become immersed in and interact with digital twins of open pit mines while sitting at their desks thousands of miles away. Important information such as stockpile volumes, ore movements and grade, and near real-time locations of machinery and trucks can be overlaid onto a spatial map of the mine site so it can be viewed using a headset remotely. A manager in Toronto can immerse himself in a near-live digital twin of an operation in western Australia.

Technology advances have reduced the barrier to entry and put mixed reality within reach. Mixed-reality tools are becoming easier to use, and within the next few years headsets will be small and comfortable enough to use all day and provide hands-free access to real-time information.

Access to real-time information will be key. In construction, so much waste comes from communication problems, and mixed reality could one day eliminate the lion’s share of them.

Jordan Lawver is portfolio manager, mixed reality, Trimble Buildings, Trimble Inc., 935 Stewart Drive, Sunnyvale, CA 94085, 408-481-8000, buildings.trimble.com.