UX Research, Conceptual Design, Human Factors

UX on the Plant Floor

As manufacturing and automation become more sophisticated with the fourth industrial revolution, the systems that drive production become more involved. This case study exposes issues of usability and learnability with existing manufacturing systems and explores alternative methods to improving user experience in Industry 4.0 systems.

The Challenge

One of the underlying problems for human-machine interfaces in Industry 4.0, from the perspective of human-centered design, rests within the holistic interactions throughout production environments. In addition to the complexity of automation itself, several other factors affect the usability of the system, such as training and education level of users, labor conditions, work environments. Industry 4.0 systems are also sophisticated with pre-programmed decision making, automatic error recovery, and the synchronization of big data being transmitted and interpreted differently at multiple levels of the automation hierarchy.

My Role

This case study is a combination of academic research and experimentation with user-centered design as a software engineer at a manufacturing consulting firm. The artifacts throughout the case study are products of opportunities I had to explore research, design, and usability testing methods during software integration testing and requirements gathering on projects. The proto-personas were created during a workshop that I led, and the wireframes and rapid prototypes were a team-based effort after evangelizing user-centered design. For the purposes of non-disclosure, some details of these artifacts have been generalized.

Industry 4.0 plant floor

Research & Discovery

Key Insight

Operators who are working in "smart factories" will be left to embrace the added complexities that these new systems bring.

Industry 4.0 systems are required to process and synthesize more data than ever, increasing the complexity of systems for designers, developers, and users. Manufacturing equipment has advanced from being aware of itself to being aware of the entire production environment. As these facilities integrate newer technologies, such as cloud computing and responsive design, into automation, production, and manufacturing, the value of user-centered design is inherent. The push for these plants built on an Internet of Things raises concern towards the usability needs of manufacturing operators more than ever.

Systems within this industry serve a broad range of users on many different levels of manufacturing and automation processes. While a manager may request services that provide high-level workflows, an equipment operator might require data to be retrieved off a sensor at the machine level. Historically, operators have not been a primary concern when designing these complex systems. Users are expected to learn and adapt to manufacturing execution systems regardless of usability, learnability, or how well the system fits into their existing workflow. However, this should not be their burden. Designers and developers are obligated to take these different groups of users into consideration when creating these systems instead of relying solely on their own conceptual model of how the system works.

Key Insight

Designers for automation and manufacturing should ultimately look to the e-commerce industry and their successes with implementing user-centered design processes.

Systems that support e-commerce are used by diverse user groups, where individuals range in levels of education and knowledge of the systems used. Because of the varied and vast amount of users in e-commerce, companies within the industry suffer from limited understanding of users and the environments in which they are interacting with e-commerce systems. Despite the complexities of an e-commerce user base, UX artifacts such as personas, storyboards, egocentric documentation, and rapid prototyping assist e-commerce in understanding their users on a deeper level every day.

Key Insight

User research methods and tools could prove helpful for manufacturing designers trying to make Industry 4.0 systems more usable.

Personas and storyboards provide developers and designers insight into the users that will be using their system. When the actual users of the system are used to develop personas, designers can dissolve stereotypes and assumptions of operators, equipment engineers, or anyone else who will be required to interact with the manufacturing and automation systems. Personas also encourage developers to break the habit of developing systems based on their own conceptual model of how the system should work.

Persona of Amy, the plant operator

Storyboards give designers and developers a better understanding of the work activities that users perform in an Industry 4.0 system. With a better understanding of a user’s work activities and the potential frustrations that arise during the user’s workflow, systems can be designed within the context of the user and do more than input, process, and output data.

The simplest method of creating egocentric attention-interaction documentation for Industry 4.0 is by observing and recording users as they interact with human-machine interfaces. While developers can learn about the context of situations where users are having to interact with the HMI’s, documentation alone does not provide enough information about a user’s awareness or comprehension of the way information is visualized on the screen. Wearable technologies, like goggles and glasses, embedded with eye-tracking hardware can be used to monitor a user’s attentional focus on certain screens or equipment.

Plant operators working with Internet of Things

In addition to user observation, studies with eye-tracking have discovered scenarios where novices and expert users focus their attention on different parts of manufacturing equipment in similar situations. Information like this can be used to guide a novice user’s eyes to the right components on automation contraptions so that they share a similar pattern of observation to expert users. Extended durations of focus on human-computer interfaces could also uncover confounding information or incomprehensible data on the screen for users.

Vision & Approach

Plant simulation software allows manufacturing designers to create digital models of production facilities to optimize existing workflow or plan out new plant floors virtually. While it is common for developers of manufacturing systems to simulate automation processes before breaking ground on the plant floor, more recent technologies can facilitate the improvement of user experience in production. Incorporating embodied interaction into plant simulation opens up opportunities to involve users early in the design process of Industry 4.0 systems and to continuously improve user work activities well into production.

Plant simulation

As technology and the Internet of Things continue to integrate deeper into all aspects of life inside and outside of factory floors, designers and developers will need to ensure that these highly interactive systems are easy to use and offer experiences that meet the needs of users. As manufacturers push for smarter factories into and beyond the age of Industry 4.0, they will need to rely more on usability experts and user-centered design processes. Some of these potential technologies like augmented reality and virtual reality are only just a couple of concepts that can improve a user's experience on the plant floor.

Conceptual Design

Augmented Reality

Manufacturers are forced to keep up with the constant changes to products influenced by growing customer demand and shorter product lifecycles. As a result, rearranging and reconfiguring production environments are very costly and time-consuming tasks. While plant simulators assist manufacturing developers in the planning process, designers must remember that these simulations are only virtual. Simulations might not work as planned in a real world environment. Technologies like augmented reality can bring simulated design to the plant floor, where systems can be tested in real-world contexts against real users.

Training on the plant floor

Virtual Reality

Using virtual reality in Industry 4.0 systems could prove useful to onboarding new employees in smart factories. Virtual reality software can reduce costs of training and improve safety on the plant floor. The operation of complex machinery by novice users, with little to no training, can have dangerous and expensive repercussions if used improperly.

Virtual reality software can be used in place of an actual production environment for users to learn how to control complex systems and machinery that could result in costly repair and downtime if operated improperly. In virtual reality, novice operators do not run the risk of damaging expensive machinery. When a user in training makes a mistake in a virtual reality environment, simulations can be easily reset for users to try again. Training software can also cut costs on the time and space of setting up extensive training facilities full of expensive manufacturing equipment.

Implications for Design

Human-machine interfaces of today are often overloaded with data that may or may not be relevant to the user’s tasks at hand. Augmented reality has the ability to improve the exposure of relevant information to users within the context that they need it. Wearables that project models and data within the user’s line of sight can offer information that directs their goals. For example, augmented reality glasses could provide various decisions that a user can make throughout their tasks, and guide them with the right scent of information to their goal.

Virtual reality training software extends learnability to novice users. Since there is less risk when mistakes are made, users are more likely to retain what they have learned. Trainees have more control to undo and correct their errors in simulated software than on the plant floor, and prevention and recovery techniques that users learn in the virtual reality training software can carry over to their work environment easily. Repetition and precision in practice reinforce learning for the user. Repeated training is also more affordable via virtual reality software than it is with valuable manufacturing resources and equipment.

Design Considerations

Key Insight

Despite the world of opportunity that augmented reality can provide both in and out of smart factories, there are limitations with current implementations of augmented reality.

First, augmented reality technologies only seem to work best in homogenous environments, where lighting and environmental conditions are relatively the same. Lighting conditions need to be stable enough that users can read digital text and instructions related to their work activities that are projected into their work environment. Second, users must always be connected to systems where their information is stored. Wireless technologies are not the most reliable means of communicating data to users who are constantly on the move and require a lot of bandwidth. Modern-day augmented reality systems are often limited to a stationary location in a smart factory.

Wireless technology on the plant floor

Users must also not be hindered by the equipment and wearable technology that is necessary to add augmented reality to their work activities. Equipment, such as glasses or goggles, that block a user’s vision could pose a safety hazard in certain environments. Equipment worn on the body may impede movement and limit the degree to which users have control while operating.

Virtual reality training also provides the benefit of safety. If incorrect operations are performed, users are not at risk of hurting themselves or others. Simulation software can warn users of the dangers within their mistakes and educate them on performing operations correctly. Virtual reality training also offers the advantage of detailed performance metrics for novice operators. Training software can also be used to provide new or more difficult challenges to users that may be hard to replicate safely in a production environment. The software can track the length of time and accuracy of tasks that operators are being trained on, and provide insights into common mistakes that users make during operations.

Design Artifacts

Canvas Wireframe

The canvas wireframe provides a guideline for developers on where to place controls, functionality, and interaction. On manufacturing executions systems, it is typical for buttons and interactions to change based on the context of the screen. Providing a consistent layout can help users know where to locate certain actions on the screen.

Canvas wireframe example

Wireframe Using Justinmind

The following wireframe was built in Justinind Prototyper following the canvas wireframe layout.

Wireframe example

Responsive Prototypes

All pages of the web application interface require ease of access on tablet devices. While all pages are viewable on a tablet, only specific pages frequently used in mobile contexts were designed with responsiveness.

Tablet prototype portrait Tablet prototype landscape Desktop prototype

High Fidelity Prototypes

The following prototypes were built in Sketch, with click-through navigation handled using InVision.

High fidelity prototype of order status page

Knowing that the system was required to be developed using AngularJS, I gained design inspiration from Teradata's Covalent Angular 2.0 Material Design platform.

High fidelity prototype of unit overview


Illustrations by Bobboz, licensed through Adobe Stock

Scientific Journals

Extending the Design Space in Industrial Manufacturing Through Mobile Projection

Augmented Reality for Manufacturing Planning

Development an Interactive VR Training for CNC Machining

Industry References

Empowering User Interfaces for Industry 4.0

Manufacturing Simulation and Validation


The Design of Everyday Things, Don Norman

Designing with the Mind in Mind, Jeff Johnson

River Valley Nonprofits