Vol.9 Eliminating "blind spots" from construction machinery. Ensuring safety through the Field View Monitor

The operator's cabin on most work-site construction machinery is located on the front left side of the unit. This results in a "blind spot" to the rear and right, one that is significantly worse than a regular passenger vehicle. In recent years, more of these machinery models come equipped with CCD cameras as well as a LCD monitor inside the cabin, allowing the operator to have a view of the periphery. Sumitomo Heavy Industries' "Field View Monitor (FVM) System" utilizes SHI's proprietary image processing technology to successfully transmit images that are more real and intuitive to the machinery operator. Through this, it is hoped that a reduction in construction accidents arising from insufficient vision can be achieved.

Weaknesses to the right and rear. Expanding the vision of the construction machinery operator.

Although work place accidents in the construction industry have reduced every year from its peak in 1978, there were still approximately 20,000 accidents resulting in death or injury in 2012. Of this number, 367 accidents resulted in death. The primary cause of these deaths is falling objects. However, the second most frequent cause of death is accidents from the use of construction machinery. Of these deaths, half are related to people being hit by or being cornered and tangled into construction machinery. The lack of peripheral verification by the operator is considered to be one factor behind these deaths.
For example, the hydraulic excavator, which is a typical type of construction machinery, comes with an operator's cabin that is located on the front-left side of the unit. This results in a "blind spot" to the rear and right, one that is significantly worse than a regular passenger vehicle. Of course, rear-view mirrors are installed within the cabin but operators need to make their best effort to verify their surroundings to ensure safety. More recent models have replaced the rear-view mirror with CCD cameras as well as LCD monitors inside the cabin, allowing the operator to have a view of the periphery. However, these peripheral monitoring systems that only show a simple camera image to the operator have several associated usability faults.

Magnify Image


By combining three separate images, an aerial view can be displayed

One camera cannot cover the area that is considered to be an operator's blind spot. In fact, to ensure full safety, the operator has to be able to see two images; one that covers the rear and one that covers the right. However, switching images between the two cameras can be cumbersome, and even if the two images are displayed on one screen, it is difficult for the operator to intuitively understand the positioning of surrounding people and objects.
To cover a wide area, a simple solution is to use a wide-angle camera. Unfortunately, this creates another problem as image distortion specific to a wide-angle lens results in the operator not being able to judge distances that well. SHI's FVM (Field View Monitor) System overcomes these weaknesses of traditional peripheral monitoring systems and realizes a higher-level of peripheral safety verification support for the operator.
The FVM system was first sold in 2011 as an option to the "LEGEST" series of hydraulic excavators that was jointly developed by Sumitomo Heavy Industries Limited and Sumitomo (S.H.I.) Construction Machinery Co., Ltd. As a result of the good reviews received from users, the FVM system now comes standard on all LEGEST series models. This technology has also been registered on the New Technology Information System (NETIS) of the Ministry of Land, Infrastructure, Transport and Tourism and is now regarded as one of the benchmark safety features in the construction industry.
The FVM System takes data from three on-board CCD cameras that capture images from the machine's rear, right and left, and instantaneously processes and combines it into one image that enables the operator to have a 270-degree view of the surroundings. In addition, the image is shown as if taken from above the machine, akin to a bird's eye view, which makes it easy to verify the surroundings of the machine in one quick view.
Moreover, in response to the construction industry's call to be able to "use the system in low light environments such as night time construction and within tunnels", the FVM system was equipped with a highly sensitive camera. By doing so, sufficient monitoring can now be carried out in any low light environment. Yoshihisa Kiyota, Senior Engineer Research Solution Engineering Dept at the Technology Research Center of Sumitomo Heavy Industries Limited who helped develop the FVM System believes that the System will be a success if it can "reduce the number of deaths relating to the use of construction machinery even a little bit by providing the operator with a wider "bird's eye view" of the equipment's surroundings".


Development of proprietary software for the on-board camera

The most significant challenge that Mr. Kiyota and his fellow engineers faced during the development of the FVM System was the technology relating to the processing and combining the camera images. In order to carry out real-time processing of the high volume of images coming from the three on-board cameras, it was necessary to design a new high-speed logic circuit board. Moreover, the design had to be one that is low cost to ensure that the system will be utilized by many. In fact, it took almost two years from initial development to product commercialization (including achieving cost optimization).
There are even special characteristics in the algorithms and software that is used for image processing. If a simple approach to processing and splicing multiple camera images is taken, it was discovered that the overlapping portions of the images would sometimes "disappear". The FVM System utilizes a unique algorithm to process images and provides a "bird's eye view" of the surroundings without any partial elimination of overlapping images.
The way in which this is done is by taking the images from the camera and first converting them from a two-dimensional setting to a three-dimensional setting to change the perspective. The images are then spliced to form a single image that does not look awkward in the overlapping areas. As the positioning of the camera changes the way in which an image is viewed, proprietary software to calculate this change was developed and a patent was approved. During its development, technology from other parts of Sumitomo Heavy Industries such as the glass substrate positioning equipment used in semiconductor manufacturing and the oven monitoring camera within a coke oven was effectively cross-utilized.
Another difficulty which the engineers faced was the development of a compact on-board monitor within the cabin that poses its own challenges because of the significant amount of vibration present in any piece of construction machinery. It took significant time to develop a user interface that was easy to understand so that the operator could use it without any confusion.
Actual operators of construction machinery with the on-board FVM System have commented that "it not only allows me to see objects that are close but also objects moving closer from a distance, which is good. It has also become easier to monitor not just the front but also all directions."
"Safety systems for construction machinery that utilize ICT will become more important in the future. At the same time, these systems can be adopted in any heavy machinery that humans operate, including cargo handling equipment and cranes. The need to further develop this type of technology is ever-increasing as demand rises from both domestic and overseas users."
This is how Hidehiko Kato (Maneger, Electric Control Sect R&D Dept Engineering Div, Sumitomo (S.H.I.) Construction Machinery Co., Ltd.), who jointly developed the system, expresses his ambitions when talking about the future development of this market.


"The Field View Monitor ensures a safer work site environment"

Previously when using construction machinery on a work site, we heard comments such as "I can’t see the rear" and "I'm not confident because the rear-view mirror is ineffective". Since introducing construction machinery equipped with the FVM system on the work site, we have heard operators comment that they are more confident in their work because they are able to carry out safety verifications with their own eye using the monitor. This is the most important thing for our company. In addition, with the V-registration of the System on NETIS, we are getting more praise from our customers. Through the utilization of the FVM System on every job-site, we are able to exhibit originality and ingenuity and as a result, we are evaluated higher in the overall evaluation bidding method. Needless to say, we hope that our ability to go into jobs with higher points will lead to more opportunities in the future to win new orders.

President and CEO Yutaka Nonaka

"FVM" is a trademark of Sumitomo Heavy Industries, Ltd.