What Is Sensor Fusion?
You pull up to a stop sign. Your office is only a block away, straight ahead. You look to your left and to your right for any cross traffic. There are no cars coming, but you notice a pedestrian entering the crosswalk. After the pedestrian crosses the road, you proceed straight ahead through the intersection. This is something we as humans do many times during our week and likely take for granted. It comes naturally to us without giving it much thought – you may have even been listening to the radio and eating food from your favorite fast-food restaurant at the same time but still managed to cross the road safely. Our ability to take in the information from our senses and to choose a course of action based on that information can be considered “sensor fusion.” Sensor fusion involves gathering information from multiple sensors at the same time and using that data to determine what action(s) to take.
Single-Sensor Systems: Simpler Automation Examples
Simple systems, ones that utilize a single sensor, have been around for a long time. Automatic light switches use a motion sensor to determine if someone has entered or exited the room to turn the light on or off. Household thermostats use a temperature sensor to determine if heating or cooling should be turned on. Simple security cameras use a single image sensor to detect movement to trigger video recording. These types of systems don’t require multiple sensors to perform their duty.
Why Simple Sensors Aren’t Enough for Complex Tasks
But what if you want to build, for example, a self-driving car? Using a simple system to handle the scenario described above of crossing an intersection won’t provide enough information to make accurate and safe decisions. If you only have one fixed-direction camera, you would only be able to see traffic, pedestrians, or other obstacles from one direction.
Multi-Sensor Systems for Advanced Applications
Instead, you’re going to need multiple sources of information. You’re going to need multiple cameras to perform object detection, street sign recognition, lane marker detection, color recognition, etc. from multiple directions. You’ll need radar sensors to determine how fast other vehicles are moving (think adaptive cruise control). You’ll need LiDAR to map your surroundings. You might want to use ultrasound to assist with parking. And you’re going to need to receive updated information from the sensors, process it, and make decisions as quickly as possible. Safety critical systems, such as Advanced Driver Assistance Systems (ADAS), need to make decisions within milliseconds of receiving the raw data.
Making Sense of the Data: Signal Processing in Sensor Fusion
To extract meaningful insights, the raw data must be organized and interpreted effectively. This is where signal processing algorithms come into play. You’re going to need to filter out the noise, possibly with Fourier Transforms. You’re going to need an algorithm such as Kalman Filtering to estimate the changing system states. To make sure you have the best estimate based on data from all the sensors, you can use Consensus Filtering. Your system is also going to need to learn from the information it receives over time, which is where Bayesian Inference and Neural Networks come into play. This is just a small sampling of the signal processing algorithms you can use and there are plenty more to choose from for your specific application.
Challenges in Implementing Sensor Fusion
Sensor fusion doesn’t come without its challenges. As the saying goes, “if it were easy, everyone would do it.” Some of the challenges you might face are as follows:
- Not all sensors are created equal, so the accuracy and precision of the data can vary. It is important to choose sensors based on the specifications and tolerances for your needs.
- Even the best sensors are susceptible to noise, erroneous data, and other uncertainties, so it’s important to understand how to use the complex signal processing algorithms.
- The raw data from the sensors is coming in very quickly and needs to be handled in real time, often with limited resources found in many embedded systems, so you need to balance accurate fusion results with efficiency in your calculations.
- Sensor fusion, as well as artificial intelligence and machine learning, are rapidly changing technologies, so you’ll want to design your system so that it can handle updates when they become available.
Future of Sensor Fusion in Embedded Systems
Sensor fusion is going to continue to play a big role in embedded systems. There are still uses for single-sensor systems, but many of today’s systems are much smarter and more complex. GPS navigation devices, factory robots, medical devices, traffic signals, and fitness trackers all require sensor fusion to do their jobs properly, accurately, and safely. As progression is made in technologies like sensor fusion, artificial intelligence, and machine learning, embedded systems will continue getting better at taking in information and interpreting their surroundings with more human-like ability and accuracy. Hopefully this high-level overview gives you a better understanding of sensor fusion and how it might fit into your current and future project needs.
If you’re working on a project that involves sensor fusion – or considering how to enhance your embedded systems with more advanced sensing capabilities – DISTek can help. Our engineers have deep expertise in embedded software and real-time systems, and we’re ready to support your development from concept to implementation. Reach out to our sales team to learn how we can assist you in bringing your ideas to life.
