Skip to main content

About this book

This book provides a concise introduction to Pervasive Computing, otherwise known as Internet of Things (IoT) and Ubiquitous Computing (Ubicomp) which addresses the seamless integration of computing systems within everyday objects. By introducing the core topics and exploring assistive pervasive systems which infer their context through pattern recognition, the author provides readers with a gentle yet robust foundation of knowledge to this growing field of research.

The author explores a range of topics including data acquisition, signal processing, control theory, machine learning and system engineering explaining, with the use of simple mathematical concepts, the core principles underlying pervasive computing systems. Real-life examples are applied throughout, including self-driving cars, automatic insulin pumps, smart homes, and social robotic companions, with each chapter accompanied by a set of exercises for the reader. Practical tutorials are also available to guide enthusiastic readers through the process of building a smart system using cameras, microphones and robotic kits. Due to the power of MATLAB™, this can be achieved with no previous programming or robotics experience.

Although Pervasive Computing is primarily for undergraduate students, the book is accessible to a wider audience of researchers and designers who are interested in exploring pervasive computing further.

Table of Contents

Chapter 1. Introduction

Pervasive computing, also known as Internet of Things , or Ubiquitous Computing , describes the emerging trend of seamlessly integrating computing into the everyday physical world. Examples of pervasive computing systems include: self-driving cars , smart homes , navigation systems for disabled people, and environmental monitoring systems. In this chapter, we will take a first glance at such systems and identify their properties and basic components. An essential feature of these systems is their context-awareness, meaning that they determine circumstances (time, location, ambient temperature, emotions, seismic activity, etc.). In order to extract this context, a pervasive computing system must sense signals from the environment and then process them. Consequently, based on the inferred context and an internal process model, the pervasive computing system takes intelligent decisions and acts upon the same environment. Context inferring, reasoning, and decision making are the tasks of a software agent , or controller . In many cases, more computer systems work together behind the scenes, exchanging information in a transparent way, in order to deliver a service to the user. To summarize, a generic pervasive computing system consists of sensors, actuators, software agents (controllers), and communication modules. Its core properties are context-awareness, implicit human–computer interaction, seamless networking, autonomy, and intelligence. The highest ideal of making computing naturally embedded in our lives cannot, however, be achieved by merely solving technical problems. Acceptance and trust of pervasive computing also requires solutions to legal, ethical and usability challenges.
Natalia Silvis-Cividjian

Chapter 2. Signals

We all send and receive signals . A letter or a phone call, a raised hand, a hunger cry—signals are our information carriers. Pervasive computing systems also send signals (to wheels, loudspeakers, or displays), and sense signals (like speech , images, heart activity, or temperature). These signals are by nature continuous, meaning that their level is known always and everywhere, and they are analog, meaning that this level is an infinite-precision real number. The problem is that these properties are in conflict with the discrete and digital character of computers. The price the real-world signals have to pay to be admitted and processed by a computer, is to undergo a process called digitization . Digitization happens in two stages. First, sampling periodically measures the signal’s level, and produces a finite-length sequence of samples of infinite precision. Next, quantization converts these samples into a sequence of finite-precision numbers. Digitization is the task of an electronic device, called analog-to-digital converter (ADC). The reverse process, needed by a computer system to decipher the carried message and send signals back to the real world, is called reconstruction. Reconstruction happens in another electronic device, called digital-to-analog converter (DAC) . High-quality digitization and reconstruction are needed, in order to preserve the information the signal is carrying. This chapter is an introduction to signals and their representation in pervasive computing systems, with a focus on audio and video signals.
Natalia Silvis-Cividjian

Chapter 3. Control Systems

Pervasive computing systems are built around a controller that coordinates actuation with sensing. But which principles govern this coordination process? Is each system unique, applying its own principles? Or do all the systems have something in common, so that we can learn from successful examples how to build our own system? In this chapter, we will try to demonstrate that the latter is indeed the case. The range of sensors that can be used to discover context in pervasive computing is impressive. Luckily, most of these sensors are based on the same principle, saying that a physical quantity variation results in an electrical voltage at the terminals of the sensor. This allows us to restrict the scope here only to sensors that measure temperature, light, touch, and distance. The story starts with the analog voltage produced by a sensor and carrying information about the environment. This signal is digitized and processed by a software controller that usually makes use of machine learning to extract context. In this chapter, two types of control principles will be discussed: deliberative, which is slow, but strong in strategical searching and planning and reactive, that offers a fast reaction instead of deep thinking. Finally, the controller makes decisions that affect back the environment. In order to execute these decisions, the controller sends out commands that eventually reach the actuators. Examples of actuators are electrical heaters, motors, light sources, and simple screens and displays. The goal of this chapter is to demonstrate the principles behind the most commonly used types of control systems.
Natalia Silvis-Cividjian

Chapter 4. Image Processing

In simple systems, the context can be easily extracted from the data coming directly from sensors, be it light intensity, distance, or temperature. This allows the right actuation decision to be easily taken, such as to open the curtains, drive straight, turn left, or start the heater. Unfortunately, context discovery from “raw” images is not as straightforward as that. The reason is that data produced by imaging sensors is only seldom perfect. Look at the image in Fig. 4.1. Suppose the goal is to simply count the miniature figures on it. You can say: “But even a child can do this!” Yes, that’s true. However, there is high chance that a computer will come up with a different and wrong answer. Can you see why? One reason is the shadows; they can make the computer algorithm believe that there are only five objects in the image rather than seven. And we did not even mention more difficult tasks, such as recognizing the figures in the image. Take a more serious, real-life example, and consider a traffic light recognition system in an autonomous car. Think about how the image taken by the vehicle camera will look, if there is mist, it is dark, and the red light is partially covered by dirt. The problem is that for different reasons, images are often corrupted by noise or blur; they suffer from uneven or poor illumination, poor contrast, or view obstruction. Hence, context information carried by image signals remains hidden, making classification and further reasoning impossible. An escape from this impasse is possible if one “cosmetically” enhances the image. This is called preprocessing, a process that encompasses a range of operations applied to a raw signal, with the purpose of extracting, against all odds, a reasonable context. This chapter will discuss different image preprocessing techniques, such as brightness and contrast optimization, thresholding, edge detection, noise filtering, and morphologic operations. The collateral damage that might be caused by these processing techniques is also exposed. Finally, automatic object counting is demonstrated using BLOB analysis.
Natalia Silvis-Cividjian

Chapter 5. Sound Processing

A pervasive computing system is considered extremely “cool” if it can recognize speech , understand a baby cry, identify daily activities from recorded noises , or name the composer of a famous music passage and, in other words, if it can find patterns in sound recordings. But prior to that, the system needs to know more about the recorded sound. Sound is a one-dimensional time-varying signal, created by a mechanical oscillation of pressure transmitted through air. Information is normally encoded in this signal through intensity or frequency patterns of variation. Measuring sound’s intensity is a relatively straightforward operation. However, extracting frequency information from a raw sound signal is far from being easy. First, because real-world sounds are a Tower of Babel of different frequencies and amplitudes. There is not one frequency, but a whole spectrum of frequencies involved in making a sound waveform. Second, because the raw sound captured by the system is often contaminated by predictable, like the 50/60 Hz power hum, or less predictable random noise , coming, for example, from an airplane flying in the neighborhood. Some processing of the recorded signal is therefore highly needed. This chapter starts by introducing Fourier analysis , a versatile and powerful mathematical tool, used to understand sound. Moreover, two types of digital filtering techniques are presented, without getting too far in mathematical details. These are (1) the moving average filters , suitable for suppression of random noise, and (2) the frequency-selective filters, used when a priori information about the spectral characteristics of noise is available. These techniques are not restricted to sound processing. They can be applied to any one-dimensional time-varying signal, such as acceleration , temperature, pressure, and bioelectrical signals (EEG, ECG, or EMG).
Natalia Silvis-Cividjian

Chapter 6. Classification

When set off for this journey, we promised to reveal the mysteries of pervasive computing systems that infer their context through pattern recognition , be it self-driving cars , voice-controlled wheelchairs, or smart homes . By now you should understand a lot about these systems; how they sense signals from environment, how they decipher simple information from these signals, and how they send control commands to affect back the environment. However, extracting a sensible context still remains a mystery; even the most accurate sound frequency spectrum cannot tell much about which words have been uttered, or by whom; and even the clearest image of the street scenery will not make a car understand that the approaching obstacle is a pedestrian. In case you start worrying about all this reading being in vain, please do not lose heart. We are almost there, yet we owe you one more story; this time, about pattern recognition , the last missing piece in the puzzle. Actually, all the previous chapters were preparation for this “big push.” Recognition of patterns in data happens mostly through classification, a process of organizing things into groups or classes. Pattern recognition answers questions, such as “Was the spoken command Right, Left, or Stop?” or “Is the detected obstacle in this image a pedestrian, or some other object?”. While for us, humans, pattern recognition is a trivial, ubiquitous task, computers find it extremely difficult. Machine learning algorithms are needed to empower computers with pattern recognition skills. Be aware that there is no single silver bullet algorithm that can solve all classification problems. The challenge is to find—for each particular case—the most realistic and acceptable solution. This is why this substantial chapter will expose you to various ideas of how to approach classification in general, and image and sound recognition in particular.
Natalia Silvis-Cividjian

Chapter 7. Systems Engineering

Previous chapters explained how pervasive computing systems work. As we are secretly hoping you would also like to build such a system, the story would not be complete without an attempt to show you how this is done in real life. We enter now the field of systems engineering, an interdisciplinary, team-based approach that means to enable the realization of successful systems. In a pervasive computing system, software and hardware are working together, in order to enable the desired functionality. For example, a smart car is a system of interconnected sensors (gyroscopes, odometers , cameras, and LIDARs ), actuators (displays, speakers, brakes, and motors) and software agents (map matching, shortest-path planning, traffic light recognition, pedestrians recognition, and automatic cruise control) that work together with other systems (GPS, traffic control systems, and digital map databases), with the purpose of a safe navigation of its passengers to their destination. In many pervasive computing systems, the physical devices, and even some of the software toolboxes, are ready-made. Therefore, the main challenge consists of developing the complete software application and integrating it with the existing hardware. This is why the main focus in this chapter will be on software. Software is nowadays the product of a team of software engineers. The first thing this software team does is, not as you might think, directly jump into programming. The team first writes down what the product should do, who and how will use it, how much time it will take to build it, what is its architecture, how it will be tested, what can go wrong, etc. This disciplined, engineering-like approach of “making” software, called software engineering , is proved to increase the project success and is benefic when it comes to maintainability and accident analysis. Software engineering is thus a part of system engineering , concerned with all aspects of software production, from system specification to maintenance. This chapter highlights the best practices in system engineering in general, and in software development process, in particular.
Natalia Silvis-Cividjian
Additional information