Research Projects

Model Selection
Large-Scale 3D Vision
Sensor Data Fusion in Brake-by-Wire Technology
Robotic Aircraft
Optical Flow Calculation
Motion Segmentation
Range Segmentation

Model Selection

Problems in Model Selection for Range Segmentation
The available information theoretic model selection criteria are based on the assumptions that noise is very small and the data size is large enough. However, in visual data segmentation algorithms, one usually deals with noisy regions of data, some of which are small localised regions that are to be extended later. As a result, neither of the above assumptions is usually valid.

The other crucial problem is that almost all the information-theoretic model selection criteria measure the complexity of a model only by its number of parameters1. Hence, if there are different models with the same number of parameters (i.e. same complexity), there will be some difficulty in selecting between those models. In fact, the available model selection information-theoretic model selection criteria assume the set of candidate models to be nested, whereas, in many applications, such as range segmentation of curved objects, the set of candidate models may consist of non-nested models.

Finally, all the available model selection methods have their roots in statistics, the physical and geometrical characteristics of the problem have been ignored.

As a result none of the existing model selection criteria is capable of identifying the true underlying model for visual data in motion segmentation and range segmentation applications.

Surface Selection Criterion
To overcome the above problems in model selection, we proposes to view the sum of bending and twisting energies of a surface as a measure of the surfaces roughness and the sum of squared residuals as a measure of fidelity to the true data. There should be an acceptable compromise between these two factors. To formulate the proposed Surface model Selection Criterion (SSC), different points of a surface are viewed as hypothetical springs constraining the surface as shown in the below figure.

If the surface has little stiffness, then the surface bends and twists to meet the points. Hence, the surface fits itself to noise and the sum of squared residuals between the range measurements and their associated points on the surface will be small. This bending and twisting in turn increases the amount of strain energy accumulated by the surface.

As one may expect, increasing the number of parameters of a surface leads to larger bending and twisting energies as the surface has more degrees of freedom and consequently the surface can be fitted to the data by bending and twisting itself so that a closer fit to the measured data results. This can be inferred from the bending energy formula Equation 1). However, the higher the number of parameters for a surface model, the less the sum of squared residuals. For instance, in the extreme case, if the number of parameters is equal to the number of data points (which are used in the fitting process), then the sum of squared residuals will be zero whereas its sum of energies will be maximised.

If a plate is bent by a uniformly distributed bending moment so that the xy and yz planes are the principal planes of the deflected surface, then the strain energy (for bending and twisting) of the plate can be expressed as:

(1)

where D is the flexural rigidity of the surface and ν is Poisson’s ratio (ν should be very small because in real world-objects the twisting energy in comparison with the bending energy is small). In all the experiments reported in this thesis, n is assumed to be 0.01. As our experiments show, the performance of SSC is not sensitive to the small variation of n . The strain energy is computed as a measure of complexity of the model.

In order to scale the strain energy, it has been divided by the strain energy of the model with the highest number of parameters (E_max). Therefore, D has been eliminated from the following formulations.

To establish the trade-off between the sum of squared residuals and the strain energy E_{Bending+Twist}, a function SSC is defined such that:

where δ is the scale of noise for the highest surface (the surface with the highest number of parameters). The reason for using the scale of noise of the highest surface (as explained by Kanatani [1]) is that the scale of noise for the correct model and the scale of noise of the higher order models (higher than the correct model) must be close for the fitting to be meaningful. Therefore, it is the best estimation of the true scale of noise, which is available at this stage.

An accurate estimate of the scale of noise δcan be computed by where N is the number of data points and P is the number of parameters of the highest surface. Use of this formula for the scale of noise can be justified by the fact that if the model is correct, then is subjected to a χ^<2 distribution with N-P degrees of freedom. The energy term has been multiplied by the number of parameters P in order to penalise the choice of a higher order (than necessary) model. Such a simple measure produces good discrimination and improves the accuracy of the model selection criterion.

Having devised a reasonable compromise between fidelity to data and the complexity of the model, the model selection task is then reduced to choosing the surface that has the minimum value of SSC.

Large-Scale 3D Vision

This research project is intended to provide the necessary resources for comprehensive large‑scale three‑dimensional mapping of man‑made environments. Achieving a complete 3‑D digital representation of the surrounding environment has, over the years, received substantial attention due to the plentifulness of its applications.

To achieve the goal of large-scale 3-D environmental mapping, a 3-D range scanner has been developed. The equipment is capable of producing range maps of far away objects (up to 300 meters) with relatively wide field-of-view. More importantly, range and intensity images produced by this equipment can be fused to have a complete set of (x, y, z, R, G, B) sextuplets of the scene.

The range scanner system is based upon a distance meter developed by MDL and a programmable pan-tilt unit manufactured by Directed Perception. The rangefinder is capable of producing distance measurement of 7-300 meters with repetition rate of 1000 Hz. The typical accuracy of the machine is 300 mm, which is 0.1% of the maximum measured distance.

In this project we mainly focus on development of an accurate and calibrated measurement device, which its measurements are sufficiently accurate for a wide range of scientific endeavours such as volumetric and surface recording of historical sites, virtual reality in environment building and modelling and architectural reproduction.

Following figures show an example of our experimental result. Range image, taken by range scanner device (left) and intensity image (middle), captured by video camera, has fused (right) to present a rich data set of the scene.

Sensor Data Fusion in Brake-by-Wire Technology

Brake-by-wire is a frontier technology that will allow many braking functions to switch to electronic actuation. It will lead to more effective and safe braking systems, elimination of hydraulic technology, release of space and reduction of maintenance. By the end of this decade, most international car manufacturers are envisaged to have adopted brake-by-wire systems across their range of passenger cars. A large variety of sensors are utilised in an Electro-Mechanical Braking (EMB) system and therefore their consistent operation is vital for the functionality of such system. To achieve a high level of coherency amongst such a large collection of sensors (mandated by the safety requirement of a brake system), the use of sophisticated data fusion techniques is unavoidable. Data fusion theory is generally considered the most appropriate tool to generate reliable and accurate information by virtue of combining all the available data. During the last two decades, theory of data fusion has been broadly developed and applied in many applications. Data fusion is generally defined as “synergistic combination of the information, provided by multiple sensory devices or multiple sources, to assist in the accomplishment of a task by a multi-sensor system”. Generally speaking, data fusion techniques are required in an EMB system for many important tasks that can be classified into the following three categories:

• Effective Utilisation of Redundant Information

An EMB, by nature, is a safety critical system and therefore fault tolerance is a vitally important characteristic of this system. As a result, EMB systems are designed in such way that many of its essential information would be derived from a variety of sources (sensors) and be handled by more than the bare necessity hardware. Three main types of redundancy usually exist in an EMB system:

- Redundant sensors in safety-critical components such as the brake pedal

- Redundant copies of some signals that are of particular safety importance such as displacement and force measurements of the brake pedal

- Redundant hardware to perform important processing tasks such as central controllers

In order to utilise the existing redundancy, voting algorithms and probability‑based fusion techniques need to be evaluated, modified and adopted to meet the stringent requirements of an EMB system. Reliability, fault tolerance and accuracy are the main targeted outcomes of the fusion techniques that are developed especially for redundancy resolution inside an EMB system.

• Estimation and Identification

In a typical EMB system, there are three categories of estimation and identification tasks that challenge the existing sensor data fusion techniques. Those categories are:

- Accurate and reliable estimation of environmental parameters (e.g. friction coefficient between the tyre and the road) by integration of multiple sensors or sources of data

-Parametric and non-parametric identification of unknown dynamics in the system (e.g. calliper dynamics)

-Estimation of system states or variables by fusing multi-source information (e.g. vehicle state estimation, estimation of the zero crossing point on the actuator characteristic curve and clamp force estimation).

• Control

The central controllers in the system are mainly responsible for high level braking functions such as four-wheel proportioning, automatic hill hold, antilock brake control, traction control, vehicle stability control and so on. Each of these control tasks must be accomplished based on the data received from multiple sensors and sources. Hence, robust and accurate multisensor fusion methods are required to be developed if high level control tasks of the EMB system are to be successfully executed.

We have been working on different fusion problems that have arisen during the design and testing of an EMB system. A new voting algorithm for fusion of the brake pedal sensors, a novel predictive filter for compensation of the missing samples of sensor data (filed as an Australian patent), and a new angle tracking observer for estimation of the position and speed of brushless DC motors based on a resolver sinusoidal signals (filed as another Australian patent) have been developed sofar.

Robotic Aircraft

The main goal of this project is to develop reliable computer vision techniques to be utilised in the control of low altitude manoeuvres for autonomous small-scale aircraft. In particular, we intend to focus our attention on recovering the relative altitude of an aircraft by using the terrain texture seen from an on-board camera. We aim to develop techniques that would work reliably on almost any terrain and weather condition. We are also committed to provide evaluation techniques capable of verifying our estimates (as they are generated) and associate a probabilistic reliability measure to the sensor output.

Click here to watch a clip which is taken by the aircraft.

The model plane	The Engine Mounting
The Camera Support

Optical Flow Calculation

Differential optic flow techniques try to relate local changes in image intensity (expressed as spatial and temporal derivatives of the image brightness function) to the optic flow. Differential techniques usually perform faster than image matching or phase based techniques and lead to a simple set of linear equations. The fundamental assumption behind all the optic flow techniques is the fact that the brightness intensity function of a moving object remains approximately constant at least for short duration of time Optical Flow is the motion of objects on the image plane as objects (of the camera) moves. This has applications in robotics and in video coding technology. Of particular recent interest, is the study of motion segmentation - how to break an image up into the parts corresponding to objects undergoing different motions. A significant factor in this research, of late, is the use of Robust Statistics. 
A major recent advance was the use of robust statistics to calculate the flow in small patches (with which the flow is assumed to be uniform) [Bab-Hadiashr & Suter 1998]. 

A (LARGE!) mpeg file shows a screen capture of our optic flow demonstration code - 

30Mb Mpeg file (created by Dr. David Suter) 

Motion Segmentation

Segmentation is a vital task in many computer vision problems. In our work we have concentrated upon applying our segmentation methods to the segmentation of optical flow (motion) and range data. However, our methods are sufficiently general and robust as to apply to virtually any segmentation problem. Our focus has shifted to motion segmentation to overcome the assumption that the flow is uniform in certain patches and on the selection of motion models. Major results in this direction can be found in [Bab-Hadiashr & Suter 1999]. 
Following figures show an example of our approach to the motion segmentation problem . 

Otte sequence and its measured flow field

Segmentation Results - First and Last Stage

A (LARGE!) mpeg file shows a sequence segmentation based upon motion. 

17Mb Mpeg file (created by Dr. David Suter)

Range Segmentation

Range data are usually clean and accurate and although they are made of multi-structures (at least in our examples), they do not have many impulsive outliers. Thus, the range data segmentation task probably does not challenge our methods as greatly as the motion segmentation task. However, clearly there is still some utility in employing robust segmentation methods on range data. Following figures show an example of our approach to range data segmentation (for detail, see Bab-Hadiashr & Suter 1999).

Intensity image, original and segmented range data