For over a century the process of capturing electronic images has remained virtually unchanged, with each pixel in the image being a discrete sample of the spatial and temporal continuum being photographed. In a conventional camera, the only recorded information for each pixel is position and colour.  The fact is that captured images remain very limited samples of the scene they represent, only giving a 2D impression of the 3D spatial build-up of the scene. This primitive process of capture is the cause of problems in a myriad of applications, ranging from the film and television post-production to  virtual reality, or object-based video formats and  industrial inspection. Although much effort has been put into surmounting these problems, all these approaches are based on the estimation of data that is simply not included in the discrete samples provided by digital images, and so are limited in the quality they can provide. The capture of additional data is a step forward to address these problems.

The study of enhanced image segmentation techniques is critical given that image segmentation is an ubiquitous operation, spanning a large set of applications, and that subsequent processes rely heavily on its performance. The availability of additional data that is synchronous with colour information should significantly boost the performance of current state of the art techniques, fostering a whole class of new applications.

This dissertation focuses on the study of image segmentation techniques assisted by metadata. In particular, the use of depth and motion information to improve the quality of segmentations of colour images is investigated. Starting from primitive fusion approaches for colour and depth, the work evolves to richer techniques, with the joint modelling of colour and depth information; the  the method is further extended to conveniently handle the noise typical of real-life depth data. Next, an alternative approach is presented, where depth is used for providing a crude identification of the objects in the image and colour is then used to attain high-quality borders, performing a guided image segmentation starting from the crude seeds obtained from the depth information. The study is concluded with the integration of motion information in the most promising fusion techniques.

Because a fair judgment of any new image segmentation algorithm needs a fair comparison metric, a preliminary study on metrics for comparing image segmentations was conducted in the first place. The poor performance of existing measures led to an exhaustive investigation on new solutions, culminating on the rediscover of the metrics based on the intersection-graph of two segmentations and on their introduction to the image engineering community for the first time.

In the numerous experiments that are reported, experimental evidence of the adequacy of the metrics and enhanced images segmentation techniques is provided.

Predictive learning has traditionally been a standard inductive learning, where different subproblem formulations have been identified. One of the most representative is classification, consisting on the estimation of a mapping from the feature space into a finite class space. Depending on the cardinality of the finite class space we are left with binary or multiclass classification problems. Finally, the presence or absence or a “natural” order among classes will separate nominal from ordinal problems.

Although two-class and nominal classification problems have been dissected in the literature, the ordinal sibling has not yet received a lot of attention, even with many learning problems involving classifying examples into classes which have a natural order. Scenarios in which it is natural to rank instances occur in many fields, such as information retrieval, collaborative filtering, econometric modeling and natural sciences.

Conventional methods for nominal classes or for regression problems could be employed to solve ordinal data problems; however, the use of techniques designed specifically for ordered classes yields simpler classifiers, making it easier to interpret the factors that are being used to discriminate among classes, and generalises better. Although the ordinal formulation seems conceptually simpler than nominal, some technical difficulties to incorporate in the algorithms this piece of additional information – the order – may explain the widespread use of conventional methods to tackle the ordinal data problem. This dissertation addresses this void by proposing a nonparametric procedure for the classification of ordinal data based on the extension of the original dataset with additional variables, reducing the classification task to the well-known two-class problem. This framework unifies two well-known approaches for the classification of ordinal categorical data, the minimum margin principle and the generic approach by Frank and Hall. It also presents a probabilistic interpretation for the neural network model. A second novel model, the unimodal model, is also introduced and a parametric version is mapped into neural networks. Several case studies are presented to assert the validity of the proposed models.