Frost formation in the heat exchangers of refrigeration systems is a well-documented phenomenon. This frost accumulation creates a thermally insulating barrier that can restrict, or even block, the airflow between fins, resulting in decreased efficiency and degradation of the food products. Several methods of frost detection and defrosting have been developed, although there is not an efficient mainstream method to measure and control frost formation. In previous works, the results of a small low-cost resistive sensor for frost detection were shown to be promising. This paper extends that research using computer vision to compare the results of this sensor with the frost formed on the heat exchanger, allowing for a better study of the sensor. This method allowed to trace and plot a frost formation curve of the sensor detected values.

Advances in Artificial Intelligence (AI) and Machine Learning (ML) obtained impressive breakthroughs and remarkable results in various problems. These advances can be largely attributed to deep learning algorithms, especially Convolutional Neural Networks (CNNs). The ever-growing success of CNNs is mainly due to the ingenuity and engineering efforts of human experts who have designed and optimized powerful neural network architectures, which obtained unprecedented results in a vast panoply of tasks. However, applying a ML method to a problem for which it has not been explicitly tailor-made usually leads to sub-optimal results, which in extreme cases can even lead to poor performances, thus hindering the sustainability of a system and the wide-spread application of ML by non-experts. Designing tailor-made CNNs for specific problems is a difficult task, as many design choices depend on each other. Thus, it became logical to automate this process by designing and developing automated Neural Architecture Search (NAS) methods. Architectures found with NAS achieve state-of-the-art performance in various tasks, outperforming human-designed networks. However, NAS methods still face several problems. Most heavily rely on human-defined assumptions constraining the search, such as the architecture’s outer-skeletons, number of layers, parameter heuristics, and search spaces. Common search spaces consist of repeatable modules (cells) instead of fully exploring the architecture’s search space by designing entire architectures (macro-search), which requires deep human expertise and restricts the search to pre-defined settings and narrows the exploration of new and diverse architectures by having forced rules. Also, considerable computation is still inherent to most NAS methods, and only a few can perform macro-search. In this thesis, we focused on proposing novel solutions to mitigate the problems mentioned above. First, we provide a comprehensive review of NAS components, methods, and benchmarks. For the latter, we conduct a study on operation importance to evaluate how the operation pool of search spaces influences the performance of generated architectures. Following, we studied how different neural networks behave for different classification problems and proposed two novel methods to improve upon existing neural networks with NAS by i) searching for a new classification head and ii) searching for a fusion method that allows performing multimodal classification. We then looked into improving the search cost of NAS methods by proposing a zero-proxy estimation strategy that scores architectures at initialization stage through the analysis of the Jacobian matrix and an evolutionary strategy that generates architectures by performing operation mutation and by leveraging the zero-cost proxy estimation to efficiently guide the search process. To further improve the capabilities of NAS methods, we extend the analysis of architectures at initialization stage by proposing a second zero-cost proxy method, which looks at the Neural Tangent Kernel of a generated architecture to infer its final performance if trained. With this, we also propose a novel search space that leverages large pre-trained feature extractors (CNNs) and forces the search only to a small middleware architecture that learns a downstream task. These two methods showed that large models can be efficiently leveraged to learn new tasks without requiring any fine-tuning or extensive computational resources. To further improve the search and memory costs of NAS methods, we proposed MANAS. This method frames NAS as a multi-agent optimization problem and uses independent agents that search for operations in a distributed manner. With MANAS, we showed that both the search cost and the memory resources can be heavily reduced while improving the final performance. Finally, to push NAS to less constrained search spaces and settings, we proposed LCMNAS, a NAS method that performs macrosearch without relying on pre-defined heuristics or bounded search spaces. LCMNAS introduces three components for the NAS pipeline: i) a method that leverages information about well-known architectures to autonomously generate complex search spaces based on weighted directed graphs with hidden properties, ii) an evolutionary search strategy that generates complete architectures from scratch, and iii) a mixed-performance estimation approach that combines information about architectures at initialization stage and lower fidelity estimates to infer their trainability and capacity to model complex functions. Results obtained by the proposed methods show that it is possible to improve NAS methods regarding search and memory costs, as well as computation requirements, while still obtaining state-of-the-art results. All proposed methods were evaluated in multiple search spaces and several data sets, showing improved performances while requiring only a fraction of previous NAS methods’ time and computation needs.

Combustion is currently the main method to produce energy in the world energy matrix. Through the burning of oil, coal and natural gas, mankind supplies its energy needs while migrating in a balanced way to renewable energy sources. From this context comes the need for efficient control of combustion, a key process in reducing the consequences of the greenhouse effect, and hence an accurate measurement of CO2 and O2 emissions in combustion processes. Due to the hostile process conditions such as high temperature, difficult access to the controlled area and the risk of explosion, the use of sensors on the flame is often impractical and image monitoring has proven to be a convenient alternative. In this paper, an inference device based on artificial neural networks that uses monochromatic flame images captured by a CCD (chargedcoupled device) camera is designed to estimate CO2 and O2 emissions in a natural gas furnace. The proposed method seems promising: it estimates without delay, unlike traditional gas analyzers, and with good accuracy on the tested equivalence ratios (root mean square error below 2%)

Ensuring safety and reliability in human–robot interaction (HRI) requires the timely detection of unexpected events that could lead to failures or unsafe behaviours. Anomaly detection therefore plays a crucial role in enabling robots to recognise and respond to deviations from normal operation during collaborative tasks. While reconstruction-based methods have been widely explored for anomaly detection in HRI, approaches that operate directly on semantic feature representations remain relatively unexplored. In this work, we propose an anomaly detection framework that transforms video streams into semantically meaningful feature vectors and performs reconstruction-based anomaly detection on these representations. The approach is further extended by incorporating additional modalities, including the robot’s internal sensor readings and scene graph representations, allowing the model to capture both external visual anomalies and internal robotic failures. To evaluate the framework, we collected a custom multimodal dataset comprising a simple pick-and-place robotic task performed under normal and anomalous conditions. Experimental results show that reconstruction based on vision-derived feature vectors is effective for identifying anomalies, and that integrating multiple modalities further improves detection performance. These findings highlight the advantages of multimodal feature reconstruction for achieving more robust and reliable anomaly detection in human–robot collaborative environments.

Visual impairment and blindness caused by infectious diseases has been greatly reduced, but increasing numbers of people are at risk of age-related visual impairment. Visual information is the basis for most navigational tasks, so visually impaired individuals are at disadvantage because appropriate information about the surrounding environment is not available. With the recent advances in inclusive technology it is possible to extend the support given to people with visual impairment during their mobility. In this context we propose a system, named SmartVision, whose global objective is to give blind users the ability to move around in unfamiliar environments, whether indoor or outdoor, through a user friendly interface. This paper is focused mainly in the development of the computer vision module of the SmartVision system.

Modern manufacturing increasingly depends on distributed, software-based systems in which computer vision and inference workloads must operate under strict latency, availability and resource constraints. In such environments, end-to-end responsiveness is determined not only by model inference, but also by the supporting messaging, storage and orchestration layers, whose design becomes a cross-cutting concern for both system architects and Development and Operations (DevOps) practitioners. This work was carried out during an internship at Twevo S.A., a Portuguese company that develops AI-powered edge computer vision solutions for the manufacturing sector, and is framed within the evolution of the company’s VIEXPAND AI platform. The objective was to study, design a distributed edge architecture capable of keeping end-to-end latency low while improving availability and lowering the cost of developing new components, while leveraging industry-standard DevOps workflows to ensure long-term maintainability. Four workstreams were implemented. A lightweight storage subsystem based on SQLite was designed to replace a legacy text log based persistence mechanism, with tuned journalling modes and a refactored worker-thread concurrency model. A containerisation strategy for the graphical components of the platform was developed, combining Docker images with a remote-desktop-based display layer to enable reproducible, isolated and securely exposed deployments. A Kafka-based messaging backbone was introduced to replace a point-to-point Transmission Control Protocol (TCP) messaging system, including topic and partition layouts, producer and consumer tuning for low latency, and a thin librdkafka wrapper to support deterministic benchmarking and integration. Finally, a deterministic perception-to-action controller was implemented to translate vendor-agnostic orders into machine-executable instructions. The artefacts were assessed through targeted validation activities, including database latency and resource measurements, Kafka broker and integration tests, a container display-latency comparison, and actuator functional and latency tests. The remaining contribution consists of a set of integrated, production-oriented components and a body of design principles and best practices for deploying low-latency, resourceefficient edge inference systems in manufacturing environments.

Firstly described by Newton in the 17th century, speckle is an optical phenomenon which can be translated into image patterns produced by wave interferences of diffused reflections. In fact, the speckle pattern is generated by the multiple interference phenomena that occur when a rough surface is illuminated with a coherent source of light, producing randomly distributed reflected waves of the same frequency but different phases and amplitudes. Although it has been known for a long time, capturing video sequences of speckle patterns was dependent on recent technological developments, in particular, related to laser technology and microsensors. The speckle acquisition setup comprises a light source, usually a laser, an optical beam expander and a CCD camera. The generated interference patterns are captured in series of video sequences, to further be processed. In previous works, several image processing algorithms have been applied to analyze video frames of speckle, aimed to capture the evolution patterns in dynamic processes. However, due to the typical high frequencies of the changing patterns, classical texture algorithms mostly fail this goal. In this work, speckle dynamics are evaluated using Local Binary Patterns (LBP) jointly with some of its main variants and a newly proposed algorithm, in a reactive hyperemia controlled test. The proposed methodology goes beyond the traditional implementations of LBPs, by considering an additional Gaussian filtering, a methodology thus coined as LBPg. The results, on one hand, confirm that the classical formulations of LBP are not sensitive to changes in the simulated patterns but, on the other hand, demonstrate that the newly proposed LBP-adapted algorithm successfully identify the dynamics of the processes under study.

Nowadays, computer vision (CV) is widely used to solve real-world problems, which pose increasingly higher challenges. In this context, the use of omnidirectional video in a growing number of applications, along with the fast development of Deep Learning (DL) algorithms for object detection, drives the need for further research to improve existing methods originally developed for conventional 2D planar images. However, the geometric distortion that common sphere-to-plane projections produce, mostly visible in objects near the poles, in addition to the lack of omnidirectional open-source labeled image datasets has made an accurate spherical image-based object detection algorithm a hard goal to achieve. This work is a contribution to develop datasets and machine learning models particularly suited for omnidirectional images, represented in planar format through the well-known Equirectangular Projection (ERP). To this aim, DL methods are explored to improve the detection of visual objects in omnidirectional images, by considering the inherent distortions of ERP. An experimental study was, firstly, carried out to find out whether the error rate and type of detection errors were related to the characteristics of ERP images. Such study revealed that the error rate of object detection using existing DL models with ERP images, actually, depends on the object spherical location in the image. Then, based on such findings, a new object detection framework is proposed to obtain a uniform error rate across the whole spherical image regions. The results show that the pre and post-processing stages of the implemented framework effectively contribute to reducing the performance dependency on the image region, evaluated by the above-mentioned metric.