Processamento Multimédia Baseado em Aprendizagem

Objectives

Modern lifestyle relies on technological devices, connected over the Internet, to access a vast array of services using multimedia data to enhance the interaction experience. Multimedia information can be explored in many ways, including the authentication of users, entertainment, or to provide an augmented reality experience of the surrounding environment, among many other examples. This course addresses the processing of multimedia signals using pattern recognition and machine learning based concepts and tools. To focus the course, most of the examples and the projects will be centered around biometric systems, covering several media modalities, as well as multimodal combinations. Several applications will be addressed, from surveillance and recognition, to forensics, but also medical applications and augmented reality.

Syllabus

1. Introduction to Multimedia and to Biometrics Definitions, MM acquisition, MM representation, human perception, applications. Ethics and data protection, notably in the context of biometrics. 2. Learning Basics and Tools for Biometric Recognition Learning concepts and tools (supervised, unsupervised, deep learning); system architecture, performance evaluation, standardization, application examples. 3. Unimodal and Multimodal Biometric Systems Biometric modalities (e.g., fingerprint, face, gait, softbiometrics), multi-biometrics, application examples. 4. Multimedia Contents and Trust The problem of artificially generated contents, deep fakes. Counteracting attacks to biometric recognition systems. Application examples. 5. Other Applications of Biometrics Biometrics and forensics, medical applications, emotion and mood recognition. 6. Future trends Challenges: covariates, operation “in the wild”, new sensors. Tendencies.

Prerequisites

Some knowledge of machine learning, deep learning, digital signal processing and image and video processing will be useful to develop more interesting and complete projects.

Cross Competence Component

Soft skills aspects covered: 1. Critical and Innovative Thinking – The project requires strategic and critical thinking, and problem-solving strategies. Creativity is required for application to new scenarios. 2. Interpersonal Competencies– The project is teamwork. It will be presented orally and debated in class. 3. Intrapersonal Competencies– For the project it is essential to manage the time and the stress level, to keep the motivation and to make decisions. 4. Global Citizenship– Ethics, tolerance and respect of diversity will be addressed in class, when discussing biometrics, and should be considered when developing the project. 5. Information and Media Literacy– State-of-the-art review and selection of the paper used as basis for the project require computing tools and search and management of information. The presentation will use computing and multimedia tools, and it will be structured and formatted. These components amount to around 50% of the grade.

Laboratorial Component

1. Laboratory sessions: during the initial weeks these sessions introduce basic concepts and tools.

Programming And Computing Component

The laboratorial activities have a strong emphasis on the computational thinking competence: 1. Laboratory sessions based on the programming of multimedia signal processing solutions and machine learning tools, using available libraries. 2. Project based on the modification of available software to implement new functionalities. These components amount to around 50% of the grade.

Ethical Principles

All members of a group are responsible for the group’s work In any assessment, every student shall honestly disclose any help received and sources used. In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.