Thesis Opportunity- Fine-tuning JEPA models with heightmap images

Background

SICK’s laser triangulation cameras produce spatially aligned 2D grayscale images and 2.5D heightmaps. Together, these two modalities provide complementary information about an object: grayscale images capture visual appearance and texture, while heightmaps provide information about surface geometry.

Heightmap images are by nature much dissimilar to 2D grayscale images: each pixel represents its height and may even be undefined. Due to this, existing AI vision encoder backbones - often only trained on natural images - are not able to represent heightmaps in a good way.

Thesis objective

JEPA (Joint-Embedding Predictive Architecture) is a self-supervised learning framework receiving much attention from the scientific research community. For missing information, JEPA-style methods learn to recover the missing information from its latent space, rather than first reconstructing pixels. This makes them an interesting candidate for learning representations that are robust to occlusions and missing height data.

The plan is to fine-tune an existing JEPA-trained backbone with aligned 2D/2.5D images, teaching the model to place the latent representations of the height map images close to its 2D counterpart. We will then compare the performance of such fine-tuned models on specific downstream tasks, for different amounts of missing data.

Research environment

The thesis will be carried out at SICK, in collaboration with the Computer Vision Laboratory (CVL) at Linköping University.

You will work closely with Matheus Bernat, an industrial PhD student at SICK and CVL, Linköping University, whose research focuses on multimodal AI, and representation learning.

The project is intended to have a strong research component. For a motivated student, there is an opportunity to work towards a joint publication towards the end of the thesis.

Contact

For more information about the position, contact:

Matheus Bernat, matheus.bernat@liu.se

We warmly welcome your application — please submit it no later than October 25th.

SICK is a world-leading supplier of sensors and sensor solutions for industrial applications. We’re part of SICK AG — a global leader in sensor technology with 10,000 employees across 50 countries and headquarters in Freiburg, Germany. Together, we build technology that makes industries more efficient, intelligent, and safe.

As a Machine Vision Innovation Center, SICK Linköping develops advanced AI-powered software that drives the future of both manufacturing and logistics automation. Whether it’s helping robots pick the right item or enabling high-precision quality control with 2D and 3D vision, our solutions bring clarity, speed, and smart decision-making to complex industrial environments — all driven by a dedicated team of 100 colleagues.

We are very proud of being a healthy and attractive workplace. We have consistently been recognized as one of the best workplaces in Sweden according to the Great Place to Work survey. We actively work to reduce our climate footprint and engage in various initiatives to contribute to society and enhance diversity at our workplace.