Interview with Anna Kukleva: From PhD to Postdoc, Winning the Otto Hahn Medal, and Her Research in Computer Vision

Anna Kukleva, now a research scientist at Meta and formerly a postdoctoral researcher at the Max Planck Institute for Informatics, shares her journey from PhD to postdoc, discusses her Otto Hahn Medal-winning research, and offers advice to aspiring computer scientists in our interview. Photo: Yağmur Akarsu

How do you go from master’s to PhD to postdoc? Most postdocs would tell you it’s hard work, perseverance, and discipline. According to Anna Kukleva, it’s those, plus a bunch of happy accidents.

Currently, she’s a research scientist at Meta. Before that, she was a postdoctoral researcher in the Computer Vision and Machine Learning department at the Max Planck Institute for Informatics (MPI-INF).

After being exposed to computer vision during her bachelor’s in Russia by coincidence, Anna decided to stay in the field throughout her entire education. Upon completing her PhD in Saarbrücken, she was awarded the Otto Hahn Medal for her thesis “Advancing Image and Video Recognition with Less Supervision,” in which she explored how to get good results on unseen data with minimal labeling.

In this interview, Anna talks about her award-winning thesis, her experience as a researcher at MPI-INF and as a woman in computer vision, and her advice to aspiring computer scientists.

What drew you to computer science and more specifically, to computer vision? 

It was almost an accident, to be honest. I actually planned to go into mathematics. In Russia, we have an international unified exam, and I was looking at the list of subjects I needed to pass. Russian and mathematics are mandatory, so you don’t need to choose them. In addition to those, I only had physics, while everyone else in my class was choosing more subjects. So I thought, why shouldn’t I? That’s when I decided to take computer science. There was nothing I didn’t like about it — it was almost math, but slightly different, and I found that I could learn it quickly. After getting my exam results, I thought, why not choose computer science as my major? 

When I started my bachelor’s, I was the only one in class who didn’t know how to program. That was slightly alarming, but in the beginning it was mostly math anyway. 

After my first year of studying, I ended up in a summer camp — again, by accident. (Lots of happy accidents!) We worked on a quadrocopter (drone) in four teams, with each team responsible for a different section. For instance, one team developed the app’s web page, while our team was responsible for computer vision. I did camera calibration in C++, which I didn’t know at the time, and it became my task for the entire camp. I had a lot of fun doing it and fell in love with computer vision. I decided to stay in the field, and in the end, it worked out! I spent two years in a computer vision lab during my bachelor’s, and when I moved to Germany for my master’s, I joined another computer vision lab as a HiWi.

I like working with images to understand how computers interpret and extract information from what we see — which is, in memory, actually just numbers representing pixels. It’s intriguing that we can reason more and more just from numbers. This is fascinating. The field is developing so fast. It’s incredible to be part of it and stay on top of how these systems work: what kinds of algorithms they use, how they are trained, and what their limitations are. I wouldn’t necessarily say it’s more interesting to be in the field right now, but it’s certainly different, because it’s applied in the real world more than ever. Right now, computer vision is deep learning, which has increasingly been integrated with language models over the past few years.

Currently, you can just upload images to a model and ask, ‘What is in this image?’ That’s related to my field and what I’m doing, because the model needs to understand the context of the image or video. Developments like these are extremely motivating to stay in the field and keep on pushing the research boundaries. The pace of the field has been crazy in these past few years. Nobody knows what’s coming because there’s always something new happening. You see papers coming in the arXiv every day.

You were awarded the “Otto Hahn Medal of the Max Planck Society” for your thesis, “Advancing Image and Video Recognition with Less Supervision”. What piqued your interest in the topic and can you tell us a bit more about your thesis?

Absolutely! The topic is about how we can learn from data. 

To give you more context, I started my path in computer vision research with unsupervised learning. Imagine you have a bunch of data, a bunch of videos — and you’re trying to understand how coffee is made. There are actions like picking up the cup, putting on the cap, stirring sugar, pouring water, and you want to recognize all these actions — but in a way that doesn’t require supervision in the first place.

Supervision is very hard to obtain. It’s very multifaceted and comes with limitations; you can’t label everything in the world. There are a lot of underrepresented classes, and things change all the time. For example, today we sit on these chairs, and in five years they might look completely different, but their purpose will be the same: we still sit on them. In this case, you would need to re-label and add these new chairs to your model, so it can recognize them. 

There are problems with labeling as well. Going back to the chair example, you can label the whole chair, or just the arm or back, depending on the granularity of your labeling and the downstream task you want to solve. 

What I was primarily interested in was how we can exploit the data in different scenarios. When we talk about video understanding, image understanding, or multi-modal understanding, the question is how we can generalize using only a subset of absolutely unlabeled data. Can we achieve good results on unseen data? On test data? That was my topic, in a nutshell.

Regarding my interest in the topic, I think I found common ground with my supervisor. I wanted to work with less supervision. When I started my PhD, we began with few-shot learning, where you’re given very few examples and need to learn your representations from them — and you then model based on these few samples. 

So we started there, and because I like videos, I steered back to working with video data. Fast-forward to later, I was working with large-scale, unlabeled data, focusing on how to get better representations and improve datasets using LLMs. 

It was always about less supervision, mostly with videos but sometimes with images, and about how different learning dynamics influence the kinds of models we actually get. The goal was to understand the internal structure of algorithms. It was also very interesting to see exactly what a model learns and what influences the structure of its representations. This focus was always there: learning with less data and overall less supervision. And then we explored different directions along these lines.

Your thesis mentions that fully supervised models are often impractical in real-world applications. Which fields or industries do you think struggle the most with this necessity for data labeling?

I think any field would be limited. Especially when you have a pre-defined, structured pipeline, like in car manufacturing. There, you have details that always have the same size and shape. But if you don’t have a window, then the outdoor lighting doesn’t make any difference — you can set up the lighting at a fixed point so everything always looks the same. When that happens, robots or algorithms can manipulate objects without much difficulty, because you can define precisely where and how things should be done. 

So the factors and the objects are fixed. It’s a sealed, perfect environment where nothing changes. Whenever something starts to change, though? You’ve got a problem. Imagine you have a table and your task is, say, recognizing objects to help you make coffee in a VR setup (like those from Apple or Meta). Your coffee-making mechanism needs to recognize the cup, the steamer, the spoon, the coffee itself — essentially, everything has to come together. Afterwards, the system should also suggest the actions you’ll need to take. 

Maybe the system doesn’t work if there’s too much or too little light, so it should be able to tell you to open the window, turn off the light, and so on. Keep in mind that this is still an example with rigid objects.

Let’s think of a plastic bag instead, which changes shape constantly. Imagine filling it with powder or liquid — a common scenario in medical applications. Sounds easy, you just take the bag and fill it, right? No! This is actually very complex because a plastic bag isn’t rigid and is constantly changing. There’s also the durability factor. We have sensations that tell us when the bag is under too much stress, close to ripping, and how much force we’re applying to carry it. Our senses give us that feedback — but robots don’t have that. For them, controlling that is extremely hard. 

Essentially, take any application where you have some degrees of freedom, and you’ll see that it’s very hard to label everything. Take autonomous driving, for example. 

Of course, large teams at every car company create proprietary datasets (which aren’t available to academia because they’re not open source). But still, they can’t label everything, as some things are simply unpredictable and it costs a lot of time and money. 

However, generally speaking, it’s not sustainable for every industry or company to collect its own datasets and guarantee high quality. You often need three annotators per data point, not one. You need to be consistent throughout the entire process, and ensure the annotators do a good job and that there aren’t mistakes involved. It’s very common to have mistakes in datasets, because labeling is hard. So essentially, name the field —and you will definitely need methods that rely on less supervision.

Fundamentally, the best approach that I see is to start with a small dataset with very high-quality labels and then propagate this to unlabeled data. Perhaps it’s not an approach that works absolutely without labels, but for some downstream tasks, some guidance is still needed in terms of what labels are expected and what kind of supervision is required. The model can catch up gradually.

However, fully labeling everything is unreasonable, because the world is constantly changing. Then you would need to constantly update and retrain models. It’s much better and effective if a model can pick up on something new. Models should be able to recognize new concepts and integrate them into their representations over time.

And currently, can this approach be used for complex tasks?

I think so. It depends, because you can’t always take something and apply it to completely different data — it might not work. You need to understand the internal structure of the data and how to apply it. There are also additional biases we have as humans, in terms of how things work and what to expect.

Let’s think about an instructional video on making coffee. You know that you’ll first get a cup and then pour water, not the other way around. One action comes first, we know that. This kind of bias can also be observed in other data structures, especially biological or molecular. There are some connections that should be there and some that shouldn’t. You can explicitly or implicitly impose this on a model while it’s learning, and thereby help it recognize things.

What were some challenges you faced during your research and how did you overcome them? 

Every PhD has certain ups and downs. Most of the time, it’s the moments when you’ve been trying to figure out something and suddenly, nothing works and you have no idea what to do next. There are different ways to overcome this. What worked for me was taking a step back — going for a walk, overall, relaxing.

Sometimes when you constantly try to make a problem work and it still doesn’t — you get lost in small details and can’t think of new solutions. When that happens, you need to do something absolutely different — distract yourself for a bit. Read papers, talk to people, sleep, whatever you do: don’t think about the problem. When you’re done, come back and try something different.

It helps! Sometimes it might take more time — maybe weeks, maybe months — but it happens eventually. This is the main PhD challenge for everyone, I think. Of course it’s always frustrating when something you think will work doesn’t, but part of the process is learning how to deal with this. You will have ups and downs. It doesn’t get easier, to be honest, but you learn how to handle it.

In addition to that, for me, starting my PhD during Covid was a huge challenge. I essentially moved to a new city, knowing no one, only a month before Covid hit. I went to the office for a month, and then it closed down. Socially, that was pretty hard. It’s interesting, really — when you’re frustrated and alone, it only adds more pressure.

I did an internship right at the beginning of my PhD as well. Thinking about it now, I think it was too early. It didn’t work out in the end. My manager left as soon as I joined, the project itself fell apart, and although I had the opportunity to continue, I think I made the right decision to say, ‘Okay, I’m not doing this anymore.’ I feel like it takes some courage to tell your supervisor that. Afterwards, I moved to another project. 

I would say it’s important to know when to step back sometimes. In this particular case, I’m quite proud of myself. 

Before working at Meta, you were a postdoctoral researcher at the Max Planck Institute for Informatics at the Saarland Informatics Campus. What were you working on?

I was involved in several projects. I helped PhD students, supervised master’s students, and also worked on my own research. One of my projects focused on understanding attention in diffusion models, which are generative models that create images or videos, such as DALL·E or Sora. You go to the app or the webpage, send a prompt that goes, ‘I want to have a cute cat,’ and the model generates it.

Diffusion models can be used for many different tasks, from image generation to extracting internal representations for applications such as segmentation. I specifically studied attention layers, which are key building blocks of these models. They are where communication between an image and the model’s internal representations takes place. Conceptually, it’s an operation — like multiplication, but far more complex — and we studied how it functions inside the diffusion models, how it affects image generation, and how it shapes the representations learned by the model. At the time, we were targeting a specific segmentation task. We had found a mechanism that amplifies certain attention regions in the internal representations, improving segmentation performance. Essentially, we discovered how to adjust the attention operator and its outputs without retraining the model.

Although the focus of my research remained on computer vision, we were increasingly incorporating language as well. We explored how images or videos could be combined with language to improve internal representations or enable reasoning about visual scenes. This included tasks such as answering questions about visual content, describing relationships between objects in an image, and incorporating additional modalities like audio, depth estimation, or optical flow.

This research was part of a collaboration with Google. Our group at the Max Planck Institute, together with Christian Theobalt’s group, participated in the VIA collaboration. I worked closely with researchers at Google, met with them regularly, and co-authored research with one of the collaborators.

What set the Max Planck Institute for Informatics and SIC apart for you in terms of research opportunities, as well as academic and professional development?

First off, during my interview with my supervisor, I immediately felt a connection. I think it’s super important to see if things click with your supervisor before starting a new position, so I was really happy that was the case. Our communication felt established from the start. 

Oh, and they have a great coffee machine! I’m telling you, I had an internship in Paris — I didn’t expect this from France, but the coffee machine? Quite bad. At MPI, though? I even learned how to make coffee art during my PhD!

Jokes aside, the more time I spent there, the more I learned about the Computer Vision and Machine Learning department and how it was structured. The infrastructure was impressive compared to any other place I had seen in academia. Not just academically, but also in terms of the clusters and setups we used to train our models. There was a dedicated IT department — handling server management, laptops, devices — so that everything ran smoothly. 

I was surrounded by truly inspiring people and there were plenty of possibilities to learn, explore, and collaborate. In our department, we had subgroups that worked on very different topics, and you can learn a lot from that. If you were interested, you could even collaborate. Overall, it was quite different from other places because we had three or four professors in close collaboration under this department.

We also had retreats twice a year, not just within our group but jointly with other groups across Germany. They were a great opportunity to meet people, connect with professors, and even make friends. You got familiar with what the other groups were working on while building nationwide connections.

Most importantly, compared to my friends at other institutions, we had less teaching responsibilities. I found that genuinely amazing — we could concentrate on research completely, which is exactly what you need during your PhD. That was a huge factor for me when I was choosing MPI-INF.

What advice would you give to students starting out in computer science, machine learning or computer vision, and to prospective SIC students – especially young women?

First things first, you need to build up your portfolio. When I first started as a HiWi, I was pushing to take on projects or help with existing ones; that’s how you learn. 

When you’re looking for a PhD, there are basically two options. First option: trying to work with a well-known, established professor, which can be great — but you have to understand you might not see them much because they’re busy. In this scenario, while it would also depend on the group structure, you’ll most likely be working quite independently, which can be challenging.

Your second option, which is very valuable but harder to find, is finding a fresh professor. They’re usually enthusiastic, more involved in your projects, and you can learn a lot from them. It can be slightly easier to get a position with a fresh professor, because there are fewer people applying to them. There are pros and cons to this though — firstly, as I said, it’s very hard to find. And you have to consider the institution and the resources it has. A fresh professor at a reputable institution is ideal. Fresh professors at less-established places, however… I’m not saying you can’t be lucky, but you could also be unlucky. It’s a hit or miss, you know? In a reputable institution, a fresh professor is more likely to give you attention and guidance. Senior professors, on the other hand, are less available because they often travel a lot and don’t attend all meetings — which is totally normal. Like I said, they’re busier in comparison.

If you feel too conflicted, asking peers for recommendations can help — although if they’re also master’s students, they might not know much themselves. All I’m saying is, it’s tempting to target the big names, I know, but it may not always be the ideal choice. Take your time and choose your destination wisely. I think I got really lucky with MPI-INF Saarbrücken. 

Another thing, and this is very important: don’t do ten projects at the same time — concentrate on one! One project is more important than ten simultaneously. It rarely works. I only know one person capable of doing that, and mind you, he’s a genius — but even he can only handle five at a time. Everyone else fails when trying to juggle more than one, myself included.

I was initially just a member, but now I’m on the board of the Women in Computer Vision workshops. We publish at major conferences like CVPR (Computer Vision and Pattern Recognition Conference), ICCV (International Conference on Computer Vision), and ECCV (European Conference on Computer Vision) rather than in journals.

From my experience, I would recommend women in computer science to simply reach out to other women — it’s a great way to meet people who can understand and support you. In our field, there’s a culture of guiding or helping out others when they’re at times of uncertainty — this applies to all genders, of course. But since women are still underrepresented in computer science, especially in computer vision, this support can be especially valuable.

As a matter of fact, many professors are trying to recruit more women for PhD positions, but it’s hard because there aren’t many applicants. So my advice is simple: just apply, that’s it! If you’re interested in someone’s research and would like to reach out, do it! Don’t be afraid to take the first step.

Interested in the Computer Vision and Machine Learning department of the Max Planck Institute of Informatics? Click here to visit their page.

Woman in Computer Vision? Visit the website of Women in Computer Vision to find out more about their activities!

Editor:
Saarland Informatics Campus Team
Email: yagmur.akarsu@uni-saarland.de