Thesis topic: Human vs. AI Code Review in Pull Requests

Are you a final-year master’s student looking for a thesis project that blends real-world impact with cutting-edge tech?

At Framna, we are passionate about creating digital products that shape markets and make everyday life better - and we want you to be part of it.

Every year, we welcome a group of master’s students into our Stockholm studio to spend their final semester with us. You will sit alongside our developers, designers, testers, analysts, and product delivery leads, get a place to sit in our cozy office, and join our social and learning activities - from lunch-and-learns to our weekly Friday demos and AWs. It is a chance to do your thesis in a collaborative setting, surrounded by people who are happy to share what they know.

Background

Code review is a cornerstone of modern software development - a practice that catches bugs, spreads knowledge, and upholds quality standards across a team. As AI-powered code review tools become increasingly capable, questions arise about how their feedback compares to that of experienced human reviewers.

Do AI tools catch the same types of issues as humans? Where do they add the most value, and where do they fall short? And how can they be designed to complement, rather than replace, the human perspective in a review workflow?

In this thesis project, you will conduct an empirical study comparing human and AI-generated code review comments on real pull requests. The goal is to produce a nuanced understanding of the complementary strengths of each, and to generate practical recommendations for how AI review tools can best support development teams.

What you will do

Some potential tasks within this thesis include:

• Collecting a dataset of pull requests with existing human review comments
• Generating AI review comments for the same pull requests using one or more code review tools or LLM-based systems
• Developing a structured coding scheme to categorise review comments - for example by correctness, security, performance, readability, architecture, naming, test coverage, edge cases, documentation, and domain-specific concerns
• Analysing whether human and AI comments address the same parts of the code and identify the same types of problems
• Combining quantitative analysis, such as comment category frequency and overlap, with qualitative analysis including expert evaluation of usefulness, accuracy, severity, and actionability
• Producing a classification of review comment types, an annotated dataset, evaluation metrics, and recommendations for how AI review tools should support human reviewers

Who you are

This thesis may suit you if you:

• Are interested in software engineering, empirical research methods, or human-AI collaboration
• Are curious about the role of AI in developer workflows and code quality
• Enjoy reading and reasoning about code across different languages and domains
• Plan to start your master's thesis project in January 2027
• Have the opportunity to work from our Stockholm studio

About Framna

Framna means ‘to bring forward’ in Old Norse. And that is exactly what we do. We partner with industry leaders (and those about to be) to create excellent digital products that define markets, reshape industries, and deliver meaningful impact.

Born from the union of digital agencies Bontouch, Move, and Shape, Framna emerged from a strong, product-led culture. Today, over 600 of us collaborate across ten studios in Denmark, the Netherlands, Poland, Sweden, Switzerland, and the US.

Every day, millions of users around the world rely on the products we help shape. From seamless payments to daily commutes, from better health management to smarter shopping. Our work supports more than €150 million in daily transactions and consistently earns an average App Store rating of 4.5.

Together with some of the world’s most ambitious brands, including Essity, SJ, Swish, and SEB, we craft products that lead markets and push boundaries. Because when products win, businesses win. We call it Win by product.

Our hiring process

• Psychometric assessment - personality and logic test
• Interview with our recruiter
• Case assignment
• Case interview

Do you want to meet the team in person? Join us on September 17th for our Master's Thesis Evening! You will hear more about our upcoming thesis topics, meet the team behind our projects, and get a feel for how we work at the intersection of creativity and technology. Sign up here!