Bar Raiser Interviews, OKRs, the Spotify Model and Agile: Companies Are Full of It

On Monday I posted this on X:

please i beg you all stop it with your bar raiser interview nonsense
you don’t need all this to spot real talent come on

That was the short version. This is the long one. Companies are full of shit: bar raiser loops, Objectives and Key Results (OKRs), the Spotify model, sprints and standups are mostly complexity somebody copied from a bigger company without checking whether it ever worked there. In a lot of cases it didn’t and I went and checked. Then I want to show you why AI makes this urgent. And at the end, how I’d hire instead, which is how I got hired at all.

What is a bar raiser interview?

A bar raiser interview is a round in a hiring loop run by a trained interviewer borrowed from another team, whose job is to judge whether you’d raise the bar of the company. Amazon started it in 1999 and scores you against its 16 Leadership Principles. Amazon’s own description calls them “an objective third party during the hiring process” who “drive the pivotal decision on whether a candidate should be hired”. There are more than 10,000 of them. Revolut runs one too now: 45 to 60 minutes with a senior leader from outside the team, scored on quantified ownership. Google’s autocomplete puts “bar raiser interview” next to “revolut” and “mastercard”, which tells you how far the copy has spread.

So a candidate spends an afternoon telling strangers stories in the Situation, Task, Action, Result (STAR) format about the time they showed Ownership or Bias for Action, which the strangers score against a list of 16 phrases. Then everybody copies it. Every company with a Notion page now has a “bar raiser” in the loop, a values list and a scoring rubric, as if the ritual is what made Amazon big.

The best data on whether any of this works comes from a company that ran the same kind of machine at scale and then published what it found. In 2013 Laszlo Bock, who ran hiring at Google, said that when they compared tens of thousands of interview scores against how those people actually did on the job, “We found zero relationship. It’s a complete random mess.” Brainteasers were “a complete waste of time” that “serve primarily to make the interviewer feel smart”. Of grades and test scores he said “they don’t predict anything”. And his analyst Todd Carlisle found that 4 interviews were enough to predict with 86% confidence whether Google should hire someone. Four!! Not a loop of 6 and a debrief.

What does predict performance is less exciting. Paul Sackett and his colleagues re-ran the big hiring meta-analyses in 2022 without the statistical corrections that had been inflating the old numbers. The structured interview came out on top, at a correlation of 0.42 with job performance. Ask every candidate the same questions about the actual job and score the answers the same way. It takes 1 interviewer who knows the job. It doesn’t take a Bar Raiser.

Structured interview

0.42

Job knowledge test

0.40

Biodata

0.38

Work sample

0.33

Cognitive ability test

0.31

I’ll tell you what I failed. In 2015 I was in Qatar with severe hemophilia, trying to get to any country with a healthcare system that would keep me alive, applying for engineering jobs in all of them. I got a lot of interviews and I failed all of them. Booking.com turned me down for not having enough A/B testing experience (go and look at Booking.com, the entire site is one enormous A/B test). I’d have failed a bar raiser loop too: no degree, barely any school and no STAR story about the time I showed Ownership. Then Rodrigo Paiva at McMakler, a small Berlin startup, interviewed me and hired me, with relocation. For years I told that story as “Germany was so short of engineers they’d hire anyone”. After he read my story he wrote to me to say the hire was nothing like an accident. I’d aced the technical test and he chose me because I wanted out and had shown the initiative to get there. 6 years later Spotify made that same guy a staff engineer.

What is leader filtration theory?

Gautam Mukunda, who taught at Harvard Business School, has a theory for what a filter like this produces. Leader filtration theory says every big organization has a process that evaluates the people who might lead it. A leader who comes up through that process is filtered: well known, thoroughly vetted and much like every other person the process would have picked. A leader who comes in through a side door is unfiltered: an outsider, a vice president who got the top job when the president died, someone nobody had the chance to screen. His claim is that filtered leaders land in the middle and unfiltered ones, in his words, “are likely to do either much better or much worse than a filtered person would”.

Then he tested it on every US president. He sorted them by how much relevant experience they had before the office and how they got there, then set that against how historians rank them. 4 of the 5 highest ranked presidents were unfiltered. So were 4 of the 5 lowest ranked. Washington and Lincoln were both unfiltered and Theodore Roosevelt was a vice president who got in when McKinley was shot. Mukunda puts the result at the 99.97% significance level and the book is Indispensable: When Leaders Really Matter.

5 highest ranked

Unfiltered

4

Filtered

1

5 lowest ranked

Unfiltered

4

Filtered

1

Now put a Bar Raiser on that. Their whole job is to make sure nobody gets in who wasn’t vetted. By design the loop gives you the predictable middle, which Mukunda says is what you should want when things are going fine. It’s also the mechanism that keeps the Lincolns out. I’m not saying hire at random off the street. I’m saying the loop gives you a drone every time because a drone is what it’s built to give you: someone with a tidy story for every principle on the list who will follow orders in the right format. Nobody inspired got through a filter by being inspired.

Does the Spotify model work?

Spotify itself never ran it. I was a staff engineer at Spotify in 2021. Squads, tribes, chapters and guilds come from a 2012 whitepaper by Henrik Kniberg and Anders Ivarsson, written when Spotify had 30 teams and about 250 people in them. Companies 10× that size copied the org chart as if it came with the music.

Jeremiah Lee, a product manager at Spotify in Stockholm, wrote up in 2020 what it was like from the inside. The matrix of squads and chapters left nobody accountable. Autonomy without alignment meant every squad did its own thing. And the mythology made the model impossible to criticize. One of the people who wrote the whitepaper told him “Even at the time we wrote it, we weren’t doing it. It was part ambition, part approximation.” That person also said “It worries me when people look at what we do and think it’s a framework they can just copy and implement.” People looked and people copied.

What I can tell you from my few months there is that the thing that made the place work wasn’t a diagram. There was no “frontend engineer” or “backend engineer”. There was engineer, senior engineer, staff engineer, principal, and you did everything, including (in my case) scaling a database across regions with region level failover. That’s a hiring decision about what kind of person you want in the building. You can’t copy it by renaming your teams.

Why OKRs don’t work

OKRs, key performance indicators (KPIs), whatever your company calls the spreadsheet this quarter. The research on goal setting is enormous and it does say a specific, hard goal changes behavior. The same research says what it does to everything else. Lisa Ordóñez, Maurice Schweitzer, Adam Galinsky and Max Bazerman titled their 2009 paper “Goals Gone Wild” and listed the side effects: a narrow focus that neglects everything outside the goal, distorted risk preferences, a rise in unethical behavior, inhibited learning, a corroded culture and reduced intrinsic motivation. Their line is that goal setting should be treated as “a prescription-strength medication that requires careful dosing”. Companies hand it out like candy.

Then Marty Cagan, who spent years telling companies to adopt OKRs, wrote in 2020 that in the majority of companies he meets he has “stopped recommending the practice”. In those companies it “ends up proving a waste of time and effort, and yields little if any results”. He says why. OKRs are an empowerment technique: you give a team an actual problem and the space to solve it. A company that hands a team objectives and a feature roadmap and deadlines has ticked the empowerment box and kept the orders. Which is what happens at most companies I’ve seen. The slide got a number on it.

Agile stopped meaning anything in 2014

Dave Thomas helped write the Agile Manifesto in 2001. In 2014 he published a post called “Agile is Dead” that said the word “has been subverted to the point where it is effectively meaningless”. Once the manifesto got popular, “the word agile became a magnet for anyone with points to espouse, hours to bill, or products to sell”. Scrum masters, certifications, 2 week sprints, story points, velocity charts, a daily standup where 9 people report to 1 person. The manifesto’s first line is “Individuals and interactions over processes and tools” and the industry built a process and tools empire on it. That’s insane… it’s literally the first line.

There’s a 2024 study of 600 engineers that claims projects run on agile style requirements fail 268% more often. Take it with a grain of salt though. The person who ran it, Junade Ali, sells the alternative, a method he calls Impact Engineering, which in the same study fails only 10% of the time. That’s the genre. The person telling you agile is broken is usually selling the next thing and I’m not selling a method.

Is agile dead in the age of AI?

The word died in 2014. AI is finishing off the process built around it. Look at what every piece of this machinery selects for. The loop wants a person who produces the expected answer in the expected format. The OKR wants a person who hits a number somebody above them chose. Scrum wants a person who takes a ticket, estimates it, closes it, repeats. All of it optimizes for following orders well. That’s also a fair description of what a language model is good at.

Erik Brynjolfsson, Bharat Chandar and Ruyu Chen at Stanford have been watching this in payroll data from Automatic Data Processing (ADP) covering millions of US workers. By June 2026, employment of 22 to 25 year olds in the occupations most exposed to AI was 19% below their less exposed peers, while experienced workers in the same jobs showed no such drop. The declines are concentrated where AI substitutes for a person’s tasks. Where it complements the person, employment is flat or rising. They call it canaries in the coal mine.

Young workers in AI-exposed jobs

−19%

And the loop itself is gameable with the same technology. In 2025 a Columbia student called Roy Lee built a desktop app, invisible to the interviewer, that solved the coding questions during his interviews and used it to get an internship offer from Amazon. He filmed it. Amazon had the video taken down, Columbia suspended him and he raised $15 million for the company he built out of it. I don’t think he’s a hero. What it proves is that a filter a 21 year old can pass with a hidden window is measuring something a machine does better than the candidate. And that’s the thing you’re paying 6 interviewers an afternoon each to measure!!

So the drone work is going. Going by the Stanford numbers, the people hired for it are the ones AI is replacing first. And the people who’d have mattered got filtered out in a debrief by a volunteer from another team. What’s left when the work that follows orders gets cheap is the stuff I wrote about last time: taste, novelty, the person who notices the thing nobody asked about. That person is the side door hire. That person has no STAR story for it. This is roughly what I say to a room of engineering leaders when a company has me in to talk about AI and their teams. There’s always one person in that room nodding who got in through the side door.

How to hire good software engineers without all this

Ask every candidate the same questions about the job, watch them do the work in private and pay them for a short trial. The prophet Samuel nearly anointed the wrong king because the oldest brother was tall. And God stopped him with “man sees the outward appearance, but the LORD sees the heart” (1 Samuel 16:7). Every loop I’ve described is built on Samuel’s first instinct: the polish, the confident STAR story, the right school on the CV. The research on getting past it is old and pretty consistent. None of it needs a bar raiser.

Ask everyone the same questions, then ask your gut last

The structured interview tops Sackett’s chart above. Its opposite, an hour of friendly chat, does worse than nothing. In a 2013 study by Robyn Dawes and 2 colleagues, some of the people being interviewed answered from a random response system. The interviewers formed impressions “just as confidently” as they did after honest ones. And the interviews “actually led participants to make worse predictions”. In a third experiment people said they’d rather have a random interview than none at all, which is insane… we’d literally take noise over nothing.

Daniel Kahneman fixed this for the Israeli army while he served in it, a story he tells in Thinking, Fast and Slow. He “insisted that interviewers ask each applicant the same series of objective, factual questions about six characteristics” (Westmont Magazine). The interviewers wanted to use their judgment instead. So he let them give one gut rating at the very end, after everything else was scored. In his words, “It’s better to collect all the data first and then close your eyes and be intuitive.”

For an engineering role, pick the handful of things the job actually needs (debugging something broken in production, reading code you didn’t write, explaining a tradeoff to a product manager, whatever yours are). Write questions about each one and ask every candidate the same ones. Score each thing before you move on to the next and ask your gut at the end. Keep it to 4 interviews. Google found 4 were enough to predict its hiring decision with 86% confidence, which Bock called the “Rule of Four” (Wharton). He says Google’s time to hire went “from something like six months to 45 days”.

Swap the whiteboard for the actual work, done in private

In 2020 Mahnaz Behroozi, Chris Parnin and their colleagues at North Carolina State University (NC State) gave 48 computer science students the same coding problem. Half solved it at a whiteboard with an interviewer watching and half solved it alone in a room. NC State titled the press release “Tech Sector Job Interviews Assess Anxiety, Not Software Skills”. The people who were watched “performed half as well”. And “all of the women who took the public interview failed” while “all of the women who took the private interview passed” (a small group so don’t stretch that one). Parnin’s conclusion was that “companies are missing out on really good programmers”.

Thomas Ptacek ran hiring at the security firm Matasano and wrote up what replaced the interview there in “The Hiring Post” in 2015. It opens with “The software developer job interview doesn’t work.” Matasano’s work samples were “standardized, so that every candidate faces the same test” and graded the same way: “If you’re scoring candidates differently, you’re missing the point.” One hire, Alex, “had Walter White’s resume, but Heisenberg’s aptitude”. He went on to write exploit code that Ptacek says “might be one of just a few implementations in the world” and “Nothing in Alex’s background offered a hint that this would happen.” Side door!!

You won’t be the first to try it. More than 750 companies and teams list themselves on Hiring Without Whiteboards, whose interviews “resemble day-to-day work”, like “pairing on a real world problem or a paid/unpaid take home exercise”. Work samples scored 0.33 in Sackett’s numbers, below the structured interview. So I’d do both. The interview tells you how somebody thinks about the job and the work sample shows you them doing it.

Pay for a short trial on a problem you actually have

Automattic, the company behind WordPress.com, replaces the technical interview for some roles with “a project that simulates real work”. Its longer trial is “up to 20 hours using real work to check mutual fit, with feedback from your hiring team throughout”. Its design team adds that “All Automattic trial projects are paid at a standard rate of $25 USD per hour” (Automattic Design, 2024). I’d pay every time. A 20 hour unpaid take home only gets finished by people with free evenings, which leaves out parents, carers and anyone who already has a job.

The longest trial you have is the one already in the contract. In Germany, where I’ve worked since 2015, a Probezeit of up to 6 months lets either side end things with 2 weeks’ notice (§ 622 BGB). Use those months the way they’re meant: hand the new person something small that’s theirs, watch what they do with it, then hand them something bigger.

Ask what they built when nobody was making them

Rodrigo hired me for a technical test I’d aced and for the initiative I’d shown to get out of Qatar. The second one never shows up on a values rubric. So put it in your structured questions. “What’s something you built that nobody asked you for?” and “What did you do to get into this room?” get scored like everything else. That’s how you find Mukunda’s unfiltered people without hiring at random.

I know how this sounds coming from a guy with no degree who got into tech through a side door. Somebody looked past the outward appearance once, in a small office in Berlin. I’m still here because of it. Somewhere in your pipeline there’s someone with a weird CV and no STAR story. Give them the same questions and the same test as everyone else and see what happens.