Notable verbatim quotes

Each quote is verbatim from transcript.md. Section references point to the section headings in transcript.md. All quotes are Farley's unless otherwise noted.

On vibe coding / natural language as a programming medium

1. [§3 Provocations] "I would argue that vibe coding programming with natural languages. While having a place. Are also kind of bad ideas."

2. [§8 Natural language fails] "Vibe coding alone is simply not good enough if we're just chatting with the computer to express our needs. That's not enough."

3. [§8 Natural language fails] "somebody said today, I think they quoted Andrej Karpathy saying that the programming language of the future is English. I don't like that idea very much for the reasons that I'm talking about. It's not precise enough. It's too vague."

4. [§8 Natural language fails] "Simple, consistent formal grammar. Does natural language give us that? No, it's complex and inconsistent. Unambiguous expression of intent. No, it's ambiguous and vague. Repeatable deterministic execution. Not repeat or not deterministic."

On tests

5. [§4 What are tests for?] "what tests are for us, they're our form of measurement. They are equivalent of a carpenter having a tape measure in his pocket that he can measure things with."

6. [§4 What are tests for?] "we can't infer the goals from the solution because the solution can always be wrong."

7. [§4 What are tests for?] "AI generated tests. If the code is the only input, we can only verify that the code remains the same. … So they're mostly a dumb idea. They have a, they have a place, but mostly a dumb idea. They tend to be a copper [cop-out] for people who don't, can't be bothered to state their goals."

8. [§15 Conclusions] "Automating testing from a solution is a bit of a joke. It's a niche. It has some utility, but it's a corner case. It doesn't solve the real problem."

On what a program / programming language is for

9. [§6 Three goals] "Programming language, I would argue, have three goals as tools. So first, they are tools that have been designed to help us to organize our thinking about a problem. … They're also a means of communicating our understanding with other programmers, other humans. … And ultimately, they're there to tell a computer what to do. But that's kind of the last part."

10. [§7 Three techniques] "we have a simple, consistent grammar … They are unambiguous expression of our intent … And they're repeatable and deterministic in terms of execution. And this matters a lot, too."

On the three problems AI introduces

11. [§9 Three problems] "how do we specify what we want with precision? That's problem one. Problem two is how do we confirm that we got what we wanted? This is the verification problem. And problem three is that we don't work in the same way as the machines."

12. [§9 Three problems] "high quality system development. In software is always an incremental process of learning and discovery."

13. [§9 Three problems] "We sped up the coding bit. That was the easy part of software development … But it also moves the bottleneck. If you've ever read The Goal, the theory of constraints, that's what we've done."

On scale and verification

14. [§10 12,000 lines/day] "He reckons he may, he writes 12,000 lines of code per day. No human being can review 12,000 lines of code per day. No human being can test, manually test the output of 12,000 lines a day behaviorally to figure out whether it's doing the right things."

On Farley's proposed approach

15. [§11 Future specification] "A program will be a precise description of what it is that we want, I think, and coded as specifications translated into execute or [executable] instructions by the AI that will verify that we got what we wanted."

16. [§12 Future verification] "The way that I do it is that I teach my AI how to do the kind of BDD style tests that I like to use. And I specify what I want. And my AI fills in the gaps from a relatively simple English descriptions of what it is."

17. [§12 Future verification] "We can verify that the AI is doing the right thing by giving test test values that it hasn't seen before, so it can't cheat the tests."

18. [§14 Worked example] "we start off with the vanquish [vision] of what we want. We get to a slightly more formal version of describing that, which is a user story. And then we come up with some examples that would demonstrate that the feature of the story describes exists. Those are our examples, those are our executable specifications."

On AI as compiler / fifth-generation programming

19. [§15 Conclusions] "effectively what we're doing is we're … invoking a fifth generation programming language, a fifth generation programming system."

20. [§15 Conclusions] "AI assistance is rather like the compiler. We're not going to care for very much longer at all if we do now. About the code that it generates because we'll verify that we've got the results. And as long as we get the results, who cares about the implementation?"