CtrlK
BlogDocsLog inGet started
Tessl Logo

ainativedev/latest-aidevcon-speakers-london-2026

AI Native DevCon 2026 London — all conference sessions as interactive skills

71

Quality

89%

Does it follow best practices?

Impact

—

No eval scenarios have been run

SecuritybySnyk

Risky

Do not use without reviewing

Overview
Quality
Evals
Security
Files

outline.mdtalk-cormack-tests-lie-observability-ai-honest/

Outline - When Tests Lie: Using Observability to Keep AI Honest

Speaker

Justin Cormack

Abstract

[inferred from filename and transcript] When Tests Lie: Using Observability to Keep AI Honest is an AI Native DevCon session covering tests and observability, AI-generated software validation, runtime signals, test reliability, and keeping AI honest.

Thesis

[inferred] The talk's main contribution is its framing of tests and observability, AI-generated software validation, runtime signals for practitioners working with AI-native software development.

Transcript Status

The source is timestamped speech-to-text output. Speaker labels, punctuation, and some technical terms may be imperfect. Use timestamps when citing.

Timeline

#Timestamp rangeSectionSummary
100:00-03:41Opening and framingThis this feedback loop that we really need to understand if AI is is truly helping us in our production code or not.
203:42-07:31Main discussion 1It's like, if we can do this in less time, we can build more of these interesting systems.
307:35-11:49Main discussion 2And you can build a test suite against a simple version first.
411:54-15:57Main discussion 3I kept finding edge cases by reading the AWS documentation thinking.
516:00-19:48Main discussion 4we just had, and it would come up with new kinds of tests.
619:55-23:46Main discussion 5I mean, it can either fail to reproduce itself or it can guess what the solution might be and get it wrong or something.
723:49-28:12Main discussion 6I mean Codex security reviews, pull requests, which is fine.
828:14-31:49Closing pointsI'm I want to next I'm looking at basically I mean I.

Named Concepts / Search Anchors

  • Tests And Observability - Topic named in the talk metadata and used as a search anchor for transcript Q&A.
  • AI-Generated Software Validation - Topic named in the talk metadata and used as a search anchor for transcript Q&A.
  • Runtime Signals - Topic named in the talk metadata and used as a search anchor for transcript Q&A.
  • Test Reliability - Topic named in the talk metadata and used as a search anchor for transcript Q&A.
  • Keeping AI Honest - Topic named in the talk metadata and used as a search anchor for transcript Q&A.

Useful Search Terms

  • tests and observability
  • AI-generated software validation
  • runtime signals
  • test reliability
  • keeping AI honest

Open Questions / Limits

  • The outline is generated from timestamped transcript text rather than speaker-provided slides.
  • Some transcript terms may be speech-to-text artifacts.
  • For precise claims, inspect transcript.md around the relevant timestamp.

Duration Marker

Last observed timestamp: 31:49

talk-cormack-tests-lie-observability-ai-honest

outline.md

quotes.md

SKILL.md

transcript.md

README.md

tile.json