Elektrine lite

← Feed

@tao@mathstodon.xyz

2026-06-21 17:53 UTC

In the last few weeks, though, autoformalization has reached the point where virtually every formalization task I had issued could be completed within hours. However, the formalization tools tended to create quite bloated proofs, often hundreds of lines longer than what a human would choose to do, with a lot of redundancy, with many lemmas not stated at the natural level of abstraction. For any individual result, this is not a major problem - the proofs here are not intended to be read by humans anyway, but because each such proof adds tens of seconds to the total build time, the cumulative effect of such large AI-generated proofs has become noticeable, and somewhat difficult to review. In particular, the process of converting a medium-sized Lean document (containing a few thousand lines of code, with some proofs AI-generated) into a nicely golfed and structured Mathlib submission has been an interesting experience. AI agents can be used to perform local golfs that can shave the size of the code somewhat, but global refactoring decisions, such as noticing that a certain argument appears multiple times across the document and can be abstracted into a standalone lemma that can may have additional utility beyond the file, is still largely beyond the reach of current AI tools. (I find that I can *explain* such a refactor to an AI agent, who can then execute it, but they struggle to spontaneously *discover* such refactors on their own.) (2/3)

Replies (2)

  • @tao@mathstodon.xyz 2026-06-21 17:53

    On the one hand, this means that the IEANTN project is now moving significantly faster than previously anticipated, with the queue of outstanding formalization unclaimed issues in the project now essentially empty. But I will also have to spend more time scoping the IEANTN formalization tasks in anticipation of rapidly recieving an AI-generated proof, in order to make it easier to review and more compatible with the global structure of the project. (3/3)

    Open ##5165525

  • @nh@mastodon.gamedev.place 2026-06-21 18:04

    @tao@mathstodon.xyz FWIW, this rhymes with my experience with using agentic coding, which is also substantiated with conversations with colleagues.

    Open ##5165526