The BEAM Edge · For Elixir developers

What Elixir 1.20's type checker catches in AI-written code

We planted 14 API-shape mistakes: Elixir 1.20's type checker flagged 11, 1.19 flagged 3. A module by Claude Sonnet 5.5 compiled clean and passed our tests.

By Agentum R&D · 5 min read
Published Oct 21, 2026 · Updated Oct 21, 2026

Short answer Elixir 1.20's type checker flags dead code and code guaranteed to fail, such as a {:error, _} clause on a function that returns :error. It flagged 11 of our 14 planted mistakes; 1.19 flagged 3. It stays silent on maybe-nil values and request params, so keep tests for those.

An AI coding assistant writes Elixir from memory. When it remembers a function's return shape wrong, say {:error, reason} from a function that returns plain :error, the code still compiles, and the bug waits for the first request that takes that path. Elixir 1.20, released on 3 June, now type-checks every program without any annotations^{Source 1}, and reports dead code and what the core team calls verified bugs: typing violations that are guaranteed to fail at runtime if executed^{Source 1}.

That is the class of mistake a model makes when it guesses wrong. So we measured it. We ran three experiments on Elixir 1.20.4 and, for comparison, 1.19.6: fourteen functions with one planted shape mistake each, a module written by an AI model, and a small project where one module calls another through a guessed API.

§1

Fourteen planted mistakes: 1.20 flags 11, 1.19 flags 3

We wrote fourteen short functions, each with one planted mistake of the guess-the-shape kind, and compiled each one alone with elixirc. A row counts as flagged when the compiler printed a type warning.

Fourteen one-mistake functions compiled with elixirc (our test, October 2026)
Mistake	Elixir 1.19.6	Elixir 1.20.4
`Integer.parse(s) + 1` (it returns a tuple or `:error`)	flagged	flagged
`Integer.to_string(total / 2)` (`/` returns a float)	flagged	flagged
`a > b` on two `DateTime` structs	flagged	flagged
`{:error, _}` clause on `Map.fetch/2` (it returns `:error`)	—	flagged
`{:error, _}` in a `with` else over `Integer.parse/1`	—	flagged
`params.name` on a literal map with string keys	—	flagged
`Map.fetch!(user, :email)` called with a map that has no `:email`	—	flagged
`length(title)` on a binary	—	flagged
`if Date.compare(a, b)` (always truthy)	—	flagged
`Keyword.get(opts, ...)` where `opts` is a map	—	flagged
`String.to_integer(qty)` where `qty` is an integer	—	flagged
`{:ok, body} = File.read!(path)` (it returns the binary)	—	—
`String.upcase(System.get_env("REGION"))` (may be `nil`)	—	—
`Enum.at(users, 0).name` (may be `nil`)	—	—

The eight new catches line up with what the 1.20 changelog adds: typing across the clauses of a case or with, typing of map keys, and inference from guards^{Source 2}. The with case shows what a catch looks like:

Illustrative

Elixir

def parse_qty(s) do
  with {qty, ""} <- Integer.parse(s) do
    {:ok, qty}
  else
    {:error, reason} -> {:error, reason}
  end
end

# warning: the following clause will never match:
#
#     {:error, reason} ->
#
# it is expected to match on type:
#
#     dynamic(:error or {integer(), binary()})

Fig. 1 — Elixir 1.20.4 output, trimmed. The warning names the type the code should have matched. It shows the idea; it isn't production code.

The warning names the shape the function actually returns, which is the fact a model needs to fix its guess.

§2

The model knew the standard library. Your API is another matter

Next we asked Claude Sonnet 5.5 to write a ten-function module from a short spec, in a clean Claude Code session with no tools, so it could not run or compile anything. Its ten tasks mirrored rows of the table: parse a quantity, look up a price, read an environment variable, compare dates, read decoded JSON. The module compiled without a single warning on 1.20.4 and passed all ten of our tests. In this one module, the model got every standard-library shape right.

What a model cannot know from training is your own code. So we wrote a two-module project: an Accounts module whose get_user/1 returns a map or nil, the way Repo.get/2 does, and a Billing module written against a guessed API, matching {:ok, user} and {:error, _}, expecting {:ok, _} from a function that returns :ok, and reading a field that does not exist.

Illustrative

Elixir

case Accounts.get_user(id) do
  {:ok, user} -> user.plan
  {:error, _} -> :free
end

# warning: the following clause will never match:
#
#     {:ok, user} ->
#
# because it attempts to match on the result of:
#
#     Accounts.get_user(id)
#
# which has type:
#
#     dynamic(%{email: binary(), id: integer(), plan: :pro} or nil)

Fig. 2 — Billing written against a guessed Accounts API, Elixir 1.20.4. It shows the idea; it isn't production code.

Both 1.19.6 and 1.20.4 flagged four of the five guesses: the two case clauses, the {:ok, _} match and the unknown key .email_address. Checking across your own modules is not new in 1.20, and it is the check that helps when an agent edits one module and calls another. One guess passed in both versions: calling get_user(id) with the id straight from the request params, a string, although the function only accepts an integer.

§3

What Elixir's gradual typing stays silent about, and why

The silence is by design. The Elixir documentation calls inference best-effort: it does not promise to find every incompatibility, only bugs where all combinations of a type will fail^{Source 3}. Many types the checker infers are ranges, printed as dynamic(...) in the warnings. If the function accepts at least one type in that range, there is no warning^{Source 3}. That rule keeps false positives extremely low^{Source 1}, and it explains every miss in our tests.

System.get_env/1 returns a string or nil, and String.upcase/1 accepts one of the two, so the call passes. Enum.at/2 returns an untyped value that may be nil, so .name on it passes. Request params arrive as untyped data, so a string id or params.name on a function argument passes too; our params.name row was flagged only because the map was a literal. And in a probe, the checker treated the result of File.read!/1 as untyped, so it accepted File.read!(path) + 1 as well.

Put simply: the checker flags code that fails, or a clause that can never match, for every possible value. It stays quiet about code that fails for some values, and nil, outside data and untyped results like File.read!/1's are exactly the "some values" cases. Those still need a test.

§4

Make the type checker part of the agent's loop

Upgrade to 1.20. It needs Erlang/OTP 27 or later^{Source 2}. The checks run during normal compilation; there is nothing to configure.
Fail the build on warnings. mix compile --warnings-as-errors exits with a non-zero status when compilation has any warning^{Source 4}. Run it in CI and in the loop your agent uses to check its own work, and give the agent the warning text: it names the shape the agent guessed wrong.
Turn outside data into known shapes at the edge. Match and convert params in the controller, for example with raw when is_binary(raw) <- params["id"], {id, ""} <- Integer.parse(raw) with an else that returns a 400, so the code behind it has types the checker can follow.
Keep tests for `nil` and for outside data. The checker is silent there by design.
Treat "will never match" in AI-written code as a guessed API, not as noise to silence.

Type signatures come after typed structs on the roadmap, with no date set^{Source 1}. Until then, inference is free: it is on by default, and on our fourteen planted mistakes it caught eleven.

Sources

Elixir v1.20 released: now a gradually typed language (elixir-lang.org, Jun 3, 2026). Retrieved Oct 2026. ↩ ↩ ↩ ↩
Changelog for Elixir v1.20 (HexDocs). Retrieved Oct 2026. ↩ ↩
Gradual set-theoretic types (Elixir 1.20.4 documentation). Retrieved Oct 2026. ↩ ↩
mix compile.elixir (Mix 1.20.4 documentation). Retrieved Oct 2026. ↩

Published Oct 21, 2026 · Updated Oct 21, 2026Report an error (support@agentum.cloud)