Independent and unofficial. The Test Fixture is not affiliated with, endorsed by, or operated by any model developer or platform mentioned here. LTX-2.3 and related marks belong to their respective owners.
A failure case is not the part of the test that went wrong. It is the part of the test that told you something.
That sentence is easy to agree with and hard to practice. When a run produces a bad clip, the instinct is to fix the prompt, re-roll the seed, and move on. Do that enough times and you end up with a folder of good outputs and no idea what the system actually does when conditions are less friendly. You have produced a highlight reel with extra steps.
This post is the opening statement for the Failure Analysis category. It is not a test report. It is the rule that test reports on this site will follow.
The test question behind this category
When a generated video fails, what failed, under what conditions, and can the failure be made to happen again?
That last clause is what separates failure analysis from complaining. A failure that cannot be reproduced is still worth recording, but it belongs in a different column. A failure that can be reproduced is evidence. It can be compared across versions, across hardware, and across workflows.
Why failures get buried
Three reasons, and all three are structural rather than malicious.
The output is selected after the fact. A creator generates twenty clips, posts the best one, and calls it a workflow. Nothing in that sequence is dishonest, but the selection step is invisible to the reader. The result is a public record of successes and a private record of failures. That is the opposite of a test report.
The failure is hard to describe. “It looked wrong at second three” is a real observation and a useless one. It does not say what was wrong, how many frames, whether it recovered, or whether it happened in other runs. Without a vocabulary, failures get dropped because they cannot be written down quickly. This is why the failure categories below exist.
Failure is treated as a verdict on the model. If a bad clip means “the model is bad,” then publishing failures feels like a brand attack, and readers who like the model push back. But most failures are not verdicts. They are conditional observations. A model that flickers at 720p after frame 48 under one motion setting has told you something specific and useful. It has not told you the model is bad. It has told you where the edge is.
What counts as a failure
On this site, a failure is any output that does not satisfy the stated test question under the stated conditions. That definition is deliberately narrow.
It is not “a clip I did not like.” Aesthetic judgment is a separate axis and I will not smuggle it into failure analysis. It is not “a clip that looks worse than a curated demo.” Demos are curated by definition. It is not “a clip that failed to match a prompt I did not specify precisely enough.” That is a test-design problem, and it gets recorded as one.
A failure is conditional, reproducible where possible, and categorized. Everything else is an observation awaiting a better test.
The failure categories
The categories are stable across tests so that failure patterns can be compared between versions instead of rebuilt from scratch each time.
Temporal flicker. Luminance, texture, or detail instability across frames that is not present in the source or the prompt. Recorded with frame ranges where possible.
Identity drift. The subject changes in ways the prompt did not request: face, clothing, proportions, or object identity across frames or shots.
Object deformation. Geometry or structure degrades: limbs bend incorrectly, objects merge, architecture warps.
Camera instability. Motion that should be smooth jitters, reverses, or breaks continuity.
Prompt contradiction. The prompt asks for things that conflict, and the output resolves the conflict in a way that fails the test question. This is often a test-design failure, and it is recorded as one when it is.
Render failure. Crash, timeout, out-of-memory, corrupted output.
Workflow-induced failure. The model may be fine; the setup caused the problem. Wrong node version, mismatched resolution, bad frame interpolation, a dependency that changed behavior.
Irreproducible success. An impressive output that cannot be made to happen again under recorded conditions. This is a failure of the test, not of the model, and it gets its own report.
The distinction between the last two and the rest matters. A model limitation and a workflow mistake call for different fixes, and a report that blurs them is not useful to anyone trying to build something.
How a failure gets recorded
Every failure entry on this site will answer the same questions:
Test question. What was the run supposed to show?
Conditions. Configuration block reference, seed, run number, hardware and software versions.
What failed. Category, frame range or timestamp, and a plain description before any interpretation.
How often. Sample count and failure rate, not a single anecdote presented as a pattern.
Reproducible? Yes, no, or partial, with the reproduction attempt described.
Attribution. Model, workflow, test design, or undetermined. Undetermined is a valid answer and will be used.
What would change the conclusion. The condition that, if different, would make this failure disappear or change category.
That last field is the one that keeps a failure report honest. If I cannot say what would change the conclusion, I probably do not understand the failure yet.
The irreproducible success
This deserves its own paragraph because it is the most underreported case in AI video.
An impressive clip with no conditions attached is not a capability result. It is a data point with the context stripped off. When I produce one and cannot reproduce it, I will report it as an irreproducible success, with the same rigor as a failure. It goes in the log. It does not go in the highlight reel without a label.
A successful sample is evidence. It is not a warranty.
What this category will not do
It will not publish unsupported brand criticism or competitor-bashing. A failure report is about conditions, not about loyalties. It will not treat one bad clip as a universal conclusion. It will not bury limitations below a call to action. It will not quietly delete a failed test when a later version fixes it; the fix gets a note, and the original report stays.
That is an observation, not yet a conclusion. The failure case is where the next test question comes from.
The configuration is part of the result.
If it can’t be reproduced, it’s a story—not a result.