InfrastructureCompute, chips & cloud
Hugging Face and AWS Launch a Public Lookup Showing Which Hub Models Run Unmodified on Trainium
Hugging Face and AWS released a public Space and dataset that record whether popular Hub models run unmodified on Trainium through the new TorchNeuron PyTorch backend. Each record includes scripts and compiler versions, but a pass checks only a short generation.
From the Hugging Face article of October 23, 2026 and cited sources: The model-support dataset records a pass/fail result for each tested model. [1] [6]
SOURCES[1] Does Your Model Run on AWS Trainium? Look It Up — Hugging… huggingface.co[6] aws-neuron/torch-neuronx-model-support dataset huggingface.co
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- Original source
- Hugging Face article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- October 23, 2026 article published
- Covered
- Oct 24 edition
- Evidence
- 9 sources across 8 sites
- 1Does Your Model Run on AWS Trainium? Look It Up — Hugging…huggingface.co
- 2AWS announces TorchNeuron (Public Beta) — AWS What's Newaws.amazon.com
- 3TorchNeuron explained — eeseleesel.ai
- 4Amazon's TorchNeuron runs PyTorch on AWS Trainium chips —…aboutamazon.com
- 5Amazon's TorchNeuron lets PyTorch code run on AWS Trainium…thenextweb.com
- 6aws-neuron/torch-neuronx-model-support datasethuggingface.co
Reach: not measured yet
The model-support dataset records a pass/fail result for each tested model. It adds the exact script, the neuronx-cc and Transformers versions, the core count, the wall time including compilation, and peak device memory. There is one Parquet file per version pair. For now it covers only Transformers text-generation and image-text-to-text models, with Diffusers and sentence-transformers planned. [1] [6]
The lookup runs on TorchNeuron, an open-source native PyTorch backend for Trainium2 and Trainium3 that entered public beta on 20 October 2026. It supports eager mode, torch.compile and NKI kernels. It requires static shapes, sends unsupported ops to the CPU, and does not support custom CUDA ops, Triton kernels or bitsandbytes. [2] [7] [8] [9]
A PASS means the model produced 32 checked tokens on at most 16 cores. Single-core runs are compared token by token with a CPU run, while sharded runs are only judged for fluent English. A PASS does not cover throughput, long generations or training, and quantized and remote-code models are excluded. [1] [6]
anyone can now check and reproduce compatibility for each compiler and library version. Implication: teams can screen models before committing to Trainium, but they still need their own throughput, latency and cost tests at real sequence lengths.
I've finished the research and here is the dossier.
Executive brief
Hugging Face and AWS now publish, as an open dataset, a pass/fail record of whether popular Hub models run unmodified on AWS Trainium. Each record comes with the exact script and compiler version used (HF blog). The lookup Space sits on top of TorchNeuron, AWS's new native PyTorch backend, which went into public beta on 20 October 2026 (AWS). This makes "does it run?" something anyone can check. But a PASS only means the model produced 32 checked tokens on at most 16 cores. It says nothing about speed, long generations or training.
What changed and event timeline
TorchNeuron announced
The PyTorch blog announced the backend a day before launch (). Hugging Face published "TorchNeuron: running HF models on AWS Trainium is easier than ever" the same day ().
Public beta at PyTorch Conference
AWS released TorchNeuron as open source for Trainium2 and Trainium3. It supports eager mode,
torch.compileand NKI kernels (,).Press flags gaps
Coverage reported that dynamic input shapes are unsupported and that about 22,000 PyTorch tests are skipped. Named early users include Cartesia, Deepgram and Hugging Face ().
Model-support Space and dataset launched
Hugging Face launched a lookup Space backed by the public
aws-neuron/torch-neuronx-model-supportdataset, with one Parquet file per compiler/library version pair (,).
Capabilities and access
- Software: TorchNeuron (
torch-neuronx) is in beta for PyTorch 2.14.0 with Neuron SDK 2.33 and Python 3.12–3.14, under the Apache-2.0 license (GitHub). - Space views: Selected, Trending and Downloads, filterable by
neuronx-ccand Transformers version. - Each row shows: result, number of cores, wall time (compilation included) and peak device memory.
Read the full section
- Software: TorchNeuron (
torch-neuronx) is in beta for PyTorch 2.14.0 with Neuron SDK 2.33 and Python 3.12–3.14, under the Apache-2.0 license (GitHub). The release notes list PyTorch 2.13 or 2.14 (release notes). - Space views: Selected, Trending and Downloads, filterable by
neuronx-ccand Transformers version. - Each row shows: result, number of cores, wall time (compilation included) and peak device memory.
- Scope: Transformers text-generation and image-text-to-text models only. Diffusers and sentence-transformers are planned (HF blog).
- Faster first runs: you can pull precompiled graphs from a public NEFF cache bucket with
hf buckets sync, which skips the first compile.
Technical analysis for researchers and developers
- Recipe: load with
from_pretrained, shard if needed, then generate with a static cache andtorch.compile(backend="neuron"). - Sharding: each model goes on the fewest cores that hold its weights. Most use tensor parallelism.
- Pass check:
Read the full section
- Recipe: load with
from_pretrained, shard if needed, then generate with a static cache andtorch.compile(backend="neuron"). The script starts its own distributed ranks, sotorchrunisn't needed (HF blog). - Sharding: each model goes on the fewest cores that hold its weights. Most use tensor parallelism. Mixture-of-experts models without a tensor-parallel plan use expert parallelism only.
- Pass check:
- On a single core, the output tokens are compared with a CPU run.
- When sharded, a small judge model only checks that the text is fluent English. That is a weaker test.
- Failure codes: FAIL(load), FAIL(gen), FAIL(output), FAIL(other) and EXCLUDED. Quantized checkpoints and models that need remote code are excluded. Each sweep covers about 100 models (dataset card).
- Backend limits: shapes must be static (
dynamic=False), tensors must be contiguous, and float64 runs as float32 (GitHub).
Claims and evidence
- "Single configuration change, no rewrite." This is Amazon's claim (About Amazon).
- 89% of the top 100 text-generation architectures work out of the box, versus 46% under Optimum Neuron v0.4.5. Hugging Face reported this (eesel).
- About 60% more audio processed per dollar than GPUs. This is Deepgram's customer claim (eesel).
Read the full section
- "Single configuration change, no rewrite." This is Amazon's claim (About Amazon). One analysis counters that you still have to change device strings, set the compile backend, remove CUDA-only options and rethink sharding (explainx).
- 89% of the top 100 text-generation architectures work out of the box, versus 46% under Optimum Neuron v0.4.5. Hugging Face reported this (eesel). It comes from the team that built the integration, not from independent testing (explainx).
- About 60% more audio processed per dollar than GPUs. This is Deepgram's customer claim (eesel).
- No independent benchmarks of throughput or compatibility appeared in the reviewed sources. The public dataset does make the compatibility claims re-runnable.
Context and prior work
- Before: Hugging Face users reached Trainium through Optimum Neuron, a separate library.
- How it plugs in: TorchNeuron adds a
neurondevice to standard PyTorch through PyTorch's PrivateUse1 extension point (eesel). - Strategy: Amazon says its custom-chip business has passed a $25B annual revenue run rate (About Amazon).
Read the full section
- Before: Hugging Face users reached Trainium through Optimum Neuron, a separate library. Hugging Face now recommends running Transformers, Diffusers, Accelerate, PEFT and TRL directly on the device (eesel).
- How it plugs in: TorchNeuron adds a
neurondevice to standard PyTorch through PyTorch's PrivateUse1 extension point (eesel). Google's TorchTPU, covered on the HF blog on 20 October, takes a similar native-backend approach (HF blog). - Strategy: Amazon says its custom-chip business has passed a $25B annual revenue run rate (About Amazon).
Limitations, safety and contested findings
- What a PASS covers: a PASS excludes throughput and long generations. Testing is capped at 4 Trainium2 devices (16 cores) (HF blog).
- Backend gaps:
- Dynamic shapes need padding or bucketing.
Read the full section
- What a PASS covers: a PASS excludes throughput and long generations. Testing is capped at 4 Trainium2 devices (16 cores) (HF blog).
- Backend gaps:
- Dynamic shapes need padding or bucketing.
- Most FFT transforms fall back to the CPU.
- TorchAO and
torch.funcare unsupported. - Unsupported ops fall back to the CPU, which you can track with
get_fallback_ops()(release notes). - Third-party code: custom CUDA ops, Triton kernels and bitsandbytes are not supported (explainx).
- Conflicting reports on repo status: explainx found the GitHub repo returning 404 around launch; the repo is now public (GitHub).
Business and practitioner implications
- Query the dataset before committing to Trainium:
pd.read_parquet("hf://datasets/aws-neuron/torch-neuronx-model-support/data"). Pin the sameneuronx-ccand Transformers versions that were tested (HF blog). - A PASS is only a first step. Customer per-dollar figures are not a basis for procurement.
- The first compile is slow unless you use the NEFF cache. Static shapes favour fixed-shape serving.
Read the full section
- Screen models first. Query the dataset before committing to Trainium:
pd.read_parquet("hf://datasets/aws-neuron/torch-neuronx-model-support/data"). Pin the sameneuronx-ccand Transformers versions that were tested (HF blog). - A PASS is only a first step. Run your own throughput, latency and cost tests at your real sequence lengths. Customer per-dollar figures are not a basis for procurement.
- Plan for compilation. The first compile is slow unless you use the NEFF cache. Static shapes favour fixed-shape serving.
- Where it fits today: quantized, remote-code and very large models (over 16 cores) are outside the tested set, so the evidence mostly covers small-to-mid open models.