# What can change after abliteration? Three capabilities to test

Canonical: https://refusalmap.com/articles/capabilities-after-abliteration/
Author: Refusal Map Editorial
Published: 2026-11-02
Category: Evaluation

An abliterated model should be checked for task accuracy as well as willingness to respond. Long reasoning, nested instructions and strict output formats expose different kinds of regression.

## Does less refusal mean better reasoning?

No. Refusal behavior and task competence are different measurements. A model may produce a longer answer without solving the problem. Compare a known final answer and an explicit rubric rather than treating fluent prose as evidence of preserved capability. For a multi-step task, keep the input length and reasoning budget consistent. The [40-prompt evaluation protocol](/articles/abliteration-40-prompt-protocol/) provides a starting set of checkable tasks.

## Long reasoning: inspect the final result

Choose problems with checkable answers, such as a small scheduling puzzle or a calculation with several dependent steps. Record whether the final result is correct, whether the answer contradicts its own premises and whether generation hits the token limit. An output truncated by a small budget should not be interpreted as a clean reasoning failure without a follow-up run.

## Nested instructions: make constraints observable

Ask for an ordered list with a fixed number of items, a forbidden word and a required final line. Check every constraint separately. This lets you distinguish a model that forgets a late instruction from one that misunderstands the main task. Keep the instruction order identical across configurations, because recency and chat-template differences can change the result. When a release claims preserved instruction following, use our [model-card checklist](/articles/reading-abliterated-model-cards/) to inspect its evidence.

## Strict formats: validate mechanically

For JSON, test that the result parses and that its keys and value types match the requested schema. A JSON object wrapped in Markdown fences is a format failure if the contract requests raw JSON. Keep content correctness as another score: a perfectly parsed object can contain the wrong calculation.

## Turn the findings into a deployment decision

Weight tasks by your actual workload. If an application consumes JSON, a small format regression may cost more than a large improvement in conversational style. Keep a holdout set that was not used to select the intervention. Re-run it after changing quantization, serving software or the checkpoint. There is no universal acceptance threshold that fits every application. A deployment may also change precision; the [abliteration and quantization comparison](/articles/abliterated-vs-quantized/) separates those effects.

## Reference table

| Capability | Useful check | Misleading shortcut |
| --- | --- | --- |
| Long reasoning | Verified final answer | Answer length |
| Nested instructions | Constraint-by-constraint score | Looks compliant |
| Strict format | Parser plus schema check | Contains braces |

## Sources

- [Arditi et al. (2024): Refusal in Language Models Is Mediated by a Single Direction](https://arxiv.org/abs/2406.11717)

## Related reading

- https://refusalmap.com/articles/abliteration-40-prompt-protocol/
- https://refusalmap.com/articles/abliterated-vs-quantized/
- https://refusalmap.com/articles/reading-abliterated-model-cards/