These tasks were added from community reports about how LLMs mishandle Nix and NixOS work.
Corpus 3.0 archived the saturated 2.0 tasks under archive/2.0/ (see the 3.0.0 release note), corpus 3.1 archived 24 saturated 3.0 tasks under archive/3.0/ (see the 3.1.0 release note), and corpus 3.2 archived 9 saturated 3.1 tasks under archive/3.1/ (see the 3.2.0 release note). Their rows stay here for provenance and are marked archived. Eight community-derived tasks were added in 3.0; they were selected from task-candidates-community.md, which holds the fuller analysis of each source.
Corpus 3.1 adds a second kind of source: changes in the pinned nixpkgs that a model trained before mid-2026 is unlikely to know. Those tasks are listed in Fresh nixpkgs material (3.1) below. Corpus 3.2 adds a third: changes whose old form still evaluates cleanly but produces a wrong system, and multi-host regression traps. Those tasks are listed in Silent-failure material (3.2).
Since corpus 3.0 the source context lives only in this document. Task prompts no longer link to the originating threads, because those threads usually contain the accepted fix and would hand the agent the answer direction. Prompts describe the observed symptom and the hard constraints instead; see Prompt rewrite 3.0.
This list should be refreshed weekly from new NixOS Discourse issues, especially threads that show concrete agent failures, incorrect advice, maintainer corrections, or novel Nix/NixOS constraints that fit a deterministic benchmark.
| Task | Status in 3.2 | Modeled complaint | Source context |
|---|---|---|---|
module-stale-option-migration | Archived in 3.0 (archive/2.0/) | LLMs suggest deprecated or removed NixOS option paths from older releases. | Reddit: https://www.reddit.com/r/NixOS/comments/1k0s6co/nixos_llms_is_really_exciting/ and https://www.reddit.com/r/NixOS/comments/1dkgkso/best_language_model_for_nix_language_questions/ |
package-name-lookup-contract | Archived in 3.0 (archive/2.0/) | LLMs make up package names or suggest names that are absent from the available package set. | Reddit: https://www.reddit.com/r/NixOS/comments/1rqy6h0/nixos_migration_was_relatively_easy_using_llm/ and https://www.reddit.com/r/NixOS/comments/1n6er8g/nixos_for_a_beginner_in_2025/ |
container-native-vs-oci | Archived in 3.0 (archive/2.0/) | LLMs mix native NixOS containers with Docker/Podman-style container abstractions. | Reddit: https://www.reddit.com/r/NixOS/comments/1ga6dbe/examples_of_questions_you_asked_an_llm_about_nix/ |
string-escaping-systemd | Archived in 3.1 (archive/3.0/) | LLMs often mishandle Nix string escaping when shell code must keep runtime variables literal. | NixOS Discourse: https://discourse.nixos.org/t/local-ai-agent-for-nixos-configuration-looking-for-experience-reports/77238 |
debug-network-false-lead | Archived in 3.0 (archive/2.0/) | LLMs chase plausible diagnostic rabbit holes instead of following observed evidence. | NixOS Discourse: https://discourse.nixos.org/t/nixos-hard-locks-driving-my-crazy/72210 and https://discourse.nixos.org/t/broken-state-of-the-system-after-25-11/73249 |
python-cuda-uv2nix-patch | Archived in 3.0 (archive/2.0/) | LLMs struggle with NixOS flakes, Python-on-NixOS, CUDA packaging, and suggest generic Linux fixes. | NixOS Discourse: https://discourse.nixos.org/t/setting-deep-learning-python-project-with-cuda-support-using-uv2nix/72028 |
issue-report-quality | Archived in 3.0 (archive/2.0/) | AI-generated reports can be low-signal, unverified, and costly for maintainers to triage. | GitHub: https://github.com/NixOS/nixpkgs/issues/410741 and NixOS Discourse: https://discourse.nixos.org/t/proposal-to-have-an-ai-usage-policy/75650 |
home-manager-extra-special-args | Archived in 3.0 (archive/2.0/) | LLMs often pass flake inputs into nixosSystem and assume Home Manager modules receive them automatically, missing the need for home-manager.extraSpecialArgs. | NixOS Discourse: https://discourse.nixos.org/t/pass-specialargs-to-the-home-manager-module/33068 and https://discourse.nixos.org/t/infinite-recursion-that-i-cannot-figure-out/71077 |
home-manager-wsl-module-import | Archived in 3.0 (archive/2.0/) | LLMs fail to distinguish Home Manager's standalone entrypoint from importing Home Manager as a NixOS module, especially in WSL setups. | NixOS Discourse: https://discourse.nixos.org/t/cant-enable-home-manager-in-wsl-nixos/49364 |
home-manager-xdg-files | Archived in 3.0 (archive/2.0/) | LLMs suggest imperative directory creation, symlink farms, or shell snippets for home folders and $HOME/.config files instead of Home Manager's XDG/file options. | Reddit: https://www.reddit.com/r/NixOS/comments/175a3e6/is_it_right_chatgpt_how_to_create_the_documents/ and NixOS Discourse: https://discourse.nixos.org/t/download-config-files-to-home-dir/60934 |
flake-input-package-selection | Archived in 3.0 (archive/2.0/) | LLMs assume every flake package set has packages.${system}.default and miss the actual named package output. | NixOS Discourse: https://discourse.nixos.org/t/error-attribute-default-missing/59474 |
module-system-boundaries | Archived in 3.1 (archive/3.0/) | LLMs blur NixOS, Home Manager, and nix-darwin module systems instead of sharing plain data and writing separate modules. | Reddit: https://www.reddit.com/r/NixOS/comments/1gwgzbd/nixos_on_macos_nix_not_picking_up/ and https://www.reddit.com/r/NixOS/comments/1rsassa/welcome_to_den_v0120/ |
overlay-module-boundary | Archived in 3.0 (archive/2.0/) | LLMs and generic advice sometimes misuse overlays as if they could also define module options such as systemd.user.*, instead of separating package overrides from module config. | NixOS Discourse: https://discourse.nixos.org/t/overlay-with-user-systemd-unit-and-timer/9295 |
fhs-binary-wrapper | Archived in 3.0 (archive/2.0/) | Generic Linux or LLM advice suggests creating global FHS directories on NixOS instead of wrapping, patching, or using AppImage/FHS tooling. | Reddit: https://www.reddit.com/r/NixOS/comments/17tw9ik/filesystem_hierarchy/ and NixOS Discourse: https://discourse.nixos.org/t/newbie-question-about-running-appimage/41564 |
mutable-config-home-manager | Archived in 3.0 (archive/2.0/) | LLMs suggest managing mutable GUI application profile files as read-only Home Manager files, which breaks apps that need writable state. | NixOS Discourse: https://discourse.nixos.org/t/purpose-of-authentications-option-home-manager/76999 and https://discourse.nixos.org/t/strategies-for-declarative-approaches-to-programs-with-mutable-configuration-files/66276 |
rust-no-network-build | Calibrating | LLMs try to fix Rust build-script downloads by weakening sandboxing or leaving runtime downloads in build.rs, instead of pinning assets and using system libraries. | NixOS Discourse: https://discourse.nixos.org/t/buildrustpackage-fails-to-download-prebuild-binary/78010 and https://discourse.nixos.org/t/rust-package-appears-to-need-network-during-build/74334 |
nushell-command-not-found | Archived in 3.0 (archive/2.0/) | LLMs assume Bash shell initialization is enough for all shells, missing Nushell's separate hooks.command_not_found configuration model. | GitHub: https://github.com/NixOS/nixpkgs/issues/532042 |
xdg-portal-merge | Archived in 3.1 (archive/3.0/) | LLMs overwrite portal configuration for a single Wayland compositor instead of preserving existing system/Home Manager portal packages and fallbacks. | NixOS Discourse: https://discourse.nixos.org/t/hyprland-home-manager-module-breaks-portal-discovery-for-other-desktop-environments/78042 and GitHub: https://github.com/nix-community/home-manager/issues/9201 |
argument-default-forwarding | Archived in 3.1 (archive/3.0/) | LLMs and hand-written wrappers forward an args@{ ... } set and expect parameter defaults to travel with it. | NixOS Wiki: https://wiki.nixos.org/w/index.php?title=Nix_Language_Quirks&oldid=31991#Default_values_are_not_bound_in_@_syntax and nix.dev: https://nix.dev/guides/best-practices#with-scopes |
mkforce-across-option-reexport | Archived in 3.1 (archive/3.0/) | Raising the priority of one definition does not resolve a conflict reported on a different option that the value is forwarded into. | NixOS Discourse: https://discourse.nixos.org/t/solved-cannot-set-environment-sessionvariables-lib-mkforce-not-working/66453 |
module-generated-jobs-fixed-point | Calibrating (new in 3.0) | Generating configuration from an option value inside the same module recurses once the list comes from config. | NixOS Discourse: https://discourse.nixos.org/t/nixos-module-infinite-recursion-when-using-map-over-config-value-in-config-block/76977 and https://discourse.nixos.org/t/trouble-understanding-why-only-one-of-two-really-similar-lines-produce-an-infinite-recursion/63890 |
python-build-backend-false-lead | Archived in 3.1 (archive/3.0/) | ChatGPT attributed a Python package failure to the interpreter version instead of reading the error. | NixOS Discourse: https://discourse.nixos.org/t/can-anyone-get-whisper-overlay-working/70049 |
follows-preserve-python-compatibility | Archived in 3.1 (archive/3.0/) | Pointing an input's nixpkgs at a newer release with follows broke a Python dependency the input relied on. | NixOS Discourse: https://discourse.nixos.org/t/nixpkgs-follows-and-breaking-change-in-a-python-library/70954 |
runtime-tools-outside-devshell | Archived in 3.1 (archive/3.0/) | A Claude Code-generated nixpkgs PR made runtime tools available only at build time. | GitHub: https://github.com/NixOS/nixpkgs/pull/424155 and https://github.com/NixOS/nixpkgs/pull/424155#discussion_r2218059433 |
package-offline-test-selection | Archived in 3.1 (archive/3.0/) | In an LLM-assisted packaging PR, a dependency's tests needed internet access and the proposed answer was to disable them. | GitHub: https://github.com/NixOS/nixpkgs/pull/512128 and https://github.com/NixOS/nixpkgs/pull/512128#discussion_r3122098541 |
editor-project-state-isolation | Archived in 3.2 (archive/3.1/; new in 3.0, evaluator repaired for 3.1) | Per-project wrapped VSCodium shells reused the first project's editor instance and extension set; the AI-assisted explanation was partly unverified. | NixOS Discourse: https://discourse.nixos.org/t/per-project-devshell-vscodium-with-project-specific-extensions-shared-home-manager-user-settings/78807 |
Archived in 3.1
Corpus 3.1 archived these 24 saturated 3.0 tasks under archive/3.0/, each with a pooled solve rate of 0.9 or more in the 3.0 calibration. Their deprecation records are in corpus/task-deprecations.toml. Nine of them have rows in the table above; the others were designed traps from task-selection-3.0.md or tasks carried over from 2.0.
argument-default-forwarding, debug-freeform-config-cycle, debug-import-argument-cycle, fetcher-fixed-output-identity, flake-nested-follows-identity, follows-preserve-python-compatibility, lazy-recursive-update-closure, lazy-selected-report-validation, mkforce-across-option-reexport, module-deferred-schema-default, module-migration-assertion-gate, module-priority-submodule-apply, module-service-options, module-system-boundaries, overlay-composed-final-prev, package-offline-test-selection, purity-explicit-release-inputs, python-build-backend-false-lead, runtime-tools-outside-devshell, scope-override-transitive-dependencies, source-filter-traversal-stability, string-command-dependency-context, string-escaping-systemd, xdg-portal-merge.
Archived in 3.2
Corpus 3.2 archived these 9 saturated 3.1 tasks under archive/3.1/, each with a pooled solve rate of 0.9 or more in the 3.1 calibration. Their deprecation records are in corpus/task-deprecations.toml. Five of them have rows in the tables on this page; the others were designed full-system tasks.
cgit-export-policy-agreement, debug-infinite-recursion-fleet, editor-project-state-isolation, home-assistant-declarative-dashboards, home-manager-nixos-module-boundary-fleet, nginx-acme-virtualhost-migration, rust-workspace-vendored-deps, systemd-unit-ordering-and-paths, users-groups-impermanence-boundary.
Fresh nixpkgs material (3.1)
Nine 3.1 tasks are built on changes in the pinned nixpkgs (774debe7a0d1b496e35677ad955a1011c6ff74f3, 2026-10-02) that the agent can discover by reading the pinned tree. Candidate IDs refer to fresh-nixpkgs-changes-2026-10.md, which records the old and new forms, the paired evaluations, and the evidence for each change's date. That report warns that release-note membership does not date a change; only some candidates have dated post-cutoff evidence.
| Task | Status in 3.2 | Candidate | Change in the pinned tree |
|---|---|---|---|
home-assistant-declarative-dashboards | Archived in 3.2 (archive/3.1/) | N09 | services.home-assistant.config.lovelace.mode is deprecated for Home Assistant 2026.8 in favour of per-dashboard configuration and resource_mode. |
stalwart-state-version-migration | Calibrating (new in 3.1) | N03 | The Stalwart module requires an explicit module stateVersion, which selects legacy or current identity, state path, and logging. |
initrd-hook-systemd-stage1-migration | Calibrating (new in 3.1) | N01 | Stage 1 is systemd-based by default, and scripted initrd hooks are rejected by assertion. |
yggdrasil-credential-key-owner | Calibrating (new in 3.1) | N02 | services.yggdrasil.settings.PrivateKey is rejected; keys are loaded from a file through systemd credentials. |
dovecot-structured-settings-versions | Calibrating (new in 3.1) | N32 | Dovecot's individual options are removed in favour of structured settings, with version-specific keys and required version settings for 2.4. |
cgit-export-policy-agreement | Archived in 3.2 (archive/3.1/) | N04 | cgit's smart-HTTP backend export check has no default and must be set explicitly. |
nginx-acme-virtualhost-migration | Archived in 3.2 (archive/3.1/) | none | services.nginx.sslCiphers defaults to a list of suites instead of a string (nixos/modules/services/web-servers/nginx/default.nix in the pinned tree). |
users-groups-impermanence-boundary | Archived in 3.2 (archive/3.1/) | R34 | ESPHome no longer runs with DynamicUser; it uses a static esphome user and a non-private state directory. |
python-package-set-override-fleet | Calibrating (new in 3.1) | none | The Python builder's current shape: dependencies is stored under passthru, and pytestFlagsArray now throws (pinned tree, pkgs/development/interpreters/python). |
The other six new 3.1 tasks (debug-infinite-recursion-fleet, fleet-shared-profile-conflict, home-manager-nixos-module-boundary-fleet, rust-workspace-vendored-deps, stdenv-cross-and-host-tools, and systemd-unit-ordering-and-paths) are designed full-system tasks that do not depend on a post-cutoff change. Four of them were archived in 3.2; fleet-shared-profile-conflict and stdenv-cross-and-host-tools remain.
Silent-failure material (3.2)
The nine tasks new in 3.2 come from silent-failure-changes-2026-10.md. S-ids are release-documented changes in the pinned tree whose old form evaluates with exit 0, empty stderr, and no warnings, yet produces a wrong system. T-ids are the silent-regression traps: a shared-profile fix that evaluates cleanly but silently changes a sibling host. As that report warns, release-note membership does not date a change. "Track" is the # track = comment in the task's metadata.toml.
| Task | Status in 3.2 | Track | Research ids | Silent defect in the starter |
|---|---|---|---|---|
hardened-profile-import-no-op | Calibrating (new in 3.2) | eval | S01, T02 | Importing the hardened profile no longer configures anything, and a fleet-wide forced firewall fix would strip a sibling's ports. |
tor-onion-socket-outside-chroot | Calibrating (new in 3.2) | eval | S02 | Tor no longer bind-mounts a Unix-socket onion target into its chroot. |
wireless-eap-key-unreadable-after-hardening | Calibrating (new in 3.2) | eval | S04 | The hardened supplicant can no longer read an EAP-TLS key under root's home. |
service-requires-removed-network-setup | Calibrating (new in 3.2) | eval | S11, T01 | A service still requires the removed network-setup.service, and a shared-profile SSH exception would silently reach sibling hosts. |
vsftpd-local-login-without-pam | Calibrating (new in 3.2) | eval | S12 | The vsftpd module no longer creates a PAM service for local users. |
esphome-persistence-wrong-state-path | Calibrating (new in 3.2) | eval | S17 + T04 | Persistence still covers ESPHome's old private state directory. |
firewall-nftables-forwarding-silently-dropped | Calibrating (new in 3.2) | vm | S11 | Forwarding rules hang off the removed network-setup.service and sit outside the nftables forward filter that now drops them. |
state-directory-ownership-after-hardening | Calibrating (new in 3.2) | vm | none (silent-regression trap from systemd exec semantics; no research id) | A shared sandbox baseline and a move to a dynamic user leave the daemon unable to create its state and socket directories. |
boot-hook-marker-survives-reboot | Calibrating (new in 3.2) | vm | none (designed; extends N01 from fresh-nixpkgs-changes-2026-10.md) | A migrated stage-1 unit runs before the volume it prepares is mounted. |
The S/T mapping above is reconstructed from each task's starter, reference, and contract fixtures; the tasks themselves do not cite research ids.
Source notes
These notes were the prose half of each prompt's former "Source Context" section. They are kept verbatim so the 3.0 prompt rewrite loses no provenance; the links are in the table above.
container-native-vs-oci: This task is modeled after a Reddit report where ChatGPT mixed native NixOS containers with third-party backend containers while trying to configure an Ubuntu-like container with USB forwarding.debug-network-false-lead: This task is modeled after NixOS Discourse help threads where ChatGPT was described as chasing diagnostic rabbit holes, including fixating on HDMI audio or other false leads instead of narrowing from observed evidence.fhs-binary-wrapper: This task is modeled after threads where ChatGPT or generic Linux advice suggested manually creating FHS directories on NixOS. Community replies pointed toward explicit Nix mechanisms such as AppImage tooling, FHS user environments, patching, ornix-ld.flake-input-package-selection: This task is modeled after a NixOS Discourse thread where a user and ChatGPT were stuck onattribute 'default' missingfrom a flake package output.home-manager-extra-special-args: This task is modeled after NixOS Discourse threads where users passedspecialArgsintonixosSystemand assumed Home Manager modules would receive the same flake inputs, leading to missing-argument or infinite-recursion errors untilhome-manager.extraSpecialArgswas used.home-manager-wsl-module-import: This task is modeled after a NixOS Discourse thread where a user could not get Home Manager working inside NixOS-WSL and said Google and ChatGPT were of no use.home-manager-xdg-files: This task is modeled after user reports where ChatGPT gave wrong or imperative answers for creating home folders or putting files under$HOME/.config, while community answers pointed toward Home Manager XDG and file options.issue-report-quality: This task is modeled after Nixpkgs discussions about AI-generated issues and maintainer burden from vague, unverified, or low-signal reports.module-stale-option-migration: This task is modeled after community complaints that LLM answers for NixOS often lag current releases and suggest deprecated or removed options.module-system-boundaries: This task is modeled after reports and discussions where NixOS, Home Manager, and nix-darwin module systems were blurred together. The correct approach is to share plain values or helper functions while keeping module outputs separate for each module system.mutable-config-home-manager: This task is modeled after NixOS Discourse discussions where LLM advice suggested using Home Manager to bootstrap mutable GUI application state, but read-only linked files broke applications that need to write their own profile data.nushell-command-not-found: This task is modeled after an open nixpkgs issue reporting thatprograms.command-not-found.enableworks in Bash but not in Nushell.overlay-module-boundary: This task is modeled after a NixOS Discourse thread where a user tried to define a package and a user systemd timer through one overlay, then learned that overlays cannot set module options and that the package/module split has to be explicit.package-name-lookup-contract: This task is modeled after reports that LLMs make up Nix packages or suggest package names/options that do not exist without a live Nix search source.python-cuda-uv2nix-patch: This task is modeled after a NixOS Discourse uv2nix/CUDA guide where the author said ChatGPT/Gemini were not trustworthy for general NixOS, flakes, or Python-on-NixOS solutions, but could help identify package-providing libraries.rust-no-network-build: This task is modeled after recent reports where Rust packages tried to download prebuilt binaries or data during sandboxed Nix builds.string-escaping-systemd: This task is modeled after a NixOS Discourse thread where a user noted that even frontier models repeatedly make Nix character escaping mistakes, especially around injected code.xdg-portal-merge: This task is modeled after reports that Home Manager-generated portal configuration for Hyprland can break portal discovery for other environments and applications.