OpenCL in FreeBSD jails (AMD GPU passthrough)

26 Apr 2026

By default, FreeBSD jails can’t see /dev/dri/* — the kernel hides it via devfsrules_jail. To run OpenCL workloads inside a jail (hashcat, whisper.cpp, llama.cpp OpenCL backend, anything Mesa rusticl can drive), you need to:

Have OpenCL working on the host first.
Tell the jail’s devfs ruleset to unhide /dev/dri/.
Install the OpenCL userland inside the jail.

This post covers the jail-specific bits. For host setup (drm-kmod, mesa, kld_list, video group), see the companion post on OpenCL with AMD Radeon GPUs on FreeBSD.

Tested on FreeBSD 15.0-RELEASE amd64 with an AMD RX 6900 XT.

TL;DR

JAIL=mycontainer

# 1. Add dri unhide to the jail's devfs ruleset (assumes ruleset 10)
doas tee -a /etc/devfs.rules <<'EOF'
add path 'dri' unhide
add path 'dri/*' unhide
EOF
doas service devfs restart

# Re-apply to the running jail without restarting it
JP=$(jls -j $JAIL path)
doas devfs -m $JP/dev ruleset 10
doas devfs -m $JP/dev rule applyset

# 2. Install OpenCL stack inside the jail
doas jexec $JAIL pkg install -y \
  graphics/mesa-dri graphics/mesa-libs devel/ocl-icd security/hashcat

# 3. Set env vars and test
doas jexec -l $JAIL env \
  OCL_ICD_VENDORS=/usr/local/etc/OpenCL/vendors \
  RUSTICL_ENABLE=radeonsi \
  hashcat -I

How GPU passthrough into a jail actually works

AMDGPU presents two character devices per card:

/dev/dri/card0 — for graphics (KMS, modesetting). You don’t need this in a jail unless you’re running an X server inside.
/dev/dri/renderD128 — the render node. Compute-only, no display surface, lower privilege bar. This is what OpenCL uses.

Both are owned root:video mode 0660, so anyone in the video group inside the jail can use them — no cap_sys_rawio or anything exotic.

The host kernel’s devfsrules_jail (ruleset 4) hides /dev/dri entirely. To expose it, add an unhide rule to whatever ruleset your jail uses.

A typical /etc/devfs.rules after this post:

[devfsrules_jail_with_bpf=10]
add include $devfsrules_jail
add path 'bpf*' unhide
add path 'dri' unhide
add path 'dri/*' unhide

The 'dri' unhide line exposes the directory itself; 'dri/*' exposes the device nodes inside.

Step-by-step

1. Confirm the host setup is good

Inside the host, before you touch the jail:

$ kldstat | grep amdgpu
3    1 ... amdgpu.ko
$ ls /dev/dri
card0  renderD128
$ OCL_ICD_VENDORS=/usr/local/etc/OpenCL/vendors RUSTICL_ENABLE=radeonsi clinfo \
    | head
Number of platforms                               1
  Platform Name                                   rusticl
  ...

If any of those don’t work, fix the host first — there’s no combination of jail config that recovers from a broken host stack.

2. Identify your jail’s devfs ruleset

$ grep devfs_ruleset /usr/local/etc/jail.conf.d/$JAIL.conf
  devfs_ruleset = 10;

Or query the running jail:

$ jls -j $JAIL devfs_ruleset
10

If your jail uses ruleset 4 (the default devfsrules_jail), you should define your own numbered ruleset rather than editing the shipped defaults — modifications to the defaults can be clobbered on upgrades.

3. Add unhide rules to the ruleset

If you already have a ruleset 10 block in /etc/devfs.rules, edit it in place:

[devfsrules_jail_with_bpf=10]
add include $devfsrules_jail
add path 'bpf*' unhide
add path 'dri' unhide                # add these
add path 'dri/*' unhide              # two lines

If you don’t have one yet, create it:

doas tee -a /etc/devfs.rules <<'EOF'

[devfsrules_jail_with_gpu=10]
add include $devfsrules_jail
add path 'dri' unhide
add path 'dri/*' unhide
EOF

Reload:

doas service devfs restart

4. Apply to the running jail (no restart needed)

/etc/devfs.rules only takes effect at devfs mount time. To re-apply to a running jail’s devfs without bouncing the jail:

JP=$(jls -j $JAIL path)
doas devfs -m $JP/dev ruleset 10
doas devfs -m $JP/dev rule applyset

Then verify from inside the jail:

$ doas jexec $JAIL ls /dev/dri
card0  renderD128

If you can wait for the next jail restart, just doas service jail restart $JAIL — the new ruleset will take effect at remount.

5. Make sure the jail user is in the video group

Inside the jail:

doas jexec $JAIL pw groupmod video -m yourjailuser

The video group inside a jail is unrelated to the host’s video group; jails have their own /etc/group.

6. Install the OpenCL stack inside the jail

doas jexec $JAIL env ASSUME_ALWAYS_YES=yes pkg bootstrap
doas jexec $JAIL pkg install -y \
  graphics/mesa-dri graphics/mesa-libs devel/ocl-icd security/hashcat

You do NOT install graphics/drm-latest-kmod inside the jail — the kernel module lives on the host and is shared via the device nodes. Installing it inside would just waste disk on firmware blobs the jail can’t use.

7. Set the env vars

In /etc/profile inside the jail:

doas jexec $JAIL tee -a /etc/profile <<'EOF'
export OCL_ICD_VENDORS=/usr/local/etc/OpenCL/vendors
export RUSTICL_ENABLE=radeonsi
EOF

8. Test

$ doas jexec -l -U yourjailuser $JAIL hashcat -I
OpenCL Platform ID #1
  Vendor..: Mesa/X.org
  Name....: rusticl
  ...
  Backend Device ID #1
    Type...........: GPU
    Vendor.........: AMD
    Name...........: AMD Radeon RX 6900 XT (...)

-l sources /etc/profile so the env vars are in scope.

Sharing the GPU between jails

Multiple jails can use the same GPU concurrently. rusticl doesn’t take an exclusive lock — two hashcat processes from two different jails will both run, with throughput split between them. There’s no quota mechanism in the FreeBSD jail framework for this; if you need fair scheduling, use process priority (renice) or hashcat’s own workload tuning.

Troubleshooting

symptom	likely cause
`ls /dev/dri` empty inside jail	devfs rule not applied; run `devfs -m $JP/dev rule applyset`
Rules look right but still empty	ruleset number in jail.conf doesn’t match the block in /etc/devfs.rules
Devices visible but `radeonsi: failed to open device`	jail user not in `video` group
`clinfo` shows zero platforms inside jail	`OCL_ICD_VENDORS` env var not set
Platform listed but zero devices	`RUSTICL_ENABLE=radeonsi` not set
Works as root inside jail, not as user	`pw groupmod video -m <user>` inside the jail
Works on host, fails inside jail with same env vars	`/dev/dri` not exposed (back to step 4)

Security notes

Exposing /dev/dri/renderD128 in a jail gives the jail’s processes direct access to the GPU. Practically that means:

They can submit arbitrary GPU work, including reading other processes’ video memory if the GPU/driver doesn’t isolate it. Modern AMDGPU enforces VMID isolation at the hardware level, but driver bugs have historically been exploited.
They can hard-hang the GPU, which on some boards forces a reboot to recover.

For untrusted code, treat this passthrough roughly like adding the jail user to the host’s video group. For your own workloads (hashcat, whisper.cpp), this is fine.

card0 is more privileged than renderD128 — if you only need compute, you can scope the unhide more tightly:

add path 'dri' unhide
add path 'dri/renderD128' unhide
# don't unhide card0

This blocks KMS/modesetting access from the jail.

spun.io

OpenCL in FreeBSD jails (AMD GPU passthrough)

TL;DR

How GPU passthrough into a jail actually works

Step-by-step

1. Confirm the host setup is good

2. Identify your jail’s devfs ruleset

3. Add unhide rules to the ruleset

4. Apply to the running jail (no restart needed)

5. Make sure the jail user is in the video group

6. Install the OpenCL stack inside the jail

7. Set the env vars

8. Test

Sharing the GPU between jails

Troubleshooting

Security notes

See also

About The Author

Nick Price

TL;DR

How GPU passthrough into a jail actually works

Step-by-step

1. Confirm the host setup is good

2. Identify your jail’s devfs ruleset

3. Add unhide rules to the ruleset

4. Apply to the running jail (no restart needed)

5. Make sure the jail user is in the video group

6. Install the OpenCL stack inside the jail

7. Set the env vars

8. Test

Sharing the GPU between jails

Troubleshooting

Security notes

See also

Related Posts

About The Author

Nick Price