`FakeOsModule.use_original` is not thread-safe under `asyncio.to_thread` workloads

[jfindlay]

[This bug was found, isolated, fixed, and reported by Claude Opus 4.7, with some minor edits by me.]

Describe the bug

FakeOsModule.use_original is a process-global class attribute flipped by the use_original_os() context manager. When one thread enters the context manager while another thread is mid-faked-filesystem-operation, the second thread observes use_original=True set by the first thread and dispatches to the real os module, raising a real PermissionError against paths that only exist in the fake filesystem.

Expected: every faked filesystem call inside Patcher stays faked, regardless of what other threads are doing.

Observed: intermittent PermissionError: [Errno 13] Permission denied: '<fake-path>' from real os.mkdir while a worker thread is calling Path.mkdir() against the fake filesystem.

Representative stack trace (from the plain repro below on 6.2.0):

Traceback (most recent call last):
  File ".../threading.py", line 1073, in _bootstrap_inner
    self.run()
  File ".../threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File ".../concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File ".../repro.py", line 6, in _write
    p.parent.mkdir(parents=True, exist_ok=True)
  File ".../pathlib/_local.py", line 770, in mkdir
    os.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '/fake-env'

The trace shows pathlib.Path.mkdir → os.mkdir(self, mode) reaching the real kernel's mkdir, which has no knowledge of /fake-env (the path exists only inside pyfakefs's in-memory tree).

How To Reproduce

Plain repro:

import asyncio
from pathlib import Path
from pyfakefs.fake_filesystem_unittest import Patcher

def _write(p):
    p.parent.mkdir(parents=True, exist_ok=True)
    with p.open("w") as cf:
        cf.write("x")

async def _dispatch(p):
    await asyncio.to_thread(_write, p)

async def many():
    await asyncio.gather(*[
        _dispatch(Path(f"/fake-env/a/m{i}/deep/file")) for i in range(8)
    ])

with Patcher() as p:
    p.fs.create_dir("/fake-env")
    asyncio.run(many())

The repro's failure rate depends on whether another pyfakefs-internal code path (linecache during traceback formatting, RealPath wrappers in fake_pathlib, etc.) concurrently enters use_original_os() while the workers run.

Deterministic repro: fail reliably by adding hammer threads entering the context manager in a tight loop from within the Patcher context so they do not contend with pyfakefs's cold init:

import asyncio
import threading
from pathlib import Path
from pyfakefs.fake_filesystem_unittest import Patcher
from pyfakefs.fake_os import use_original_os

def _write(p):
    p.parent.mkdir(parents=True, exist_ok=True)
    with p.open("w") as cf:
        cf.write("x")

async def _dispatch(p):
    await asyncio.to_thread(_write, p)

async def many(root, n):
    await asyncio.gather(
        *[_dispatch(Path(f"{root}/a/m{i}/deep/file")) for i in range(n)]
    )

with Patcher() as p:
    stop = threading.Event()
    def hammer():
        while not stop.is_set():
            with use_original_os():
                pass
    hammers = [threading.Thread(target=hammer, daemon=True) for _ in range(4)]
    for h in hammers:
        h.start()
    try:
        for round_idx in range(3):
            root = f"/fake-env-{round_idx}"
            p.fs.create_dir(root)
            asyncio.run(many(root, 32))
    finally:
        stop.set()
        for h in hammers:
            h.join(timeout=1)

With hammers running inside the Patcher context, the failure is reliable.

Your environment

Linux-6.17.0-20-generic-x86_64-with-glibc2.39
Python 3.12.3 (main, Mar  3 2026, 12:15:18) [GCC 13.3.0]
pyfakefs 6.2.0
pytest 9.0.2

Also reproduces on current main (6.3.dev0, commit b907b2b).

[mrbean-bremen]

Thanks! pyfakefs is currently not guaranteed to work correctly in multi-threading environments. That being said, there is already a couple of locks in obvious places in the code, so this is something to strive for.

[jfindlay]

Thanks! pyfakefs is currently not guaranteed to work correctly in multi-threading environments.

lol, that would explain the 4 other races I found. I totally should have asked this question before getting into deracing pyfakefs. You can take this one fix, or I can do a thorough scan for other races, or point me at a strategy/design doc on async, or close this out in favor of your own approach: I have a few tokens to burn. (Sorry if I sound like an agent; I've got agent brain right now having gone full agent mode the past several days.)

[mrbean-bremen]

Fixing this is totally fine, I guess - support of multi-threading is something that is possible in pyfakefs (while support for multi-processing is not). So yeah, if it doesn't impact test performance notably, it would be helpful to add the other fixes, thanks!

[jfindlay]

if it doesn't impact test performance

The one test I've proposed so far is ~< 10 s. Is that acceptable, or should I try some other testing strategy besides trying to simulate a nondeterministic race in a test with a short compute burn so that the race signal is 0.99999 for each test?

[mrbean-bremen]

Sorry, I meant the performance of the tests run using pyfakefs, not your tests testing pyfakefs. This is ok.