diff --git a/Documentation/guides/fork_vfork_migration.rst b/Documentation/guides/fork_vfork_migration.rst new file mode 100644 index 0000000000000..fa9a305d3c8e6 --- /dev/null +++ b/Documentation/guides/fork_vfork_migration.rst @@ -0,0 +1,237 @@ +========================================= +Migrating to separate ``fork``/``vfork`` +========================================= + +What changed +============ + +NuttX used to implement ``fork()`` and ``vfork()`` as the same function. Both +were thin libc wrappers around a single ``up_fork()`` syscall; ``vfork()`` +differed only by a trailing ``waitpid()``. Underneath, the child joined the +parent's address environment -- the same ``addrenv_join()`` that +``pthread_create()`` uses -- and got a private *copy of the stack*. So the +child shared ``.data``, ``.bss`` and the heap with its parent, and ran +concurrently with it. + +That was not ``fork()``. It was ``vfork()``-with-a-private-stack published +under ``fork()``'s name. The history says so plainly: the ``fork()`` this +replaces was NuttX's old ``vfork()``, renamed in 2023 without any change of +behaviour. And the consequence was silent: a program written against POSIX +``fork()`` compiled and ran, and its child's writes quietly landed in the +parent's variables. + +There are now two distinct primitives, and each means what its name says: + +.. list-table:: + :header-rows: 1 + :widths: 14 30 26 30 + + * - API + - Memory + - Parent + - Availability + * - ``fork()`` + - child gets **its own copy** at the same virtual addresses + - runs concurrently + - ``CONFIG_ARCH_HAVE_FORK`` -- only where an address environment can be + duplicated + * - ``vfork()`` + - child **shares** the parent's memory + - **suspended** until the child ``_exit()``\ s or ``exec()``\ s + - ``CONFIG_ARCH_HAVE_VFORK`` -- no address environment needed + +.. note:: + + ``fork()`` is provided only where the architecture implements + ``up_addrenv_fork()`` and therefore selects ``CONFIG_ARCH_HAVE_FORK``; it + becomes available architecture by architecture as that hook lands. Check + ``CONFIG_ARCH_HAVE_FORK`` in your own configuration rather than assuming + either way. Where it is unset, ``vfork()`` is what the configuration + offers. + +This is a breaking change +========================= + +Two things break, and they break loudly rather than quietly: + +**Code calling** ``fork()`` **on a target without a duplicable address +environment no longer builds.** ``fork()`` is not declared in ``unistd.h`` +there, so you get a compile error naming the function. That is the intended +outcome: a build error is strictly better than the silent wrongness it +replaces. Today that is every in-tree architecture, so every caller of +``fork()`` has to be looked at. + +**Code calling** ``fork()`` **on a target that does have real** ``fork()`` +**changes behaviour** -- from sharing to copying. Code that (perhaps +unknowingly) relied on the sharing will now see the parent and child diverge. + +Which replacement do I want? +============================ + +Answer the question "why did I call ``fork()``?". + +*I want the child to run a different program.* + Use :c:func:`posix_spawn` or ``task_spawn()``. This is the single most + common reason to call ``fork()``, NuttX has always provided a better answer + for it, and that answer does not have the pid discontinuity that + ``fork()``\ +\ ``exec()`` has. If you must keep the two-step idiom, use + ``vfork()`` + ``exec*()``: that is exactly what ``vfork()`` is for, and + unlike ``fork()`` it needs no duplicable address environment. + + .. code-block:: c + + #include + + pid = vfork(); /* was: pid = fork(); */ + if (pid == 0) + { + execv(path, argv); /* or _exit() on failure */ + _exit(EXIT_FAILURE); + } + + Note the restriction that comes with it: between the ``vfork()`` and the + ``exec()`` the child shares the parent's memory and runs on the parent's + behalf, so it must not modify anything, must not return from the calling + function, and must not call anything other than ``_exit()`` or an ``exec`` + family function. + +*I want a second flow of control that shares my memory.* + Use ``pthread_create()``. That is the same memory relationship the old + ``fork()`` gave you, spelled clearly, with a normal entry point instead of a + function that returns twice. There is no longer a returns-twice primitive + with concurrent sharing semantics: the old behaviour was not POSIX, and + ``vfork()`` is not a drop-in for it -- the parent is suspended, so parent and + child never run concurrently. + +*I want a genuinely independent copy of this process.* + Keep calling ``fork()``, and make sure your configuration selects + ``CONFIG_ARCH_HAVE_FORK``. Be aware there is no copy-on-write: the copy is + eager, so forking a large process needs as much free memory as the process + occupies and fails with ``ENOMEM`` otherwise. + +Configuration symbols +===================== + +``CONFIG_ARCH_HAVE_VFORK`` + Hidden. The architecture can implement POSIX ``vfork()``. Selected exactly + where ``ARCH_HAVE_FORK`` used to be, so every configuration that had the old + ``fork()`` has ``vfork()``. + +``CONFIG_ARCH_HAVE_FORK`` + Hidden, ``depends on ARCH_ADDRENV``. It no longer means "``fork()`` + exists"; it means "this configuration can provide POSIX ``fork()`` + semantics", which requires an address environment and an + ``up_addrenv_fork()`` to duplicate it with. + +Notes for architecture maintainers +================================== + +The register/stack snapshot machinery is common to both primitives. Each +architecture exposes one entry point, ``up_fork(bool vfork)``, whose argument +says which primitive the caller used and is handed to ``nxtask_setup_fork()``. +That is where the memory semantics are decided: ``addrenv_join()`` for +``vfork()``, ``addrenv_fork()`` for ``fork()``. + +Adding real ``fork()`` to an architecture +----------------------------------------- + +Two things are needed, and the second is the one that is easy to miss. + +**Implement** ``up_addrenv_fork()``. It duplicates an address environment: +allocate fresh pages, copy the parent's contents into them, and map them at the +*same* virtual addresses. ``up_addrenv_clone()`` is not this -- it copies only +the representation and leaves both processes pointing at one set of page +tables. Then give ``ARCH_HAVE_FORK`` a ``default y if `` line in +``arch/Kconfig``. + +**Build the child from the caller's saved system call frame.** In a kernel +build ``fork()`` is reached through a system call, so the return address and +stack pointer the architecture's fork entry point can observe for itself belong +to the *kernel*, not to the caller; a child built from those resumes at a +kernel address on a kernel stack. The architecture must record the caller's +exception frame when it traps -- ``xcp.sregs`` is the field that exists for +this -- and build the child from that instead, while a kernel thread that calls +the entry point directly still takes the ordinary path. + +Four architectures do it, and they are worth copying: + +* RISC-V: ``riscv_swint.c`` stores the frame in ``xcp.sregs``, and + ``riscv_fork.c`` rebuilds the child from it. +* arm64: ``arm64_vectors.S`` hands the frame to ``dispatch_syscall()``, which + stores it in ``xcp.sregs``; ``arm64_fork()`` then dispatches to + ``arm64_fork_syscall()`` or ``arm64_fork_direct()`` according to whether + ``TCB_FLAG_SYSCALL`` is set, so a kernel thread that calls the entry point + directly still works. +* armv7-a: ``arm_syscall.c`` stores the frame in ``xcp.sregs``, and + ``arm_fork()`` dispatches to ``arm_fork_syscall()`` or + ``arm_fork_direct()``. The discriminator here is a saved user stack + pointer, ``xcp.ustkptr``, rather than ``TCB_FLAG_SYSCALL``: armv7-a + dispatches a system call by re-pointing the caller's own exception frame at + ``dispatch_syscall()``, so the caller *is* the task that runs the kernel + side of the call. What makes its snapshot useless is not the system call + as such but the switch to the kernel stack, which leaves the kernel-side + frames on a stack the child gets no copy of. A build without a kernel + stack dispatches on the caller's own stack, so there the frames are copied + along with the caller's and ``arm_fork_direct()`` remains correct. Because + ``arm_syscall()`` has already re-pointed the frame by the time + ``arm_fork()`` runs, its PC, CPSR and SP are the kernel's; the caller's are + read from where ``arm_syscall()`` put them -- ``syscall[0].sysreturn``, + ``syscall[0].cpsr`` and ``ustkptr``. +* x86_64: ``x86_64_syscall()`` stores the frame in ``xcp.sregs``, and + ``x86_64_fork()`` dispatches to ``x86_64_fork_syscall()`` or + ``x86_64_fork_direct()``. The discriminator here is ``xcp.sregs`` itself + being non-NULL, because raising ``TCB_FLAG_SYSCALL`` would also defer signal + actions -- something x86_64 has never done and its kernel-build signal path + does not currently survive. Two properties of ``SYSCALL``/``SYSRET`` shape + the child's frame: the instruction leaves the caller's RIP and RFLAGS in + RCX and R11 rather than on a stack, so they have to be moved into the RIP + and RFLAGS slots of the interrupt frame the child is resumed from; and the + hardware never records the caller's CS and SS at all -- ``SYSRETQ`` + reconstructs them from ``IA32_STAR`` -- so the child's have to be filled in + with the user code and data selectors at RPL 3. For the same reason the + saved frame is not copied wholesale: only the extended state and the + general registers are inherited, and the segment registers and thread + pointer come from the frame ``up_initial_state()`` built for the child. + +Nothing else is required: the ``up_fork()`` entry point and the libc wrapper +are already there and become live automatically. + +Note that ``ARCH_HAVE_FORK`` is about a *per-process* address environment. A +protected build has one address space carved up once at boot, whether the +boundaries are drawn by an MPU or by a fixed set of MMU mappings; its +``up_addrenv_*()`` are stubs, and there is no mapping to duplicate at the same +virtual addresses. ``CONFIG_ARCH_ADDRENV`` being set is therefore not by +itself evidence that ``fork()`` can be provided. ``vfork()``, which shares the +parent's memory, works there as everywhere else. + +Known gaps +========== + +``fork()`` **is gained one architecture at a time.** The generic machinery is +complete -- ``addrenv_fork()``, the ``up_addrenv_fork()`` hook, the syscall, the +libc wrapper and the ``ostest`` case -- so an architecture provides ``fork()`` +by implementing ``up_addrenv_fork()`` and selecting ``CONFIG_ARCH_HAVE_FORK``, +with no further generic work. + +**A windowed ABI needs its stack rebased, not just copied.** On Xtensa, +giving a child a relocated copy of the parent's stack takes more than the copy: +the register-window save areas embedded in the stack hold absolute stack +pointers, so each one has to be rebased along with the copy, or the child +reloads a pointer into the *parent's* stack on its very first window underflow. +That rebasing is architecture-specific and belongs with the Xtensa entry points +rather than here. + +Note also on ``waitpid()`` after ``vfork()`` +============================================ + +The ``vfork()`` parent is now resumed when the child's TCB is torn down, so by +the time it runs the child is completely gone. Where the child called +``exec()`` this makes no difference -- ``exec_swap()`` has already given the +loaded program the child's pid, and that program is still running, so +``waitpid()`` behaves normally. Where the child called ``_exit()``, +``waitpid()`` can only return its status if ``CONFIG_SCHED_CHILD_STATUS`` is +enabled; otherwise it returns ``ECHILD``, because NuttX does not retain the +status of a task that no longer exists. That is a pre-existing property of +that configuration, not a change: the previous implementation blocked in a +libc ``waitpid(WNOWAIT)`` and an application's own ``waitpid()`` afterwards hit +the same wall. diff --git a/Documentation/guides/index.rst b/Documentation/guides/index.rst index acbb2a118b554..9b6114409aa00 100644 --- a/Documentation/guides/index.rst +++ b/Documentation/guides/index.rst @@ -37,6 +37,7 @@ Guides logging_rambuffer.rst ipv6.rst integrate_newlib.rst + fork_vfork_migration.rst protected_build.rst platform_directories.rst port_drivers_to_stm32f7.rst diff --git a/Documentation/implementation/memory_configurations.rst b/Documentation/implementation/memory_configurations.rst index 6ad536df5d6c3..f8f5b8b9b97de 100644 --- a/Documentation/implementation/memory_configurations.rst +++ b/Documentation/implementation/memory_configurations.rst @@ -30,7 +30,7 @@ On-Demand Paging NuttX also supports on-demand paging via ``CONFIG_PAGING``. On-demand paging is a method of virtual memory management and requires -the the CPU architecutre support a MMU. +the the CPU architecture support a MMU. In a system that uses on-demand paging, the OS responds to a page fault by copying data from some storage media into physical memory and setting up @@ -410,7 +410,7 @@ of functions that: 1. Have only one ``.text`` space in RAM, but 2. Separate ``.data`` and ``.bass`` space, and are -3. Separately linked into with the program in each address environmnet. +3. Separately linked into with the program in each address environment. (not implemented). @@ -484,7 +484,7 @@ at least in its current form. That full implementation of ``mmap()`` plus the minor changes to the NuttX ELF loader are all that are required to support fully share-able ``.text`` sections – as well as the memory savings -from not carrying aroung the relocation and symbol information +from not carrying around the relocation and symbol information (Not implemented). @@ -632,10 +632,13 @@ the contemplate in any real detail: and swap the state into physical memory as needed?(not implemented). * ``mmap()``. True shared memory and true file mapping could be supported. I am repeating myself (not implemented). -* ``fork()``. The ``fork()`` interface could be supported. NuttX currently - supports the "crippled" version, ``vfork()`` but with these process address - environments, the real ``fork()`` interface could be supported. - (not implemented). +* ``fork()``. The real ``fork()`` interface can be supported on configurations + with a duplicable process address environment: an architecture implements + ``up_addrenv_fork()`` and selects ``CONFIG_ARCH_HAVE_FORK``. What is not + implemented is + copy-on-write: the duplication copies the parent's pages eagerly, which + needs as much free memory as the parent occupies. Demand paging would fix + that. * Dynamic Stack Allocation. Completely eliminate the need for constant tuning of static stack sizes.(not implemented). * Shared Libraries. Am I repeating myself again?(not implemented). @@ -688,8 +691,8 @@ There are two problems here: So how do you create new tasks/processes in such a context. There is only one way possible; by using an interface that takes a file name as an argument (rather than absolute address). -New processes started with ``vfork()`` and ``exec()`` or with -``posix_spawn()`` should not have any of these issues. +New processes started with ``fork()`` or ``vfork()`` and ``exec()``, or with +``posix_spawn()``, should not have any of these issues. ARM Memory Management diff --git a/Documentation/reference/user/01_task_control.rst b/Documentation/reference/user/01_task_control.rst index 163030776c35b..05c064d4ecdcc 100644 --- a/Documentation/reference/user/01_task_control.rst +++ b/Documentation/reference/user/01_task_control.rst @@ -59,8 +59,9 @@ Standard interfaces - :c:func:`exit` - :c:func:`getpid` -Standard ``vfork`` and ``exec[v|l]`` interfaces: +Standard ``fork``/``vfork`` and ``exec[v|l]`` interfaces: + - :c:func:`fork` - :c:func:`vfork` - :c:func:`exec` - :c:func:`execv` @@ -347,6 +348,46 @@ Functions **POSIX Compatibility:** Compatible with the POSIX interface of the same name. +.. c:function:: pid_t fork(void) + + ``fork()`` creates a new process. The child process is an exact copy of + the calling process: it receives **its own copy** of the parent's memory, + at the same virtual addresses. Writes by the child are invisible to the + parent and writes by the parent are invisible to the child. The child may + modify anything, call any function, return from the function in which + ``fork()`` was called, and run indefinitely; it runs concurrently with the + parent. None of ``vfork()``'s restrictions apply. + + NOTE: ``fork()`` requires an address environment that can be + duplicated, and so it is available only where ``CONFIG_ARCH_HAVE_FORK`` + is selected -- which in turn requires ``CONFIG_ARCH_ADDRENV`` and an + architecture that implements ``up_addrenv_fork()``. **Where it cannot + be provided it is not provided at all**: the declaration is + absent from ``unistd.h`` and code that calls it fails to build. That + is deliberate. A build error naming the function is strictly better + than a ``fork()`` that silently gives the child the parent's memory. + See :doc:`/guides/fork_vfork_migration` for how to move code that + relied on the previous behaviour. + + There is no copy-on-write, because NuttX has no demand paging to build + it on, so the copy is eager: forking a large process needs as much free + memory as the process occupies and fails with ``ENOMEM`` otherwise. + Spawn-heavy code should prefer :c:func:`posix_spawn` or + :c:func:`vfork`, on NuttX as anywhere. + + Applications that relied on the historical NuttX ``fork()`` behaviour -- + shared memory, private stack, both running -- want + :c:func:`pthread_create`, which is that memory relationship spelled + clearly, or :c:func:`posix_spawn`. + + :return: Upon successful completion, ``fork()`` returns 0 to the child + process and returns the process ID of the child process to the parent + process. Otherwise, -1 is returned to the parent, no child process is + created, and ``errno`` is set to indicate the error. + + **POSIX Compatibility:** Compatible with the POSIX interface of the same + name. + .. c:function:: pid_t vfork(void) The ``vfork()`` function has the same effect as @@ -357,12 +398,18 @@ Functions function before successfully calling ``_exit()`` or one of the ``exec`` family of functions. - NOTE: ``vfork()`` is not an independent NuttX feature, but is - implemented in architecture-specific logic (using only helper - functions from the NuttX core logic). As a result, ``vfork()`` may - not be available on all architectures. The current implementation in - NuttX arm64 only guarantees that ``vfork()`` works when - CONFIG_BUILD_FLAT=y. + The child **shares** the parent's memory -- nothing is copied, which is the + entire point of ``vfork()`` and the reason a caller chooses it -- and the + parent is **suspended** until the child calls ``_exit()`` or one of the + ``exec`` family. That suspension is what makes the sharing safe, and it is + also the price: the restrictions above exist because the child is running + in the parent's address space on borrowed time. + + NOTE: ``vfork()`` is implementable with or without an MMU and is + available wherever ``CONFIG_ARCH_HAVE_VFORK`` is selected. The + suspension lives in the kernel primitive rather than in a libc + ``waitpid()``, so the parent is resumed at ``exec()`` as POSIX requires, + and ``vfork()`` does not depend on ``CONFIG_SCHED_WAITPID``. :return: Upon successful completion, ``vfork()`` returns 0 to the child process and returns the process ID of the child process to the @@ -449,7 +496,15 @@ Functions thread, then (2) call ``execv()`` or ``execl()`` to replace the new thread with a program from the file system. Since the new thread will be terminated by the ``execv()`` or ``execl()`` call, it really served no - purpose other than to support POSIX compatibility. + purpose other than to support POSIX compatibility. :c:func:`posix_spawn` + does the same job in one step and should be preferred. + + Note also that ``exec()`` does not overlay the calling process: it starts + the new program as a separate task. ``exec_swap()`` then exchanges the two + pids, so that from the parent's point of view the pid ``vfork()`` returned + does name the running program -- but the ``vfork()`` stub itself exits, and + it is that exit which releases the suspended ``vfork()`` parent. The parent + therefore resumes at ``exec()``, as POSIX requires. The non-standard binfmt function ``exec()`` needs to have (1) a symbol table that provides the list of symbols exported by the base code, and diff --git a/Documentation/standards/posix.rst b/Documentation/standards/posix.rst index cdca6eb748ea9..47622ec7921c2 100644 --- a/Documentation/standards/posix.rst +++ b/Documentation/standards/posix.rst @@ -1421,7 +1421,7 @@ Multiple Processes: +--------------------------------+---------+ | :c:func:`exit` | Yes | +--------------------------------+---------+ -| fork() | No | +| :c:func:`fork` | Cond. | +--------------------------------+---------+ | :c:func:`getpgrp` | Yes | +--------------------------------+---------+ @@ -2364,7 +2364,7 @@ XSI Multiple Process: +--------------------------------+---------+ | :c:func:`usleep` | Yes | +--------------------------------+---------+ -| :c:func:`vfork` | Yes | +| :c:func:`vfork` | Cond. | +--------------------------------+---------+ | :c:func:`waitid` | Yes | +--------------------------------+---------+ diff --git a/arch/Kconfig b/arch/Kconfig index ab0a0c066c7e5..60039f1991e4e 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -11,7 +11,7 @@ config ARCH_ARM bool "ARM" select ARCH_HAVE_BACKTRACE select ARCH_HAVE_INTERRUPTSTACK - select ARCH_HAVE_FORK + select ARCH_HAVE_VFORK select ARCH_HAVE_STACKCHECK select ARCH_HAVE_CUSTOMOPT select ARCH_HAVE_STDARG_H @@ -29,7 +29,7 @@ config ARCH_ARM64 select ARCH_64BIT select ARCH_HAVE_BACKTRACE select ARCH_HAVE_INTERRUPTSTACK - select ARCH_HAVE_FORK if !BUILD_KERNEL && !BUILD_PROTECTED + select ARCH_HAVE_VFORK select ARCH_HAVE_STACKCHECK select ARCH_HAVE_CUSTOMOPT select ARCH_HAVE_STDARG_H @@ -87,7 +87,7 @@ config ARCH_RISCV select ARCH_HAVE_CPUINFO select ARCH_HAVE_INTERRUPTSTACK select ARCH_HAVE_STACKCHECK - select ARCH_HAVE_FORK + select ARCH_HAVE_VFORK select ARCH_HAVE_CUSTOMOPT select ARCH_HAVE_SETJMP select ARCH_HAVE_STDARG_H @@ -111,7 +111,7 @@ config ARCH_SIM select ARCH_HAVE_TICKLESS select ARCH_HAVE_POWEROFF select ARCH_HAVE_TESTSET - select ARCH_HAVE_FORK if !HOST_WINDOWS + select ARCH_HAVE_VFORK if !HOST_WINDOWS select ARCH_HAVE_SETJMP select ARCH_HAVE_CUSTOMOPT select ARCH_HAVE_TCBINFO @@ -147,7 +147,7 @@ config ARCH_X86_64 select PCI_LATE_DRIVERS_REGISTER if PCI select ARCH_TOOLCHAIN_GNU select ARCH_HAVE_BACKTRACE - select ARCH_HAVE_FORK if !BUILD_KERNEL + select ARCH_HAVE_VFORK select ARCH_HAVE_SETJMP select ARCH_HAVE_PERF_EVENTS select ARCH_HAVE_POWEROFF @@ -482,9 +482,39 @@ config ARCH_HAVE_CPUID_MAPPING default n depends on ARCH_HAVE_MULTICPU +config ARCH_HAVE_VFORK + bool + default n + ---help--- + The architecture can implement POSIX vfork(): the child shares the + parent's memory and the parent is suspended until the child calls + _exit() or one of the exec family of functions. + config ARCH_HAVE_FORK bool default n + depends on ARCH_ADDRENV + ---help--- + The architecture can implement POSIX fork(): the child receives its + own copy of the parent's memory at the same virtual addresses, may + modify anything, may return from the function that called fork(), and + runs concurrently with the parent. + + This requires an address environment to duplicate, and an + up_addrenv_fork() to duplicate it with: the copy is backed by freshly + allocated pages holding a copy of the parent's contents, mapped at the + same virtual addresses. + + No architecture selects this yet. Two things are needed. First, + up_addrenv_fork() itself. Second, the architecture must build the + child's register context from the *user's* saved system call frame: + in a kernel build fork() is reached through a system call, so the + return address and stack pointer the architecture's fork entry point + can see for itself are the kernel's, not the caller's, and a child + built from those resumes at a kernel address. + + Where this is not selected fork() is not provided at all, and code + that calls it fails to build. config ARCH_HAVE_CRC32 bool diff --git a/arch/arm/include/armv7-a/irq.h b/arch/arm/include/armv7-a/irq.h index b9f973b627bce..096f9ea99ba08 100644 --- a/arch/arm/include/armv7-a/irq.h +++ b/arch/arm/include/armv7-a/irq.h @@ -301,6 +301,22 @@ struct xcptcontext uint8_t nsyscalls; struct xcpt_syscall_s syscall[CONFIG_SYS_NNEST]; + + /* Where the register save area of the caller of the outermost system call + * is, which is the exception frame arm_vectorsvc built on the caller's own + * stack. It is recorded by arm_syscall() and is what the cloning + * primitives build the child's context from: a fork() or vfork() reached + * through a system call has to give the child the registers of the task + * that trapped, not those of the kernel-side stub that arm_fork() is + * called from. See arm_fork(). + * + * The frame is the one arm_syscall() has already re-pointed at + * dispatch_syscall(): its PC, CPSR, R0 and SP are the kernel's. The + * caller's own values are in syscall[0].sysreturn, syscall[0].cpsr and + * ustkptr respectively; R0 is the return value and belongs to neither. + */ + + uint32_t *sregs; #endif #ifdef CONFIG_ARCH_ADDRENV diff --git a/arch/arm/src/armv7-a/arm_syscall.c b/arch/arm/src/armv7-a/arm_syscall.c index 5c0039bf9b7ac..bcc9c187be532 100644 --- a/arch/arm/src/armv7-a/arm_syscall.c +++ b/arch/arm/src/armv7-a/arm_syscall.c @@ -513,6 +513,20 @@ uint32_t *arm_syscall(uint32_t *regs) rtcb->xcp.syscall[index].cpsr = regs[REG_CPSR]; #endif + /* Remember where the caller's registers are. The cloning + * primitives need them: the child of a fork() or vfork() made + * from user space resumes from this very SVC, so it is built from + * this frame and not from the registers the kernel-side stub + * happens to be running with. Only the outermost system call is + * of interest, since that is the one the caller made. See + * arm_fork(). + */ + + if (index == 0) + { + rtcb->xcp.sregs = regs; + } + regs[REG_PC] = (uint32_t)dispatch_syscall; #ifdef CONFIG_BUILD_KERNEL cpsr = regs[REG_CPSR] & ~PSR_MODE_MASK; diff --git a/arch/arm/src/common/arm_fork.c b/arch/arm/src/common/arm_fork.c index f98f8ec896a3c..55adec0510fa6 100644 --- a/arch/arm/src/common/arm_fork.c +++ b/arch/arm/src/common/arm_fork.c @@ -42,66 +42,53 @@ #include "sched/sched.h" /**************************************************************************** - * Public Functions + * Private Functions ****************************************************************************/ /**************************************************************************** - * Name: arm_fork + * Name: arm_fork_direct * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. - * - * The overall sequence is: - * - * 1) User code calls fork(). fork() collects context information and - * transfers control up arm_fork(). - * 2) arm_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: - * - Allocation of the child task's TCB. - * - Initialization of file descriptors and streams - * - Configuration of environment variables - * - Allocate and initialize the stack - * - Setup the input parameters for the task. - * - Initialization of the TCB (including call to up_initial_state()) - * 4) arm_fork() provides any additional operating context. arm_fork must: - * - Initialize special values in any CPU registers that were not - * already configured by up_initial_state() - * 5) arm_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. - * - * nxtask_abort_fork() may be called if an error occurs between steps 3 and - * 6. + * Clone a caller that reached up_fork() by an ordinary function call, so + * that the register snapshot the entry point in fork.S took describes the + * caller itself. That is the case in a flat build, in a protected build + * -- where a system call is dispatched on the caller's own stack, so the + * caller's frames are copied along with the kernel-side ones and the child + * unwinds back through them -- and for a kernel thread in any build. * * Input Parameters: - * context - Caller context information saved by fork() + * vfork - true for vfork(), false for fork() + * parent - The calling task's TCB + * context - Caller context information saved by the entry point * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * The process ID of the child, or ERROR on failure. * ****************************************************************************/ -pid_t arm_fork(const struct fork_s *context) +static pid_t arm_fork_direct(bool vfork, struct tcb_s *parent, + const struct fork_s *context) { - struct tcb_s *parent = this_task(); struct tcb_s *child; uint32_t newsp; uint32_t newfp; uint32_t newtop; uint32_t stacktop; uint32_t stackutil; -#ifdef CONFIG_ARCH_KERNEL_STACK - uint32_t oldsp = (uint32_t)parent->xcp.ustkptr; -#else uint32_t oldsp = context->sp; + +#ifdef CONFIG_ARCH_KERNEL_STACK + /* A caller that trapped into a system call and was switched onto its + * kernel stack left its own stack pointer here; the snapshot in `context' + * is the kernel-side stub's. Where the whole exception frame is available + * arm_fork_syscall() has already taken the call, so this is reached only + * where the caller's frames are copied along with the kernel-side ones. + */ + + if (parent->xcp.ustkptr != NULL) + { + oldsp = (uint32_t)parent->xcp.ustkptr; + } #endif sinfo("fork context [%p]:\n", context); @@ -115,7 +102,7 @@ pid_t arm_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)(context->lr & ~1)); + child = nxtask_setup_fork((start_t)(context->lr & ~1), vfork); if (!child) { serr("ERROR: nxtask_setup_fork failed\n"); @@ -137,43 +124,60 @@ pid_t arm_fork(const struct fork_s *context) sinfo("Parent: stackutil:%" PRIu32 "\n", stackutil); - /* Make some feeble effort to preserve the stack contents. This is - * feeble because the stack surely contains invalid pointers and other - * content that will not work in the child context. However, if the - * user follows all of the caveats of fork() usage, even this feeble - * effort is overkill. - */ + if (child->stack_base_ptr == parent->stack_base_ptr) + { + /* The child is running at the parent's stack addresses, inside its + * own duplicated address environment. There is nothing to relocate: + * every stack address the child inherits is still the address it + * names. + */ - newtop = (uint32_t)child->stack_base_ptr + - child->adj_stack_size; + newsp = oldsp; + newfp = context->fp; + } + else + { + /* Make some feeble effort to preserve the stack contents. This is + * feeble because the stack surely contains invalid pointers and other + * content that will not work in the child context. However, if the + * user follows all of the caveats of vfork() usage, even this feeble + * effort is overkill. + * + * For a POSIX fork() child the stack contents are not merely a feeble + * effort: the child is entitled to use them, and it does. + */ - newsp = newtop - stackutil; + newtop = (uint32_t)child->stack_base_ptr + + child->adj_stack_size; - /* Move the register context to newtop. */ + newsp = newtop - stackutil; - memcpy((void *)(newsp - XCPTCONTEXT_SIZE), - child->xcp.regs, XCPTCONTEXT_SIZE); + /* Move the register context to newtop. */ - child->xcp.regs = (void *)(newsp - XCPTCONTEXT_SIZE); + memcpy((void *)(newsp - XCPTCONTEXT_SIZE), + child->xcp.regs, XCPTCONTEXT_SIZE); - memcpy((void *)newsp, (const void *)oldsp, stackutil); + child->xcp.regs = (void *)(newsp - XCPTCONTEXT_SIZE); - /* Was there a frame pointer in place before? */ + memcpy((void *)newsp, (const void *)oldsp, stackutil); - if (context->fp >= oldsp && context->fp < stacktop) - { - uint32_t frameutil = stacktop - context->fp; - newfp = newtop - frameutil; - } - else - { - newfp = context->fp; - } + /* Was there a frame pointer in place before? */ + + if (context->fp >= oldsp && context->fp < stacktop) + { + uint32_t frameutil = stacktop - context->fp; + newfp = newtop - frameutil; + } + else + { + newfp = context->fp; + } - sinfo("Old stack top:%08" PRIx32 " SP:%08" PRIx32 " FP:%08" PRIx32 "\n", - stacktop, oldsp, context->fp); - sinfo("New stack top:%08" PRIx32 " SP:%08" PRIx32 " FP:%08" PRIx32 "\n", - newtop, newsp, newfp); + sinfo("Old stack top:%08" PRIx32 " SP:%08" PRIx32 + " FP:%08" PRIx32 "\n", stacktop, oldsp, context->fp); + sinfo("New stack top:%08" PRIx32 " SP:%08" PRIx32 + " FP:%08" PRIx32 "\n", newtop, newsp, newfp); + } /* Update the stack pointer, frame pointer, and volatile registers. When * the child TCB was initialized, all of the values were set to zero. @@ -245,9 +249,225 @@ pid_t arm_fork(const struct fork_s *context) } #endif - /* And, finally, start the child task. On a failure, nxtask_start_fork() - * will discard the TCB by calling nxtask_abort_fork(). + /* And, finally, start the child task. A vfork() additionally suspends us + * until the child calls _exit() or exec(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); +} + +#if defined(CONFIG_ARCH_ARMV7A) && defined(CONFIG_ARCH_KERNEL_STACK) + +/**************************************************************************** + * Name: arm_fork_syscall + * + * Description: + * Clone a caller that reached up_fork() through a system call that was + * switched onto a kernel stack. The register snapshot fork.S took is + * useless here: it describes the kernel-side stub, and the frames below + * it are on a stack the child does not get a copy of, so a child built + * from it would resume at a kernel address with a stack pointer into its + * own user stack. + * + * What the caller was actually doing is the exception frame arm_vectorsvc + * built on the caller's stack and arm_syscall() recorded in xcp.sregs; the + * child is built from that. It therefore returns from the very same SVC + * instruction as the parent, in the same mode, differing only in that it + * sees 0 as the return value and runs on its own stack. The child is not + * in a system call at all, so it inherits none of the parent's nesting + * state. + * + * Input Parameters: + * vfork - true for vfork(), false for fork() + * parent - The calling task's TCB + * + * Returned Value: + * The process ID of the child, or ERROR on failure. + * + ****************************************************************************/ + +static pid_t arm_fork_syscall(bool vfork, struct tcb_s *parent) +{ + uint32_t *sregs = parent->xcp.sregs; + struct tcb_s *child; + uint32_t newsp; + uint32_t newfp; + uint32_t newtop; + uint32_t stacktop; + uint32_t stackutil; + uint32_t oldsp = (uint32_t)parent->xcp.ustkptr; + uint32_t sysreturn; + uint32_t cpsr; + + DEBUGASSERT(sregs != NULL && parent->xcp.nsyscalls > 0); + + /* Where the caller resumes, and in which mode. arm_syscall() re-pointed + * the frame at dispatch_syscall() before this was reached, so these two + * come from where it put the originals rather than from the frame. In a + * protected build there is no mode change, so the frame still holds the + * caller's CPSR. + */ + + sysreturn = parent->xcp.syscall[0].sysreturn; +#ifdef CONFIG_BUILD_KERNEL + cpsr = parent->xcp.syscall[0].cpsr; +#else + cpsr = sregs[REG_CPSR]; +#endif + + /* Allocate and initialize a TCB for the child task. */ + + child = nxtask_setup_fork((start_t)(sysreturn & ~1), vfork); + if (!child) + { + serr("ERROR: nxtask_setup_fork failed\n"); + return (pid_t)ERROR; + } + + stacktop = (uint32_t)parent->stack_base_ptr + + parent->adj_stack_size; + DEBUGASSERT(stacktop > oldsp && oldsp >= (uint32_t)parent->stack_base_ptr); + stackutil = stacktop - oldsp; + + if (child->stack_base_ptr == parent->stack_base_ptr) + { + /* The child is running at the parent's stack addresses, inside its + * own duplicated address environment. There is nothing to relocate; + * see the same case in arm_fork_direct(). + */ + + newsp = oldsp; + newfp = sregs[REG_FP]; + } + else + { + newtop = (uint32_t)child->stack_base_ptr + + child->adj_stack_size; + newsp = newtop - stackutil; + + /* Put the child's register save area where the parent's is: just + * below the stack the caller was using. It cannot be left at the top + * of the child's stack, which is where up_initial_state() put it, + * because the copy of the parent's stack below is about to land there. + */ + + child->xcp.regs = (uint32_t *)(newsp - XCPTCONTEXT_SIZE); + + memcpy((void *)newsp, (const void *)oldsp, stackutil); + + /* Was there a frame pointer in place before? */ + + if (sregs[REG_FP] >= oldsp && sregs[REG_FP] < stacktop) + { + uint32_t frameutil = stacktop - sregs[REG_FP]; + newfp = newtop - frameutil; + } + else + { + newfp = sregs[REG_FP]; + } + } + + /* Inherit the caller's whole exception frame, integer and floating point + * registers alike, then fix up only what has to differ: the child sees 0 + * as the return value and runs on its own stack. + */ + + memcpy(child->xcp.regs, sregs, XCPTCONTEXT_SIZE); + + child->xcp.regs[REG_R0] = 0; + child->xcp.regs[REG_FP] = newfp; + child->xcp.regs[REG_SP] = newsp; + child->xcp.regs[REG_PC] = sysreturn; + child->xcp.regs[REG_CPSR] = cpsr; + + /* And, finally, start the child task. A vfork() additionally suspends us + * until the child calls _exit() or exec(). + */ + + return nxtask_start_fork(child, vfork); +} + +#endif /* CONFIG_ARCH_ARMV7A && CONFIG_ARCH_KERNEL_STACK */ + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: arm_fork + * + * Description: + * The common ARM worker behind up_fork(). vfork() and fork() snapshot + * the caller's registers identically; `vfork' says which primitive was + * called, and is passed straight through to nxtask_setup_fork(), which is + * where the memory semantics are decided. + * + * What differs here is only the stack. Normally the child has a stack of + * its own, and this function fills it with a relocated copy of the + * parent's, rebasing the stack and frame pointers to match. When the + * child shares the parent's stack addresses -- a fork() child, inside its + * duplicated address environment -- there is nothing to relocate and the + * pointers are carried over unchanged. + * + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). The libc wrapper enters + * up_fork(), which collects context information and transfers control + * to arm_fork(). + * 2) arm_fork() calls nxtask_setup_fork(). + * 3) nxtask_setup_fork() allocates and configures the child task's TCB. + * This consists of: + * - Allocation of the child task's TCB. + * - Initialization of file descriptors and streams + * - Configuration of environment variables + * - Establishing the child's address environment: joined to the + * parent's for vfork(), duplicated from it for fork() + * - Allocating the stack, or inheriting the parent's for fork() + * - Setup the input parameters for the task. + * - Initialization of the TCB (including call to up_initial_state()) + * 4) arm_fork() provides any additional operating context. arm_fork must: + * - Initialize special values in any CPU registers that were not + * already configured by up_initial_state() + * 5) arm_fork() then calls nxtask_start_fork(), which for vfork() + * additionally suspends the caller. + * 6) which executes the child thread. + * + * nxtask_abort_fork() may be called if an error occurs between steps 3 and + * 6. + * + * Everything above is common to the two ways this can be reached, which + * differ only in where the caller's registers are to be found -- see + * arm_fork_direct() and arm_fork_syscall(). + * + * Input Parameters: + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by the entry point + * + * Returned Value: + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. + * + ****************************************************************************/ + +pid_t arm_fork(bool vfork, const struct fork_s *context) +{ + struct tcb_s *parent = this_task(); + +#if defined(CONFIG_ARCH_ARMV7A) && defined(CONFIG_ARCH_KERNEL_STACK) + /* A saved user stack pointer means this was reached through a system call + * that switched to the task's kernel stack, so the caller is the user task + * that trapped and not the code that called into fork.S. A kernel thread + * has no kernel stack to switch to and never gets here with one saved. + */ + + if (parent->xcp.ustkptr != NULL) + { + DEBUGASSERT((parent->flags & TCB_FLAG_SYSCALL) != 0); + return arm_fork_syscall(vfork, parent); + } +#endif + + return arm_fork_direct(vfork, parent, context); } diff --git a/arch/arm/src/common/gnu/fork.S b/arch/arm/src/common/gnu/fork.S index 83e22326c26c8..60a1a2f07be71 100644 --- a/arch/arm/src/common/gnu/fork.S +++ b/arch/arm/src/common/gnu/fork.S @@ -1,5 +1,5 @@ /**************************************************************************** - * arch/arm/src/common/fork.S + * arch/arm/src/common/gnu/fork.S * * SPDX-License-Identifier: Apache-2.0 * @@ -38,43 +38,41 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives. Both need exactly the same thing from assembly -- a + * snapshot of the caller's callee-saved registers, stack pointer and + * return address -- and differ only in what the C code then does with it, + * so there is one entry point and the caller's r0 says which primitive was + * called. It is passed straight through to arm_fork(). * - * This thin layer implements fork by simply calling up_fork() with the - * fork() context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) arm_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point. + * 2) The entry point collects the context and calls arm_fork(). + * 3) arm_fork() calls nxtask_setup_fork(), which allocates and configures + * the child task's TCB. This consists of: * - Allocation of the child task's TCB. * - Initialization of file descriptors and streams * - Configuration of environment variables - * - Allocate and initialize the stack + * - Establishing the child's address environment + * - Allocating the stack, or inheriting the parent's for fork() * - Setup the input parameters for the task. * - Initialization of the TCB (including call to up_initial_state()) - * 4) arm_fork() provides any additional operating context. arm_fork must: + * 4) arm_fork() provides any additional operating context: * - Initialize special values in any CPU registers that were not * already configured by up_initial_state() + * - Relocate the copied stack, unless the child shares the parent's * 5) arm_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * 6) which executes the child thread. * * Input Parameters: - * None + * r0 - true for vfork(), false for fork() * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. * ****************************************************************************/ @@ -88,7 +86,7 @@ up_fork: /* Create a stack frame */ - mov r0, sp /* Save the value of the stack on entry */ + mov r3, sp /* Save the value of the stack on entry */ sub sp, sp, #FORK_SIZEOF /* Allocate the structure on the stack */ /* CPU registers */ @@ -102,11 +100,13 @@ up_fork: mov r7, r11 stmia r1!, {r4-r7} /* Save r8-r11 in the structure */ mov r5, lr /* Copy lr to a low register */ - stmia r1!, {r0,r5} /* Save sp and lr in the structure */ + stmia r1!, {r3,r5} /* Save sp and lr in the structure */ - /* Then, call arm_fork(), passing it a pointer to the stack structure */ + /* Then, call arm_fork(). r0 still holds the vfork flag: nothing above + * touches it. + */ - mov r0, sp + mov r1, sp bl arm_fork /* Recover r4-r7 that were destroyed before arm_fork was called */ @@ -114,7 +114,7 @@ up_fork: mov r1, sp ldmia r1!, {r4-r7} - /* Release the stack data and return the value returned by up_fork */ + /* Release the stack data and return the value returned by arm_fork */ ldr r1, [sp, #FORK_LR_OFFSET] mov r14, r1 diff --git a/arch/arm/src/common/iar/fork.S b/arch/arm/src/common/iar/fork.S index 3f16484a36530..53e3e221e3aca 100644 --- a/arch/arm/src/common/iar/fork.S +++ b/arch/arm/src/common/iar/fork.S @@ -50,43 +50,41 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives. Both need exactly the same thing from assembly -- a + * snapshot of the caller's callee-saved registers, stack pointer and + * return address -- and differ only in what the C code then does with it, + * so there is one entry point and the caller's r0 says which primitive was + * called. It is passed straight through to arm_fork(). * - * This thin layer implements fork by simply calling up_fork() with the - * fork() context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) arm_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point. + * 2) The entry point collects the context and calls arm_fork(). + * 3) arm_fork() calls nxtask_setup_fork(), which allocates and configures + * the child task's TCB. This consists of: * - Allocation of the child task's TCB. * - Initialization of file descriptors and streams * - Configuration of environment variables - * - Allocate and initialize the stack + * - Establishing the child's address environment + * - Allocating the stack, or inheriting the parent's for fork() * - Setup the input parameters for the task. * - Initialization of the TCB (including call to up_initial_state()) * 4) arm_fork() provides any additional operating context. arm_fork must: * - Initialize special values in any CPU registers that were not * already configured by up_initial_state() + * - Relocate the copied stack, unless the child shares the parent's * 5) arm_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * 6) which executes the child thread. * * Input Parameters: - * None + * r0 - true for vfork(), false for fork() * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. * ****************************************************************************/ @@ -95,7 +93,7 @@ up_fork: /* Create a stack frame */ - mov r0, sp /* Save the value of the stack on entry */ + mov r3, sp /* Save the value of the stack on entry */ sub sp, sp, #FORK_SIZEOF /* Allocate the structure on the stack */ /* CPU registers */ @@ -112,14 +110,16 @@ up_fork: /* Save the frame pointer, stack pointer, and return address */ str r11, [sp, #FORK_FP_OFFSET] /* fp not defined. use r11 */ - str r0, [sp, #FORK_SP_OFFSET] + str r3, [sp, #FORK_SP_OFFSET] str lr, [sp, #FORK_LR_OFFSET] /* Floating point registers (not yet) */ - /* Then, call arm_fork(), passing it a pointer to the stack structure */ + /* Then, call arm_fork(). r0 still holds the vfork flag: nothing above + * touches it. + */ - mov r0, sp + mov r1, sp bl arm_fork /* Release the stack data and return the value returned by arm_fork */ diff --git a/arch/arm64/include/irq.h b/arch/arm64/include/irq.h index 3fae95b1ff392..6dd9a4e282158 100644 --- a/arch/arm64/include/irq.h +++ b/arch/arm64/include/irq.h @@ -284,6 +284,21 @@ struct xcptcontext uint64_t *initregs; #endif +#ifdef CONFIG_LIB_SYSCALL + /* The caller's register context, as saved by the SVC exception entry, for + * the duration of a system call. This is what the *user* was doing when + * it trapped, as opposed to `regs' above, which during a system call + * describes the kernel. + * + * vfork() and fork() need it: they are reached through a system call, so + * the return address and stack pointer their architecture entry point can + * see for itself are the kernel's, and a child built from those would + * resume at a kernel address on a kernel stack. + */ + + uint64_t *sregs; +#endif + #ifdef CONFIG_ARCH_FPU uint64_t *fpu_regs; uint64_t *saved_fpu_regs; diff --git a/arch/arm64/src/common/arm64_fork.c b/arch/arm64/src/common/arm64_fork.c index fbbf6a56da0e5..3566c64ded810 100644 --- a/arch/arm64/src/common/arm64_fork.c +++ b/arch/arm64/src/common/arm64_fork.c @@ -53,119 +53,146 @@ ****************************************************************************/ /**************************************************************************** - * Public Functions + * Private Functions ****************************************************************************/ -#ifdef CONFIG_ARCH_FPU +/**************************************************************************** + * Name: arm64_fork_stack + * + * Description: + * Give the child the part of the parent's stack that is in use, copied to + * the top of the child's own stack. + * + * The copy is aligned with the top of each stack rather than the bottom, + * so a single offset carries any address in the copied region from the + * parent's stack to the child's; that offset is what is returned, and + * arm64_fork_reloc() applies it. + * + * Input Parameters: + * parent - The parent task's TCB + * child - The child task's TCB + * sp - The parent's stack pointer where the primitive was called + * + * Returned Value: + * The offset from an address in the parent's stack to the same place in + * the child's copy of it. + * + ****************************************************************************/ -void arm64_fork_fpureg_save(struct fork_s *context) +static uint64_t arm64_fork_stack(struct tcb_s *parent, struct tcb_s *child, + uint64_t sp) { - /* Take a snapshot of the thread fpu reg context right now */ + uint64_t stacktop; + uint64_t stackutil; + uint64_t newtop; - arm64_fpu_save(context->fpu); - UP_DSB(); -} + /* How much of the parent's stack was utilized? The ARM uses a push-down + * stack so that the current stack pointer should be lower than the + * initial, adjusted stack pointer. The stack usage should be the + * difference between those two. + */ -#endif + stacktop = (uint64_t)parent->stack_base_ptr + parent->adj_stack_size; + DEBUGASSERT(stacktop > sp); + stackutil = stacktop - sp; + + /* Make some feeble effort to preserve the stack contents. This is + * feeble because the stack surely contains invalid pointers and other + * content that will not work in the child context. However, if the + * user follows all of the caveats of vfork() usage, even this feeble + * effort is overkill. + */ + + newtop = (uint64_t)child->stack_base_ptr + child->adj_stack_size; + memcpy((void *)(newtop - stackutil), (const void *)sp, stackutil); + + return newtop - stacktop; +} /**************************************************************************** - * Name: fork + * Name: arm64_fork_reloc * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * Carry one address from the parent's stack over to the child's copy of + * it. Addresses outside the region that arm64_fork_stack() copied are + * returned unchanged: they point somewhere the child shares with the + * parent, or somewhere that has no counterpart at all. * - * The overall sequence is: + * Input Parameters: + * parent - The parent task's TCB + * addr - The address to relocate + * sp - The parent's stack pointer where the primitive was called, + * which is the low end of the region that was copied + * offset - The offset returned by arm64_fork_stack() * - * 1) User code calls fork(). fork() collects context information and - * transfers control up arm64_fork(). - * 2) arm64_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: - * - Allocation of the child task's TCB. - * - Initialization of file descriptors and streams - * - Configuration of environment variables - * - Allocate and initialize the stack - * - Setup the input parameters for the task. - * - Initialization of the TCB (including call to up_initial_state()) - * 4) arm64_fork() provides any additional operating context. arm64_fork - * must: - * - Initialize special values in any CPU registers that were not - * already configured by up_initial_state() - * 5) arm64_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * Returned Value: + * The relocated address. * - * nxtask_abort_fork() may be called if an error occurs between steps 3 and - * 6. + ****************************************************************************/ + +static uint64_t arm64_fork_reloc(struct tcb_s *parent, uint64_t addr, + uint64_t sp, uint64_t offset) +{ + uint64_t stacktop = (uint64_t)parent->stack_base_ptr + + parent->adj_stack_size; + + /* The top of the stack is included: a stack pointer resting there is one + * past the last byte copied, and still has to move with it. + */ + + if (addr >= sp && addr <= stacktop) + { + return addr + offset; + } + + return addr; +} + +/**************************************************************************** + * Name: arm64_fork_direct + * + * Description: + * Clone a caller that reached up_fork() by an ordinary function call, so + * that the register snapshot taken by arm64_fork_func.S describes the + * caller itself. That is the case in a flat build, and for a kernel + * thread in any build. + * + * The child has no exception frame to inherit, so one is synthesised: it + * resumes at the caller's return address, at the same privilege level, + * with the callee-saved registers the caller had. * * Input Parameters: - * context - Caller context information saved by fork() + * vfork - true for vfork(), false for fork() + * parent - The calling task's TCB + * context - Caller context information saved by arm64_fork_func.S * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * The process ID of the child, or ERROR on failure. * ****************************************************************************/ -pid_t arm64_fork(const struct fork_s *context) +static pid_t arm64_fork_direct(bool vfork, struct tcb_s *parent, + const struct fork_s *context) { - struct tcb_s *parent = this_task(); struct tcb_s *child; + uint64_t offset; uint64_t newsp; uint64_t newfp; - uint64_t newtop; - uint64_t stacktop; - uint64_t stackutil; /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)context->lr); + child = nxtask_setup_fork((start_t)context->lr, vfork); if (!child) { serr("ERROR: nxtask_setup_fork failed\n"); return (pid_t)ERROR; } - /* How much of the parent's stack was utilized? The ARM uses - * a push-down stack so that the current stack pointer should - * be lower than the initial, adjusted stack pointer. The - * stack usage should be the difference between those two. - */ - - stacktop = (uint64_t)parent->stack_base_ptr + - parent->adj_stack_size; - DEBUGASSERT(stacktop > context->sp); - stackutil = stacktop - context->sp; - - /* Make some feeble effort to preserve the stack contents. This is - * feeble because the stack surely contains invalid pointers and other - * content that will not work in the child context. However, if the - * user follows all of the caveats of fork() usage, even this feeble - * effort is overkill. - */ - - newtop = (uint64_t)child->stack_base_ptr + - child->adj_stack_size; - newsp = newtop - stackutil; - memcpy((void *)newsp, (const void *)context->sp, stackutil); + /* Copy the parent's stack to the child and relocate the pointers into it */ - /* Was there a frame pointer in place before? */ - - if (context->fp >= context->sp && context->fp < stacktop) - { - uint64_t frameutil = stacktop - context->fp; - newfp = newtop - frameutil; - } - else - { - newfp = context->fp; - } + offset = arm64_fork_stack(parent, child, context->sp); + newsp = context->sp + offset; + newfp = arm64_fork_reloc(parent, context->fp, context->sp, offset); /* Update the stack pointer, frame pointer, and volatile registers. When * the child TCB was initialized, all of the values were set to zero. @@ -235,5 +262,208 @@ pid_t arm64_fork(const struct fork_s *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); +} + +#ifdef CONFIG_LIB_SYSCALL + +/**************************************************************************** + * Name: arm64_fork_syscall + * + * Description: + * Clone a caller that reached up_fork() through a system call. The + * register snapshot taken by arm64_fork_func.S is useless here: it + * describes the kernel-side stub, so a child built from it would resume at + * a kernel address on a kernel stack. What the caller was actually doing + * is the exception frame the SVC handler recorded in xcp.sregs; the child + * is built from that. + * + * The child therefore returns from the very same SVC instruction as the + * parent, at the same privilege level, differing only in that it sees 0 + * as the return value and runs on its own stack. + * + * Input Parameters: + * vfork - true for vfork(), false for fork() + * parent - The calling task's TCB + * + * Returned Value: + * The process ID of the child, or ERROR on failure. + * + ****************************************************************************/ + +static pid_t arm64_fork_syscall(bool vfork, struct tcb_s *parent) +{ + uint64_t *sregs = parent->xcp.sregs; + struct tcb_s *child; + uint64_t offset; + uint64_t newsp; + uint64_t newfp; + uint64_t regtop; + uint64_t sp; + + DEBUGASSERT(sregs != NULL); + + /* Which stack the caller was on depends on the level it trapped from: a + * user task uses SP_EL0, anything running in the kernel uses SP_ELx. + */ + + if ((sregs[REG_SPSR] & SPSR_MODE_MASK) == SPSR_MODE_EL0T) + { + sp = sregs[REG_SP_EL0]; + } + else + { + sp = sregs[REG_SP_ELX]; + } + + /* Allocate and initialize a TCB for the child task. The child resumes at + * the instruction after the SVC, which is where the parent resumes too. + */ + + child = nxtask_setup_fork((start_t)sregs[REG_ELR], vfork); + if (!child) + { + serr("ERROR: nxtask_setup_fork failed\n"); + return (pid_t)ERROR; + } + + /* Copy the parent's stack to the child and relocate the pointers into it */ + + offset = arm64_fork_stack(parent, child, sp); + newsp = sp + offset; + newfp = arm64_fork_reloc(parent, sregs[REG_FP], sp, offset); + + /* Where does the register save area the child is resumed from go? The + * parent's is wherever SP_ELx pointed when it trapped, so put the child's + * at the matching place: the top of its own kernel stack if it has one -- + * a user process in a kernel build -- or else the same offset into its + * copy of the parent's stack. + */ + +#ifdef CONFIG_ARCH_KERNEL_STACK + if (child->xcp.kstack) + { + regtop = (uint64_t)child->xcp.kstack + ARCH_KERNEL_STACKSIZE; + } + else +#endif + { + regtop = arm64_fork_reloc(parent, sregs[REG_SP_ELX], sp, offset); + } + + child->xcp.regs = (void *)(regtop - XCPTCONTEXT_SIZE); + + /* Inherit the parent's whole exception frame, integer and FPU registers + * alike, then fix up only what has to differ. + */ + + memcpy(child->xcp.regs, sregs, XCPTCONTEXT_SIZE); + +#ifdef CONFIG_ARCH_FPU + child->xcp.fpu_regs = (void *)(regtop - FPU_CONTEXT_SIZE); +#endif + + child->xcp.regs[REG_X0] = 0; + child->xcp.regs[REG_FP] = newfp; + child->xcp.regs[REG_EXE_DEPTH] = 0; + child->xcp.regs[REG_SP_ELX] = regtop - XCPTCONTEXT_SIZE; + + if ((sregs[REG_SPSR] & SPSR_MODE_MASK) == SPSR_MODE_EL0T) + { + child->xcp.regs[REG_SP_EL0] = newsp; +#ifdef CONFIG_ARCH_KERNEL_STACK + child->xcp.ustkptr = (uintptr_t *)newsp; +#endif + } + + /* And, finally, start the child task. On a failure, nxtask_start_fork() + * will discard the TCB by calling nxtask_abort_fork(). + */ + + return nxtask_start_fork(child, vfork); +} + +#endif /* CONFIG_LIB_SYSCALL */ + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +#ifdef CONFIG_ARCH_FPU + +void arm64_fork_fpureg_save(struct fork_s *context) +{ + /* Take a snapshot of the thread fpu reg context right now */ + + arm64_fpu_save(context->fpu); + UP_DSB(); +} + +#endif + +/**************************************************************************** + * Name: arm64_fork + * + * Description: + * The common ARM64 worker behind up_fork(). vfork() and fork() snapshot + * the caller's registers identically; `vfork' says which primitive was + * called, and is passed straight through to nxtask_setup_fork(), which is + * where the memory semantics are decided. + * + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). up_fork() collects context + * information and transfers control to arm64_fork(). + * 2) arm64_fork() and calls nxtask_setup_fork(). + * 3) nxtask_setup_fork() allocates and configures the child task's TCB. + * This consists of: + * - Allocation of the child task's TCB. + * - Initialization of file descriptors and streams + * - Configuration of environment variables + * - Allocate and initialize the stack + * - Setup the input parameters for the task. + * - Initialization of the TCB (including call to up_initial_state()) + * 4) arm64_fork() provides any additional operating context. arm64_fork + * must: + * - Initialize special values in any CPU registers that were not + * already configured by up_initial_state() + * 5) arm64_fork() then calls nxtask_start_fork() + * 6) nxtask_start_fork() then executes the child thread. + * + * nxtask_abort_fork() may be called if an error occurs between steps 3 and + * 6. + * + * Everything above is common to the two ways this can be reached, which + * differ only in where the caller's registers are to be found -- see + * arm64_fork_direct() and arm64_fork_syscall(). + * + * Input Parameters: + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by up_fork() + * + * Returned Value: + * Upon successful completion, fork() returns 0 to the child process and + * returns the process ID of the child process to the parent process. + * Otherwise, -1 is returned to the parent, no child process is created, + * and errno is set to indicate the error. + * + ****************************************************************************/ + +pid_t arm64_fork(bool vfork, const struct fork_s *context) +{ + struct tcb_s *parent = this_task(); + +#ifdef CONFIG_LIB_SYSCALL + /* If a system call is in progress then this was reached from its kernel- + * side stub, and the caller is the user task that trapped, not the code + * that called into arm64_fork_func.S. + */ + + if ((parent->flags & TCB_FLAG_SYSCALL) != 0) + { + return arm64_fork_syscall(vfork, parent); + } +#endif + + return arm64_fork_direct(vfork, parent, context); } diff --git a/arch/arm64/src/common/arm64_fork_func.S b/arch/arm64/src/common/arm64_fork_func.S index 79cd9566076d5..83fb6017879e4 100644 --- a/arch/arm64/src/common/arm64_fork_func.S +++ b/arch/arm64/src/common/arm64_fork_func.S @@ -41,46 +41,46 @@ ****************************************************************************/ /**************************************************************************** - * Name: fork + * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives. Both need exactly the same thing from assembly -- a + * snapshot of the caller's registers, stack pointer and return address -- + * and differ only in what the C code then does with it, so there is one + * entry point and the caller's x0 says which primitive was called. It is + * saved into the snapshot along with the other argument registers, which + * is harmless: x0-x18 are caller-saved and arm64_fork() does not + * propagate them to the child. * - * This thin layer implements fork by simply calling up_fork() with the - * fork() context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) arm64_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point. + * 2) The entry point collects the context and calls arm64_fork(). + * 3) arm64_fork() calls nxtask_setup_fork(), which allocates and + * configures the child task's TCB. This consists of: * - Allocation of the child task's TCB. * - Initialization of file descriptors and streams * - Configuration of environment variables - * - Allocate and initialize the stack + * - Establishing the child's address environment + * - Allocating the stack, or inheriting the parent's for fork() * - Setup the input parameters for the task. * - Initialization of the TCB (including call to up_initial_state()) - * 4) arm64_fork() provides any additional operating context. arm64_fork must: + * 4) arm64_fork() provides any additional operating context: * - Initialize special values in any CPU registers that were not * already configured by up_initial_state() + * - Relocate the copied stack, unless the child shares the parent's * 5) arm64_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * 6) which executes the child thread. * * Input Parameters: - * None + * x0 - true for vfork(), false for fork() * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. * ****************************************************************************/ @@ -122,10 +122,13 @@ SECTION_FUNC(text, up_fork) ldp x0, x30, [sp], #16 #endif - /* Then, call arm64_fork(), passing it a pointer to the stack structure */ + /* Then, call arm64_fork(), passing it the vfork flag and a pointer to + * the stack structure. The flag comes back out of the snapshot: the + * sequence above clobbers x0, but it saved it first. + */ - mov x0, sp - mov x1, #0 + ldr x0, [sp, #8 * FORK_REG_X0] + mov x1, sp bl arm64_fork /* Release the stack data and return the value returned by arm64_fork */ diff --git a/arch/arm64/src/common/arm64_syscall.c b/arch/arm64/src/common/arm64_syscall.c index 8d503aa64e3dc..8e9bc569986da 100644 --- a/arch/arm64/src/common/arm64_syscall.c +++ b/arch/arm64/src/common/arm64_syscall.c @@ -95,7 +95,7 @@ static void arm64_dump_syscall(const char *tag, uint64_t cmd, uintptr_t dispatch_syscall(unsigned int nbr, uintptr_t parm1, uintptr_t parm2, uintptr_t parm3, uintptr_t parm4, uintptr_t parm5, - uintptr_t parm6) + uintptr_t parm6, void *context) { struct tcb_s *rtcb = this_task(); register long x0 asm("x0") = (long)(nbr); @@ -117,6 +117,10 @@ uintptr_t dispatch_syscall(unsigned int nbr, uintptr_t parm1, return -ENOSYS; } + /* Set the user register context to TCB */ + + rtcb->xcp.sregs = context; + /* Indicate that we are in a syscall handler */ rtcb->flags |= TCB_FLAG_SYSCALL; diff --git a/arch/arm64/src/common/arm64_vectors.S b/arch/arm64/src/common/arm64_vectors.S index eaedd7508bea2..2dae675644e95 100644 --- a/arch/arm64/src/common/arm64_vectors.S +++ b/arch/arm64/src/common/arm64_vectors.S @@ -172,6 +172,13 @@ SECTION_FUNC(text, arm64_sync_exc) msr daifclr, #IRQ_DAIF_MASK /* Re-enable interrupts */ 1: + /* Pass the caller's exception frame as the last argument. x0-x6 hold + * the system call number and its six parameters, so x7 is free. The + * cloning primitives need this frame to build the child's context; see + * dispatch_syscall() and arm64_fork(). + */ + + mov x7, sp bl dispatch_syscall msr daifset, #IRQ_DAIF_MASK /* Disable interrupts */ diff --git a/arch/ceva/src/common/ceva_fork.c b/arch/ceva/src/common/ceva_fork.c index 2eca6db1b7620..7cf6bd684ff18 100644 --- a/arch/ceva/src/common/ceva_fork.c +++ b/arch/ceva/src/common/ceva_fork.c @@ -50,11 +50,16 @@ * called, or calls any other function before successfully calling _exit() * or one of the exec family of functions. * + * Those are vfork()'s semantics, and vfork() is all this architecture can + * provide: POSIX fork() needs an address environment to duplicate and + * CEVA has none. So up_fork()'s argument is always true here, and the + * entry point does not carry it through the context-saving trap. + * * The overall sequence is: * * 1) User code calls fork(). fork() collects context information and * transfers control up ceva_fork(). - * 2) ceva_fork()and calls nxtask_forksetup(). + * 2) ceva_fork() and calls nxtask_forksetup(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: * - Allocation of the child task's TCB. @@ -73,7 +78,7 @@ * nxtask_abort_fork() may be called if an error occurs between steps 3 & 6. * * Input Parameters: - * regs - Caller context information saved by fork() + * regs - Caller context information saved by up_fork() * * Return: * Upon successful completion, fork() returns 0 to the child process and @@ -97,9 +102,14 @@ pid_t ceva_fork(const uint32_t *regs) void *argv; int ret; + /* How large is the parent's stack argument area? */ + + argsize = (uintptr_t)parent->stack_base_ptr - + (uintptr_t)parent->stack_alloc_ptr; + /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork(parent->start, &argsize); + child = nxtask_setup_fork(parent->start, true); if (!child) { serr("ERROR: nxtask_setup_fork failed\n"); @@ -204,7 +214,7 @@ pid_t ceva_fork(const uint32_t *regs) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, true); #else /* CONFIG_SCHED_WAITPID */ return (pid_t)ERROR; #endif diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig index 1d1f2a2965d87..6efaa22ba23a4 100644 --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -56,7 +56,7 @@ endchoice config ARCH_MIPS32 bool default n - select ARCH_HAVE_FORK + select ARCH_HAVE_VFORK config ARCH_MIPS_M4K bool diff --git a/arch/mips/src/mips32/Kconfig b/arch/mips/src/mips32/Kconfig index 16fa348165463..c30962ed7b2e3 100644 --- a/arch/mips/src/mips32/Kconfig +++ b/arch/mips/src/mips32/Kconfig @@ -93,7 +93,7 @@ config MIPS32_TOOLCHAIN_MICROCHIP_XC32_LICENSED config MIPS32_FRAMEPOINTER bool "ABI Uses Frame Pointer" default n - depends on ARCH_HAVE_FORK + depends on ARCH_HAVE_VFORK ---help--- Register r30 may be a frame pointer in some ABIs. Or may just be saved register s8. It makes a difference for fork handling. diff --git a/arch/mips/src/mips32/fork.S b/arch/mips/src/mips32/fork.S index 790aa8cd43f42..be2b263a3351a 100644 --- a/arch/mips/src/mips32/fork.S +++ b/arch/mips/src/mips32/fork.S @@ -47,20 +47,17 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the behavior is - * undefined if the process created by fork() either modifies any data other than - * a variable of type pid_t used to store the return value from fork(), or returns - * from the function in which fork() was called, or calls any other function before - * successfully calling _exit() or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). Both need exactly the same snapshot from + * assembly and differ only in what the C code then does with it, so there + * is one entry point and the caller's $a0 says which primitive was called. + * It is passed straight through to mips_fork(). * - * This thin layer implements fork by simply calling up_fork() with the fork() - * context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) mips_fork() and calls nxtask_setup_fork(). + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls mips_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. This * consists of: * - Allocation of the child task's TCB. @@ -76,7 +73,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * $a0 - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and returns @@ -88,16 +85,19 @@ .text .align 2 - .globl up_fork - .type up_fork, function .set nomips16 #ifdef CONFIG_MIPS_MICROMIPS .set micromips #endif + + .globl up_fork + .type up_fork, function .ent up_fork up_fork: - /* Create a stack frame */ + /* Create a stack frame. $a0 holds the vfork flag and is not part of the + * snapshot, so it is still there when mips_fork() is called below. + */ move $t0, $sp /* Save the value of the stack on entry */ addiu $sp, $sp, -FORK_SIZEOF /* Allocate the structure on the stack */ @@ -130,9 +130,11 @@ up_fork: /* Floating point registers (not yet) */ - /* Then, call mips_fork(), passing it a pointer to the stack structure */ + /* Then, call mips_fork(), passing it a pointer to the stack structure. + * $a0 already holds the vfork flag. + */ - move $a0, $sp + move $a1, $sp jal mips_fork nop diff --git a/arch/mips/src/mips32/mips_fork.c b/arch/mips/src/mips32/mips_fork.c index 3d931d1da1d56..6ae724a52f466 100644 --- a/arch/mips/src/mips32/mips_fork.c +++ b/arch/mips/src/mips32/mips_fork.c @@ -79,7 +79,8 @@ * and 6 * * Input Parameters: - * context - Caller context information saved by fork() + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by up_fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and @@ -89,7 +90,7 @@ * ****************************************************************************/ -pid_t mips_fork(const struct fork_s *context) +pid_t mips_fork(bool vfork, const struct fork_s *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -113,7 +114,7 @@ pid_t mips_fork(const struct fork_s *context) context->fp, context->sp, context->ra, context->gp); #else sinfo("fp:%08" PRIx32 " sp:%08" PRIx32 " ra:%08" PRIx32 "\n", - context->fp context->sp, context->ra); + context->fp, context->sp, context->ra); #endif #else sinfo("s5:%08" PRIx32 " s6:%08" PRIx32 " s7:%08" PRIx32 @@ -130,7 +131,7 @@ pid_t mips_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)context->ra); + child = nxtask_setup_fork((start_t)context->ra, vfork); if (!child) { sinfo("nxtask_setup_fork failed\n"); @@ -217,5 +218,5 @@ pid_t mips_fork(const struct fork_s *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } diff --git a/arch/risc-v/src/common/CMakeLists.txt b/arch/risc-v/src/common/CMakeLists.txt index 84a1d43fb47c8..639cf94afef0d 100644 --- a/arch/risc-v/src/common/CMakeLists.txt +++ b/arch/risc-v/src/common/CMakeLists.txt @@ -86,7 +86,7 @@ if(CONFIG_STACK_COLORATION) list(APPEND SRCS riscv_checkstack.c) endif() -if(CONFIG_ARCH_HAVE_FORK) +if(CONFIG_ARCH_HAVE_VFORK OR CONFIG_ARCH_HAVE_FORK) list(APPEND SRCS fork.S riscv_fork.c) endif() diff --git a/arch/risc-v/src/common/Make.defs b/arch/risc-v/src/common/Make.defs index d98a828c508e0..4b46fc9ccad11 100644 --- a/arch/risc-v/src/common/Make.defs +++ b/arch/risc-v/src/common/Make.defs @@ -86,7 +86,7 @@ ifeq ($(CONFIG_STACK_COLORATION),y) CMN_CSRCS += riscv_checkstack.c endif -ifeq ($(CONFIG_ARCH_HAVE_FORK),y) +ifneq ($(CONFIG_ARCH_HAVE_VFORK)$(CONFIG_ARCH_HAVE_FORK),) CMN_ASRCS += fork.S CMN_CSRCS += riscv_fork.c endif diff --git a/arch/risc-v/src/common/fork.S b/arch/risc-v/src/common/fork.S index 108ff00f80db1..a516e57fd96c7 100644 --- a/arch/risc-v/src/common/fork.S +++ b/arch/risc-v/src/common/fork.S @@ -46,46 +46,36 @@ ****************************************************************************/ /**************************************************************************** - * Name: fork + * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives. Both need exactly the same thing from assembly -- a + * snapshot of the caller's callee-saved registers, stack pointer and + * return address -- and differ only in what the C code then does with it, + * so there is one entry point and the caller's a0 says which primitive was + * called. It is passed straight through to riscv_fork(). * - * This thin layer implements fork by simply calling up_fork() with the - * fork() context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) riscv_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: - * - Allocation of the child task's TCB. - * - Initialization of file descriptors and streams - * - Configuration of environment variables - * - Allocate and initialize the stack - * - Setup the input parameters for the task. - * - Initialization of the TCB (including call to up_initial_state()) - * 4) riscv_fork() provides any additional operating context. riscv_fork must: - * - Initialize special values in any CPU registers that were not - * already configured by up_initial_state() - * 5) riscv_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point. + * 2) The entry point collects the context and calls riscv_fork(). + * 3) riscv_fork() calls nxtask_setup_fork(), which allocates and + * configures the child task's TCB. + * 4) riscv_fork() provides any additional operating context and relocates + * the copied stack. + * 5) riscv_fork() then calls nxtask_start_fork(), which for vfork() + * additionally suspends the caller. + * 6) which executes the child thread. * * Input Parameters: - * None + * a0 - true for vfork(), false for fork() * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. * ****************************************************************************/ @@ -94,7 +84,9 @@ up_fork: #ifdef CONFIG_LIB_SYSCALL - /* When coming via system call, everything is in place already */ + /* When coming via system call, everything is in place already: a0 already + * holds the vfork flag, and riscv_fork() takes its snapshot from the TCB. + */ tail riscv_fork #else @@ -129,8 +121,8 @@ up_fork: REGSTORE gp, FORK_GP_OFFSET(sp) #endif - addi a0, sp, FORK_SIZEOF - REGSTORE a0, FORK_SP_OFFSET(sp) /* original SP */ + addi a2, sp, FORK_SIZEOF + REGSTORE a2, FORK_SP_OFFSET(sp) /* original SP */ REGSTORE x1, FORK_RA_OFFSET(sp) /* return address */ /* Floating point registers */ @@ -150,9 +142,11 @@ up_fork: FSTORE fs11, FORK_FS11_OFFSET(sp) #endif - /* Then, call riscv_fork(), passing it a pointer to the stack frame */ + /* Then, call riscv_fork(). a0 still holds the vfork flag: nothing above + * touches it. + */ - mv a0, sp + mv a1, sp call riscv_fork /* Release the stack frame and return the value returned by riscv_fork */ diff --git a/arch/risc-v/src/common/riscv_fork.c b/arch/risc-v/src/common/riscv_fork.c index f25e5cf4e4062..107576297857c 100644 --- a/arch/risc-v/src/common/riscv_fork.c +++ b/arch/risc-v/src/common/riscv_fork.c @@ -41,8 +41,6 @@ #include "sched/sched.h" -#ifdef CONFIG_ARCH_HAVE_FORK - /**************************************************************************** * Pre-processor Definitions ****************************************************************************/ @@ -59,17 +57,15 @@ * Name: riscv_fork * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The common RISC-V worker behind up_fork(). vfork() and fork() snapshot + * the caller's registers identically; `vfork' says which primitive was + * called, and is passed straight through to nxtask_setup_fork(), which is + * where the memory semantics are decided. * * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up riscv_fork(). + * 1) User code calls vfork() or fork(). up_fork() collects context + * information and transfers control to riscv_fork(). * 2) riscv_fork() and calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: @@ -90,7 +86,8 @@ * and 6. * * Input Parameters: - * context - Caller context information saved by fork() + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by up_fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and @@ -102,7 +99,7 @@ #ifdef CONFIG_LIB_SYSCALL -pid_t riscv_fork(const struct fork_s *context) +pid_t riscv_fork(bool vfork, const struct fork_s *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -117,7 +114,7 @@ pid_t riscv_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)parent->xcp.sregs[REG_RA]); + child = nxtask_setup_fork((start_t)parent->xcp.sregs[REG_RA], vfork); if (!child) { sinfo("nxtask_setup_fork failed\n"); @@ -130,12 +127,26 @@ pid_t riscv_fork(const struct fork_s *context) DEBUGASSERT(stacktop > parent->xcp.sregs[REG_SP]); stackutil = stacktop - parent->xcp.sregs[REG_SP]; - /* Copy goes to child's user stack top */ + if (child->stack_base_ptr == parent->stack_base_ptr) + { + /* The child is running at the parent's stack addresses, inside its + * own duplicated address environment. There is nothing to relocate: + * every stack address the child inherits is still the address it + * names. + */ + + newsp = parent->xcp.sregs[REG_SP]; + } + else + { + /* Copy goes to child's user stack top */ - newtop = (uintptr_t)child->stack_base_ptr + child->adj_stack_size; - newsp = newtop - stackutil; + newtop = (uintptr_t)child->stack_base_ptr + child->adj_stack_size; + newsp = newtop - stackutil; - memcpy((void *)newsp, (const void *)parent->xcp.sregs[REG_SP], stackutil); + memcpy((void *)newsp, (const void *)parent->xcp.sregs[REG_SP], + stackutil); + } #ifdef CONFIG_SCHED_THREAD_LOCAL /* Save child's thread pointer */ @@ -184,12 +195,12 @@ pid_t riscv_fork(const struct fork_s *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } #else -pid_t riscv_fork(const struct fork_s *context) +pid_t riscv_fork(bool vfork, const struct fork_s *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -215,7 +226,7 @@ pid_t riscv_fork(const struct fork_s *context) context->fp, context->sp, context->ra, context->gp); #else sinfo("fp:%" PRIxREG " sp:%" PRIxREG " ra:%" PRIxREG "\n", - context->fp context->sp, context->ra); + context->fp, context->sp, context->ra); #endif #else sinfo("s5:%" PRIxREG " s6:%" PRIxREG " s7:%" PRIxREG " s8:%" PRIxREG "\n", @@ -231,7 +242,7 @@ pid_t riscv_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)(uintptr_t)context->ra); + child = nxtask_setup_fork((start_t)(uintptr_t)context->ra, vfork); if (!child) { sinfo("nxtask_setup_fork failed\n"); @@ -252,52 +263,71 @@ pid_t riscv_fork(const struct fork_s *context) sinfo("Parent: stackutil:%" PRIxPTR "\n", stackutil); - /* Make some feeble effort to preserve the stack contents. This is - * feeble because the stack surely contains invalid pointers and other - * content that will not work in the child context. However, if the - * user follows all of the caveats of fork() usage, even this feeble - * effort is overkill. - */ + if (child->stack_base_ptr == parent->stack_base_ptr) + { + /* The child is running at the parent's stack addresses, inside its + * own duplicated address environment. There is nothing to relocate: + * every stack address the child inherits is still the address it + * names. + */ - newtop = (uintptr_t)child->stack_base_ptr + child->adj_stack_size; - newsp = newtop - stackutil; + newsp = (uintptr_t)context->sp; +#ifdef CONFIG_RISCV_FRAMEPOINTER + newfp = (uintptr_t)context->fp; +#endif + } + else + { + /* Make some feeble effort to preserve the stack contents. This is + * feeble because the stack surely contains invalid pointers and other + * content that will not work in the child context. However, if the + * user follows all of the caveats of vfork() usage, even this feeble + * effort is overkill. + * + * For a POSIX fork() child the stack contents are not merely a feeble + * effort: the child is entitled to use them, and it does. + */ - /* Set up frame for context and copy the initial context there */ + newtop = (uintptr_t)child->stack_base_ptr + child->adj_stack_size; + newsp = newtop - stackutil; - memcpy((void *)(newsp - XCPTCONTEXT_SIZE), - child->xcp.regs, XCPTCONTEXT_SIZE); + /* Set up frame for context and copy the initial context there */ - /* Copy the parent stack contents (overwrites child's SP and TP) */ + memcpy((void *)(newsp - XCPTCONTEXT_SIZE), + child->xcp.regs, XCPTCONTEXT_SIZE); - memcpy((void *)newsp, (const void *)(uintptr_t)context->sp, stackutil); + /* Copy the parent stack contents (overwrites child's SP and TP) */ - /* Set the new register restore area to the new stack top */ + memcpy((void *)newsp, (const void *)(uintptr_t)context->sp, stackutil); - child->xcp.regs = (void *)(newsp - XCPTCONTEXT_SIZE); + /* Set the new register restore area to the new stack top */ - /* Was there a frame pointer in place before? */ + child->xcp.regs = (void *)(newsp - XCPTCONTEXT_SIZE); -#ifdef CONFIG_RISCV_FRAMEPOINTER - if (context->fp >= context->sp && context->fp < stacktop) - { - uintptr_t frameutil = stacktop - context->fp; - newfp = newtop - frameutil; - } - else - { - newfp = context->fp; - } + /* Was there a frame pointer in place before? */ - sinfo("Old stack top:%" PRIxPTR " SP:%" PRIxREG " FP:%" PRIxREG "\n", - stacktop, context->sp, context->fp); - sinfo("New stack top:%" PRIxPTR " SP:%" PRIxPTR " FP:%" PRIxPTR "\n", - newtop, newsp, newfp); +#ifdef CONFIG_RISCV_FRAMEPOINTER + if (context->fp >= context->sp && context->fp < stacktop) + { + uintptr_t frameutil = stacktop - context->fp; + newfp = newtop - frameutil; + } + else + { + newfp = context->fp; + } + + sinfo("Old stack top:%" PRIxPTR " SP:%" PRIxREG " FP:%" PRIxREG "\n", + stacktop, context->sp, context->fp); + sinfo("New stack top:%" PRIxPTR " SP:%" PRIxPTR " FP:%" PRIxPTR "\n", + newtop, newsp, newfp); #else - sinfo("Old stack top:%" PRIxPTR " SP:%" PRIxREG "\n", - stacktop, context->sp); - sinfo("New stack top:%" PRIxPTR " SP:%" PRIxPTR "\n", - newtop, newsp); + sinfo("Old stack top:%" PRIxPTR " SP:%" PRIxREG "\n", + stacktop, context->sp); + sinfo("New stack top:%" PRIxPTR " SP:%" PRIxPTR "\n", + newtop, newsp); #endif + } /* Update the stack pointer, frame pointer, global pointer and saved * registers. When the child TCB was initialized, all of the values @@ -346,8 +376,7 @@ pid_t riscv_fork(const struct fork_s *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } #endif /* CONFIG_LIB_SYSCALL */ -#endif /* CONFIG_ARCH_HAVE_FORK */ diff --git a/arch/sim/src/Makefile b/arch/sim/src/Makefile index 280c3a8b05041..cc6ad23c1843d 100644 --- a/arch/sim/src/Makefile +++ b/arch/sim/src/Makefile @@ -95,7 +95,7 @@ ifeq ($(CONFIG_SCHED_BACKTRACE),y) CSRCS += sim_backtrace.c endif -ifeq ($(CONFIG_ARCH_HAVE_FORK),y) +ifneq ($(CONFIG_ARCH_HAVE_VFORK)$(CONFIG_ARCH_HAVE_FORK),) CSRCS += sim_fork.c endif diff --git a/arch/sim/src/sim/CMakeLists.txt b/arch/sim/src/sim/CMakeLists.txt index a378448d1a324..046564e127432 100644 --- a/arch/sim/src/sim/CMakeLists.txt +++ b/arch/sim/src/sim/CMakeLists.txt @@ -82,7 +82,7 @@ if(CONFIG_SCHED_BACKTRACE) list(APPEND SRCS sim_backtrace.c) endif() -if(CONFIG_ARCH_HAVE_FORK) +if(CONFIG_ARCH_HAVE_VFORK OR CONFIG_ARCH_HAVE_FORK) list(APPEND SRCS sim_fork.c) endif() diff --git a/arch/sim/src/sim/sim_fork.c b/arch/sim/src/sim/sim_fork.c index 47ea655cbb65f..72ff6c98a7ef9 100644 --- a/arch/sim/src/sim/sim_fork.c +++ b/arch/sim/src/sim/sim_fork.c @@ -48,17 +48,15 @@ * Name: sim_fork * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The common simulator worker behind up_fork(). vfork() and fork() + * snapshot the caller's registers identically; `vfork' says which + * primitive was called, and is passed straight through to + * nxtask_setup_fork(), which is where the memory semantics are decided. * * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up sim_fork(). + * 1) User code calls vfork() or fork(). up_fork() collects context + * information and transfers control to sim_fork(). * 2) sim_fork() and calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: @@ -77,6 +75,10 @@ * nxtask_abort_fork() may be called if an error occurs between steps 3 and * 6. * + * Input Parameters: + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by up_fork() + * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and * returns the process ID of the child process to the parent process. @@ -88,7 +90,7 @@ #ifdef CONFIG_SIM_ASAN nosanitize_address #endif -pid_t sim_fork(const xcpt_reg_t *context) +pid_t sim_fork(bool vfork, const xcpt_reg_t *context) { struct tcb_s *parent = this_task(); struct tcb_s *child; @@ -106,7 +108,7 @@ pid_t sim_fork(const xcpt_reg_t *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)context[JB_PC]); + child = nxtask_setup_fork((start_t)context[JB_PC], vfork); if (!child) { serr("ERROR: nxtask_setup_fork failed\n"); @@ -175,5 +177,5 @@ pid_t sim_fork(const xcpt_reg_t *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); } diff --git a/arch/sim/src/sim/sim_fork_arm.S b/arch/sim/src/sim/sim_fork_arm.S index ee39f54e003b5..8449c691bea3f 100644 --- a/arch/sim/src/sim/sim_fork_arm.S +++ b/arch/sim/src/sim/sim_fork_arm.S @@ -46,20 +46,20 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the behavior is - * undefined if the process created by fork() either modifies any data other than - * a variable of type pid_t used to store the return value from fork(), or returns - * from the function in which fork() was called, or calls any other function before - * successfully calling _exit() or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). The caller says which one it is, and + * the flag is passed straight through to sim_fork(). * - * This thin layer implements fork by simply calling up_fork() with the fork() - * context as an argument. The overall sequence is: + * On the simulator the caller's context is captured with setjmp() rather + * than by hand, and the child re-enters through longjmp() -- which is why + * the entry point tests setjmp()'s return value to tell which of the two + * returns it is on. * - * 1) User code calls fork(). fork() collects context information and - * transfers control up sim_fork(). - * 2) sim_fork() and calls nxtask_setup_fork(). + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls sim_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. This * consists of: * - Allocation of the child task's TCB. @@ -75,7 +75,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * r0 - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and returns @@ -86,18 +86,28 @@ ************************************************************************************/ .text + .globl up_fork .type up_fork, @function up_fork: + /* r4 is callee-saved, so it carries the vfork flag across setjmp() */ + + push {r4, lr} + mov r4, r0 + sub sp, sp, #XCPTCONTEXT_SIZE mov r0, sp bl setjmp subs r0, #1 - jz child + beq 1f + + mov r0, r4 + mov r1, sp bl sim_fork -child: +1: add sp, sp, #XCPTCONTEXT_SIZE - ret + pop {r4, lr} + bx lr .size up_fork, . - up_fork .end diff --git a/arch/sim/src/sim/sim_fork_arm64.S b/arch/sim/src/sim/sim_fork_arm64.S index 5d813822fa620..3a88939f6bf29 100644 --- a/arch/sim/src/sim/sim_fork_arm64.S +++ b/arch/sim/src/sim/sim_fork_arm64.S @@ -55,21 +55,20 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). The caller says which one it is, and + * the flag is passed straight through to sim_fork(). * - * This thin layer implements fork by simply calling sim_fork() with the - * fork() context as an argument. The overall sequence is: + * On the simulator the caller's context is captured with setjmp() rather + * than by hand, and the child re-enters through longjmp() -- which is why + * the entry point tests setjmp()'s return value to tell which of the two + * returns it is on. * - * 1) User code calls fork(). fork() collects context information and - * transfers control up sim_fork(). - * 2) sim_fork() and calls nxtask_setup_fork(). + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls sim_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: * - Allocation of the child task's TCB. @@ -85,7 +84,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * x0 - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and @@ -96,27 +95,35 @@ ***************************************************************************/ .text - .globl SYMBOL(up_fork) .align 4 + .globl SYMBOL(up_fork) + SYMBOL(up_fork): stp x29, x30, [sp] /* save FP/LR register */ - sub sp, sp, #XCPTCONTEXT_SIZE /* area from stack for setjmp() */ - mov x0, sp /* pass stack area to setjmp() */ + /* Area from stack for setjmp(), plus a slot below it holding the vfork + * flag: setjmp() is allowed to clobber every argument register, so the + * flag cannot simply stay in one. + */ + + sub sp, sp, #XCPTCONTEXT_SIZE+16 + str x0, [sp] + + add x0, sp, #16 /* pass stack area to setjmp() */ bl SYMBOL(setjmp) /* save register for longjmp() */ subs x0, x0, #1 /* 0: parent / 1: child */ cbz x0, 1f /* child --> return */ - mov x0, sp /* pass stack area to sim_fork() */ + ldr x0, [sp] /* the vfork flag */ + add x1, sp, #16 /* pass stack area to sim_fork() */ bl SYMBOL(sim_fork) /* further process task creation */ 1: - add sp, sp, #XCPTCONTEXT_SIZE /* release area from stack */ + add sp, sp, #XCPTCONTEXT_SIZE+16 /* release area from stack */ ldp x29, x30, [sp] /* restore FP/LR register */ ret - .end diff --git a/arch/sim/src/sim/sim_fork_x86.S b/arch/sim/src/sim/sim_fork_x86.S index ec7486664c568..127dd6b92d7de 100644 --- a/arch/sim/src/sim/sim_fork_x86.S +++ b/arch/sim/src/sim/sim_fork_x86.S @@ -54,20 +54,20 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the behavior is - * undefined if the process created by fork() either modifies any data other than - * a variable of type pid_t used to store the return value from fork(), or returns - * from the function in which fork() was called, or calls any other function before - * successfully calling _exit() or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). The caller says which one it is, and + * the flag is passed straight through to sim_fork(). * - * This thin layer implements fork by simply calling up_fork() with the fork() - * context as an argument. The overall sequence is: + * On the simulator the caller's context is captured with setjmp() rather + * than by hand, and the child re-enters through longjmp() -- which is why + * the entry point tests setjmp()'s return value to tell which of the two + * returns it is on. * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) sim_fork() and calls nxtask_setup_fork(). + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls sim_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. This * consists of: * - Allocation of the child task's TCB. @@ -83,7 +83,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * arg0 - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and returns @@ -94,21 +94,35 @@ ************************************************************************************/ .text + .globl SYMBOL(up_fork) #ifdef __ELF__ .type SYMBOL(up_fork), @function #endif SYMBOL(up_fork): + /* %ebx is callee-saved, so it carries the vfork flag across setjmp() */ + + push %ebx + mov 8(%esp), %ebx + sub $XCPTCONTEXT_SIZE, %esp push %esp call SYMBOL(setjmp) sub $1, %eax - jz child + jz 1f + + /* sim_fork(vfork, context). The context pointer pushed for setjmp() + * is still in place, and is the second argument. + */ + + push %ebx call SYMBOL(sim_fork) -child: + add $4, %esp +1: add $XCPTCONTEXT_SIZE+4, %esp + pop %ebx ret #ifdef __ELF__ .size SYMBOL(up_fork), . - SYMBOL(up_fork) diff --git a/arch/sim/src/sim/sim_fork_x86_64.S b/arch/sim/src/sim/sim_fork_x86_64.S index 85b072acf866e..4d1e6412ab5cf 100644 --- a/arch/sim/src/sim/sim_fork_x86_64.S +++ b/arch/sim/src/sim/sim_fork_x86_64.S @@ -54,20 +54,20 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the behavior is - * undefined if the process created by fork() either modifies any data other than - * a variable of type pid_t used to store the return value from fork(), or returns - * from the function in which fork() was called, or calls any other function before - * successfully calling _exit() or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). The caller says which one it is, and + * the flag is passed straight through to sim_fork(). * - * This thin layer implements fork by simply calling up_fork() with the fork() - * context as an argument. The overall sequence is: + * On the simulator the caller's context is captured with setjmp() rather + * than by hand, and the child re-enters through longjmp() -- which is why + * the entry point tests setjmp()'s return value to tell which of the two + * returns it is on. * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) sim_fork() and calls nxtask_setup_fork(). + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls sim_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. This * consists of: * - Allocation of the child task's TCB. @@ -83,7 +83,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * arg0 - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and returns @@ -94,26 +94,41 @@ ************************************************************************************/ .text + .globl SYMBOL(up_fork) #ifdef __ELF__ .type SYMBOL(up_fork), @function #endif SYMBOL(up_fork): + push %rbx sub $XCPTCONTEXT_SIZE, %rsp + + /* %rbx is callee-saved, so it carries the vfork flag across setjmp() */ + #ifdef CONFIG_SIM_X8664_MICROSOFT + mov %rcx, %rbx mov %rsp, %rcx #else /* if defined(CONFIG_SIM_X8664_SYSTEMV) */ + mov %rdi, %rbx mov %rsp, %rdi #endif call SYMBOL(setjmp) sub $1, %eax - jz child + jz 1f +#ifdef CONFIG_SIM_X8664_MICROSOFT + mov %rbx, %rcx + mov %rsp, %rdx +#else /* if defined(CONFIG_SIM_X8664_SYSTEMV) */ + mov %rbx, %rdi + mov %rsp, %rsi +#endif call SYMBOL(sim_fork) -child: +1: add $XCPTCONTEXT_SIZE, %rsp + pop %rbx ret #ifdef __ELF__ .size SYMBOL(up_fork), . - SYMBOL(up_fork) diff --git a/arch/x86_64/include/intel64/irq.h b/arch/x86_64/include/intel64/irq.h index 24fd088d29cb7..84a04a4436fef 100644 --- a/arch/x86_64/include/intel64/irq.h +++ b/arch/x86_64/include/intel64/irq.h @@ -544,6 +544,18 @@ struct xcptcontext uint64_t *regs; +#ifdef CONFIG_LIB_SYSCALL + /* The register context of the user code that is currently in a system + * call, as x86_64_syscall_entry() saved it on the kernel stack. This is + * what the caller of a system call was doing, as opposed to xcp.regs, + * which during a system call describes the kernel side of it. + * x86_64_fork() needs it to build a child from the caller rather than + * from the stub. + */ + + uint64_t *sregs; +#endif + #ifdef CONFIG_ARCH_ADDRENV # ifdef CONFIG_ARCH_KERNEL_STACK /* In this configuration, all syscalls execute from an internal kernel diff --git a/arch/x86_64/src/common/CMakeLists.txt b/arch/x86_64/src/common/CMakeLists.txt index 841fa13679935..de170a4d73945 100644 --- a/arch/x86_64/src/common/CMakeLists.txt +++ b/arch/x86_64/src/common/CMakeLists.txt @@ -34,7 +34,7 @@ set(SRCS x86_64_tcbinfo.c x86_64_tlb.c) -if(CONFIG_ARCH_HAVE_FORK) +if(CONFIG_ARCH_HAVE_VFORK OR CONFIG_ARCH_HAVE_FORK) list(APPEND SRCS x86_64_fork.c fork.S) endif() diff --git a/arch/x86_64/src/common/Make.defs b/arch/x86_64/src/common/Make.defs index a5a21aff0a63a..e72813edcdabc 100644 --- a/arch/x86_64/src/common/Make.defs +++ b/arch/x86_64/src/common/Make.defs @@ -29,7 +29,7 @@ CMN_CSRCS += x86_64_getintstack.c x86_64_initialize.c x86_64_nputs.c CMN_CSRCS += x86_64_modifyreg8.c x86_64_modifyreg16.c x86_64_modifyreg32.c CMN_CSRCS += x86_64_switchcontext.c x86_64_tlb.c -ifeq ($(CONFIG_ARCH_HAVE_FORK),y) +ifneq ($(CONFIG_ARCH_HAVE_VFORK)$(CONFIG_ARCH_HAVE_FORK),) CMN_CSRCS += x86_64_fork.c CMN_ASRCS += fork.S endif diff --git a/arch/x86_64/src/common/fork.S b/arch/x86_64/src/common/fork.S index 1621b560474a1..a1e5396d04090 100644 --- a/arch/x86_64/src/common/fork.S +++ b/arch/x86_64/src/common/fork.S @@ -39,21 +39,18 @@ * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The architecture-specific entry point of both of NuttX's cloning + * primitives, vfork() and fork(). Both need exactly the same thing from + * assembly -- a snapshot of the caller's registers, stack pointer and + * return address -- and differ only in what the C code then does with it, + * so there is one entry point and the caller's %rdi says which primitive + * was called. It is passed straight through to x86_64_fork(). * - * This thin layer implements fork by simply calling up_fork() with the - * fork() context as an argument. The overall sequence is: + * The overall sequence is: * - * 1) User code calls fork(). fork() collects context information and - * transfers control up up_fork(). - * 2) x86_64_fork() and calls nxtask_setup_fork(). + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * this entry point, which collects context information and + * 2) calls x86_64_fork(), which calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: * - Allocation of the child task's TCB. @@ -69,7 +66,7 @@ * 6) nxtask_start_fork() then executes the child thread. * * Input Parameters: - * None + * %rdi - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, fork() returns 0 to the child process and @@ -91,7 +88,7 @@ * | [r15] | i * | [r14] | n * | [r13] | g - * | [r12] | <- rsp before calling x86_64_fork, rdi = rsp + * | [r12] | <- rsp before calling x86_64_fork, rsi = rsp * | ......... | */ @@ -99,13 +96,17 @@ .type up_fork, @function up_fork: + /* %rdi still holds the vfork flag on entry and is left alone; %rdx is + * the scratch used to push %ss and %cs. + */ + movq %rsp, %rax addq $8, %rax - movq %ss, %rdi - pushq %rdi + movq %ss, %rdx + pushq %rdx pushfq - movq %cs, %rdi - pushq %rdi + movq %cs, %rdx + pushq %rdx /* push %rsp */ @@ -116,7 +117,7 @@ up_fork: pushq %r14 pushq %r13 pushq %r12 - movq %rsp, %rdi + movq %rsp, %rsi subq $8, %rsp diff --git a/arch/x86_64/src/common/x86_64_fork.c b/arch/x86_64/src/common/x86_64_fork.c index ea4d90ac044b4..ef5cfb83e4519 100644 --- a/arch/x86_64/src/common/x86_64_fork.c +++ b/arch/x86_64/src/common/x86_64_fork.c @@ -35,6 +35,7 @@ #include #include +#include #include #include "x86_64_fork.h" @@ -42,56 +43,156 @@ #include "sched/sched.h" /**************************************************************************** - * Public Functions + * Pre-processor Definitions ****************************************************************************/ +#ifdef CONFIG_LIB_SYSCALL + +/* Requested privilege level 3 in a segment selector. A caller that reached + * here through the `syscall' instruction was in user mode, and SYSRETQ is + * going to put it back there with the selectors IA32_STAR describes -- see + * x86_64_fork_syscall(). + */ + +# define X86_GDT_RPL_USER 3 + +#endif + /**************************************************************************** - * Name: x86_64_fork + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: x86_64_fork_stacktop * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. + * The high end of the part of a task's stack that a fork copies: the + * bottom of the register save area that up_initial_state() reserved at the + * very top of the stack. The save area itself is not stack and must not + * be copied over -- it is where the child's own resume frame is built. * - * The overall sequence is: + * Input Parameters: + * tcb - The task whose stack is in question * - * 1) User code calls fork(). fork() collects context information and - * transfers control up x86_64_fork(). - * 2) x86_64_fork() and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: - * - Allocation of the child task's TCB. - * - Initialization of file descriptors and streams - * - Configuration of environment variables - * - Allocate and initialize the stack - * - Setup the input parameters for the task. - * - Initialization of the TCB (including call to up_initial_state()) - * 4) x86_64_fork() provides any additional operating context. It must: - * - Initialize special values in any CPU registers that were not - * already configured by up_initial_state() - * 5) x86_64_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * Returned Value: + * The address one past the last byte of stack that is copied. * - * nxtask_abort_fork() may be called if an error occurs between steps 3 and - * 6. + ****************************************************************************/ + +static uint64_t x86_64_fork_stacktop(struct tcb_s *tcb) +{ + return (uint64_t)XCP_ALIGN_DOWN((uintptr_t)tcb->stack_base_ptr + + tcb->adj_stack_size - XCPTCONTEXT_SIZE); +} + +/**************************************************************************** + * Name: x86_64_fork_reloc + * + * Description: + * Carry one address from the parent's stack over to the child's copy of + * it. Addresses outside the copied region are returned unchanged: they + * point somewhere the child shares with the parent, or somewhere that has + * no counterpart at all. * * Input Parameters: - * context - Caller context information saved by fork() + * addr - The address to relocate + * rsp - The parent's stack pointer where the primitive was called, + * which is the low end of the region that was copied + * stacktop - The high end of the region that was copied + * offset - The distance from the parent's stack to the child's copy * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * The relocated address. * ****************************************************************************/ -pid_t x86_64_fork(const struct fork_s *context) +static uint64_t x86_64_fork_reloc(uint64_t addr, uint64_t rsp, + uint64_t stacktop, uint64_t offset) +{ + if (addr >= rsp && addr < stacktop) + { + return addr + offset; + } + + return addr; +} + +/**************************************************************************** + * Name: x86_64_fork_relocfp + * + * Description: + * Relocate the saved frame-pointer chain inside the child's copy of the + * parent's stack. + * + * This is x86_64-specific and it is not optional. A function returns here + * with `leave', which is `mov %rbp,%rsp' followed by `pop %rbp': the + * frame pointer feeds the *stack* pointer. Relocating only the RBP the + * child resumes with therefore gets it exactly one frame; the moment it + * returns through the next one it loads a saved RBP that still points + * into the parent's stack, and from then on the child runs on the + * parent's stack. It looks like the child is working -- it is even at the + * right offset -- until something returns through a slot the parent has + * since reused. + * + * The other architectures with this fork path do not need it: they return + * through a link register, so a stale frame pointer spoils a backtrace and + * nothing else. + * + * The walk stops at the first link that leaves the copied region -- the + * outermost frame's saved RBP does -- and refuses to move backwards, so a + * corrupt chain terminates it rather than looping. + * + * Input Parameters: + * rbp - The parent's frame pointer where the primitive was called + * rsp - The parent's stack pointer, the low end of the copied region + * stacktop - The high end of the copied region + * offset - The distance from the parent's stack to the child's copy + * + ****************************************************************************/ + +static void x86_64_fork_relocfp(uint64_t rbp, uint64_t rsp, + uint64_t stacktop, uint64_t offset) +{ + while (rbp >= rsp && rbp < stacktop) + { + uint64_t *slot = (uint64_t *)(rbp + offset); + uint64_t next = *slot; + + if (next <= rbp || next >= stacktop) + { + break; + } + + *slot = next + offset; + rbp = next; + } +} + +/**************************************************************************** + * Name: x86_64_fork_direct + * + * Description: + * Clone a caller that reached up_fork() by an ordinary function call, so + * that the register snapshot fork.S took describes the caller itself. + * That is the case in a flat build, and for a kernel thread in any build. + * + * The child has no exception frame to inherit, so one is synthesised: it + * resumes at the caller's return address, in the caller's own segments, + * with the callee-saved registers the caller had. + * + * Input Parameters: + * vfork - true for vfork(), false for fork() + * parent - The calling task's TCB + * context - Caller context information saved by fork.S + * + * Returned Value: + * The process ID of the child, or ERROR on failure. + * + ****************************************************************************/ + +static pid_t x86_64_fork_direct(bool vfork, struct tcb_s *parent, + const struct fork_s *context) { - struct tcb_s *parent = this_task(); struct tcb_s *child; uint64_t newsp; uint64_t newfp; @@ -110,7 +211,7 @@ pid_t x86_64_fork(const struct fork_s *context) /* Allocate and initialize a TCB for the child task. */ - child = nxtask_setup_fork((start_t)context->rip); + child = nxtask_setup_fork((start_t)context->rip, vfork); if (!child) { serr("ERROR: nxtask_setup_fork failed\n"); @@ -119,57 +220,62 @@ pid_t x86_64_fork(const struct fork_s *context) sinfo("TCBs: Parent=%p Child=%p\n", parent, child); - /* How much of the parent's stack was utilized? The ARM uses - * a push-down stack so that the current stack pointer should - * be lower than the initial, adjusted stack pointer. The - * stack usage should be the difference between those two. + /* How much of the parent's stack was utilized? x86_64 uses a push-down + * stack so that the current stack pointer should be lower than the + * initial, adjusted stack pointer. The stack usage should be the + * difference between those two. */ - stacktop = (uint64_t)XCP_ALIGN_DOWN((uintptr_t)parent->stack_base_ptr + - parent->adj_stack_size - - XCPTCONTEXT_SIZE); + stacktop = x86_64_fork_stacktop(parent); DEBUGASSERT(stacktop > context->rsp); stackutil = stacktop - context->rsp; sinfo("Parent: stackutil:%" PRIu64 "\n", stackutil); - /* Make some feeble effort to preserve the stack contents. This is - * feeble because the stack surely contains invalid pointers and other - * content that will not work in the child context. However, if the - * user follows all of the caveats of fork() usage, even this feeble - * effort is overkill. - */ - - newtop = (uint64_t)XCP_ALIGN_DOWN((uintptr_t)child->stack_base_ptr + - child->adj_stack_size - - XCPTCONTEXT_SIZE); - - newsp = newtop - stackutil; - - /* Move the register context (from parent) to newtop. */ + /* Move the register context (from parent) to the child. */ memcpy(child->xcp.regs, parent->xcp.regs, XCPTCONTEXT_SIZE); - memcpy((void *)newsp, (const void *)context->rsp, stackutil); - - /* Was there a frame pointer in place before? */ - - if (context->rbp >= context->rsp && context->rbp < stacktop) + if (child->stack_base_ptr == parent->stack_base_ptr) { - uint32_t frameutil = stacktop - context->rbp; - newfp = newtop - frameutil; + /* The child is running at the parent's stack addresses, inside its + * own duplicated address environment. There is nothing to relocate: + * every stack address the child inherits is still the address it + * names. + */ + + newsp = context->rsp; + newfp = context->rbp; } else { - newfp = context->rbp; - } + /* Make some feeble effort to preserve the stack contents. This is + * feeble because the stack surely contains invalid pointers and other + * content that will not work in the child context. However, if the + * user follows all of the caveats of vfork() usage, even this feeble + * effort is overkill. + * + * For a POSIX fork() child the stack contents are not merely a feeble + * effort: the child is entitled to use them, and it does. + */ + + newtop = x86_64_fork_stacktop(child); + newsp = newtop - stackutil; + + memcpy((void *)newsp, (const void *)context->rsp, stackutil); - /* We do not need to update the frame-pointer */ + /* Was there a frame pointer in place before? */ - sinfo("Old stack top:%08" PRIx64 " RSP:%08" PRIx64 " RBP:%08" PRIx64 "\n", - stacktop, context->rsp, context->rbp); - sinfo("New stack top:%08" PRIx64 " RSP:%08" PRIx64 "\n", - newtop, newsp); + newfp = x86_64_fork_reloc(context->rbp, context->rsp, stacktop, + newtop - stacktop); + x86_64_fork_relocfp(context->rbp, context->rsp, stacktop, + newtop - stacktop); + + sinfo("Old stack top:%08" PRIx64 " RSP:%08" PRIx64 + " RBP:%08" PRIx64 "\n", stacktop, context->rsp, context->rbp); + sinfo("New stack top:%08" PRIx64 " RSP:%08" PRIx64 "\n", + newtop, newsp); + } /* Update the stack pointer, frame pointer, and volatile registers. When * the child TCB was initialized, all of the values were set to zero. @@ -195,5 +301,244 @@ pid_t x86_64_fork(const struct fork_s *context) * will discard the TCB by calling nxtask_abort_fork(). */ - return nxtask_start_fork(child); + return nxtask_start_fork(child, vfork); +} + +#ifdef CONFIG_LIB_SYSCALL + +/**************************************************************************** + * Name: x86_64_fork_syscall + * + * Description: + * Clone a caller that reached up_fork() through a system call. The + * register snapshot fork.S took is useless here: it describes the + * kernel-side stub, so a child built from it would resume at a kernel + * address on a kernel stack. What the caller was actually doing is the + * frame x86_64_syscall_entry() saved and x86_64_syscall() recorded in + * xcp.sregs; the child is built from that. + * + * The child therefore returns from the very same `syscall' instruction as + * the parent, in user mode, differing only in that it sees 0 as the return + * value and runs on its own stack. + * + * Two details of the SYSCALL/SYSRET pair shape this: + * + * 1. `syscall' does not save the caller's RIP and RFLAGS on a stack; it + * leaves them in RCX and R11, which is where the saved frame has them. + * The child is resumed by IRETQ (x86_64_fullcontextrestore()), so they + * have to be moved into the RIP and RFLAGS slots of its frame. + * 2. The hardware never tells the kernel which CS and SS the caller had -- + * SYSRETQ reconstructs them from IA32_STAR -- so those slots of the + * saved frame hold nothing, and the child's have to be filled with the + * selectors SYSRETQ would have produced, which is where the parent is + * about to return to. + * + * Everything the frame does hold -- the general registers and the extended + * (FPU/SSE) state -- is inherited. Everything it does not is taken from + * the frame up_initial_state() built for the child, so that the child + * keeps its own segment registers and, importantly, its own thread + * pointer: the child's stack is a fresh allocation at a different virtual + * address, so the parent's FS base does not describe it. + * + * Input Parameters: + * vfork - true for vfork(), false for fork() + * parent - The calling task's TCB + * + * Returned Value: + * The process ID of the child, or ERROR on failure. + * + ****************************************************************************/ + +static pid_t x86_64_fork_syscall(bool vfork, struct tcb_s *parent) +{ + uint64_t *sregs = parent->xcp.sregs; + struct tcb_s *child; + uint64_t newsp; + uint64_t newtop; + uint64_t offset; + uint64_t stacktop; + uint64_t stackutil; + uint64_t rsp; + uint64_t rip; + + DEBUGASSERT(sregs != NULL); + + /* Where the caller was and what it was doing */ + + rsp = sregs[REG_RSP]; + rip = sregs[REG_RCX]; + + sinfo("syscall frame [%p]: RSP:%08" PRIx64 " RIP:%08" PRIx64 "\n", + sregs, rsp, rip); + + /* Allocate and initialize a TCB for the child task. The child resumes at + * the instruction after the `syscall', which is where the parent resumes + * too. + */ + + child = nxtask_setup_fork((start_t)rip, vfork); + if (!child) + { + serr("ERROR: nxtask_setup_fork failed\n"); + return (pid_t)ERROR; + } + + sinfo("TCBs: Parent=%p Child=%p\n", parent, child); + + /* Give the child the part of the parent's stack that is in use, copied to + * the same place in its own stack. The copy is aligned with the top of + * each stack rather than the bottom, so a single offset carries any + * address in the copied region from one to the other. + */ + + stacktop = x86_64_fork_stacktop(parent); + DEBUGASSERT(stacktop > rsp); + stackutil = stacktop - rsp; + + newtop = x86_64_fork_stacktop(child); + newsp = newtop - stackutil; + offset = newtop - stacktop; + + memcpy((void *)newsp, (const void *)rsp, stackutil); + + sinfo("Old stack top:%08" PRIx64 " RSP:%08" PRIx64 "\n", stacktop, rsp); + sinfo("New stack top:%08" PRIx64 " RSP:%08" PRIx64 "\n", newtop, newsp); + + /* Inherit the parent's extended (FPU/SSE) state, which + * x86_64_syscall_entry saved at the front of the frame, exactly where + * the child's belongs. + */ + + memcpy(child->xcp.regs, sregs, XCPTCONTEXT_XMM_AREA_SIZE); + + /* Inherit the general registers. RCX and R11 are included deliberately: + * SYSRETQ leaves the return address in RCX and RFLAGS in R11, so the + * parent resumes with those values and the child must too. + */ + + child->xcp.regs[REG_RBX] = sregs[REG_RBX]; + child->xcp.regs[REG_R8] = sregs[REG_R8]; + child->xcp.regs[REG_R9] = sregs[REG_R9]; + child->xcp.regs[REG_R10] = sregs[REG_R10]; + child->xcp.regs[REG_R11] = sregs[REG_R11]; + child->xcp.regs[REG_R12] = sregs[REG_R12]; + child->xcp.regs[REG_R13] = sregs[REG_R13]; + child->xcp.regs[REG_R14] = sregs[REG_R14]; + child->xcp.regs[REG_R15] = sregs[REG_R15]; + child->xcp.regs[REG_RCX] = sregs[REG_RCX]; + child->xcp.regs[REG_RDX] = sregs[REG_RDX]; + child->xcp.regs[REG_RSI] = sregs[REG_RSI]; + child->xcp.regs[REG_RDI] = sregs[REG_RDI]; + + /* The frame pointer moves with the stack it points into */ + + child->xcp.regs[REG_RBP] = x86_64_fork_reloc(sregs[REG_RBP], rsp, + stacktop, offset); + x86_64_fork_relocfp(sregs[REG_RBP], rsp, stacktop, offset); + + /* Build the interrupt frame the child is resumed from. RIP and RFLAGS + * come out of RCX and R11, and the selectors are the ones SYSRETQ derives + * from IA32_STAR: CS is the user code segment and SS the user data + * segment, both at RPL 3. See x86_64_cpu_priv_set(), which programs + * IA32_STAR. + */ + + child->xcp.regs[REG_RAX] = 0; + child->xcp.regs[REG_RIP] = rip; + child->xcp.regs[REG_RFLAGS] = sregs[REG_R11]; + child->xcp.regs[REG_RSP] = newsp; + child->xcp.regs[REG_CS] = X86_GDT_USERCODE_SEL | X86_GDT_RPL_USER; + child->xcp.regs[REG_SS] = X86_GDT_USERDATA_SEL | X86_GDT_RPL_USER; + +#ifdef CONFIG_ARCH_KERNEL_STACK + /* The child's own user stack pointer, for the signal dispatch path */ + + child->xcp.ustkptr = (uintptr_t *)newsp; +#endif + + /* And, finally, start the child task. On a failure, nxtask_start_fork() + * will discard the TCB by calling nxtask_abort_fork(). + */ + + return nxtask_start_fork(child, vfork); +} + +#endif /* CONFIG_LIB_SYSCALL */ + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: x86_64_fork + * + * Description: + * The common x86_64 worker behind up_fork(). vfork() and fork() snapshot + * the caller's registers identically; `vfork' says which primitive was + * called, and is passed straight through to nxtask_setup_fork(), which is + * where the memory semantics are decided. + * + * The overall sequence is: + * + * 1) User code calls vfork() or fork(). The libc wrapper enters + * up_fork(), which collects context information and transfers control + * to x86_64_fork(). + * 2) x86_64_fork() calls nxtask_setup_fork(). + * 3) nxtask_setup_fork() allocates and configures the child task's TCB. + * This consists of: + * - Allocation of the child task's TCB. + * - Initialization of file descriptors and streams + * - Configuration of environment variables + * - Allocate and initialize the stack + * - Setup the input parameters for the task. + * - Initialization of the TCB (including call to up_initial_state()) + * 4) x86_64_fork() provides any additional operating context. It must: + * - Initialize special values in any CPU registers that were not + * already configured by up_initial_state() + * 5) x86_64_fork() then calls nxtask_start_fork() + * 6) nxtask_start_fork() then executes the child thread. + * + * nxtask_abort_fork() may be called if an error occurs between steps 3 and + * 6. + * + * Everything above is common to the two ways this can be reached, which + * differ only in where the caller's registers are to be found -- see + * x86_64_fork_direct() and x86_64_fork_syscall(). + * + * Input Parameters: + * vfork - true for vfork(), false for fork() + * context - Caller context information saved by fork.S + * + * Returned Value: + * Upon successful completion, 0 is returned to the child and the process + * ID of the child is returned to the parent. Otherwise, -1 is returned to + * the parent, no child is created, and errno is set to indicate the error. + * + ****************************************************************************/ + +pid_t x86_64_fork(bool vfork, const struct fork_s *context) +{ + struct tcb_s *parent = this_task(); + +#ifdef CONFIG_LIB_SYSCALL + /* A non-NULL xcp.sregs means a system call is in progress: + * x86_64_syscall() publishes the caller's frame there for the duration of + * the call and nowhere else. So this was reached from a kernel-side stub, + * and the caller to clone is the user task that trapped, not the code that + * called into fork.S. + * + * arm64 and RISC-V discriminate on TCB_FLAG_SYSCALL instead. x86_64 + * cannot: that flag also defers signal actions, which x86_64 has never + * done and which its kernel-build signal path does not currently survive + * -- see the note in x86_64_syscall(). xcp.sregs says exactly what is + * needed here and means nothing to anyone else. + */ + + if (parent->xcp.sregs != NULL) + { + return x86_64_fork_syscall(vfork, parent); + } +#endif + + return x86_64_fork_direct(vfork, parent, context); } diff --git a/arch/x86_64/src/common/x86_64_syscall.c b/arch/x86_64/src/common/x86_64_syscall.c index 4e92a68641026..651d0d28a44f8 100644 --- a/arch/x86_64/src/common/x86_64_syscall.c +++ b/arch/x86_64/src/common/x86_64_syscall.c @@ -301,15 +301,42 @@ uint64_t *x86_64_syscall(uint64_t *regs) #ifdef CONFIG_LIB_SYSCALL int nbr = cmd - CONFIG_SYS_RESERVED; syscall_stub_t stub = (syscall_stub_t)g_stublookup[nbr]; - -#ifdef CONFIG_ARCH_KERNEL_STACK struct tcb_s *rtcb = nxsched_self(); + uint64_t *sregs; +#ifdef CONFIG_ARCH_KERNEL_STACK /* Store reference to user RSP for signals */ rtcb->xcp.saved_ursp = regs[REG_RSP]; #endif + /* Publish the caller's register context. up_fork() has to clone + * the caller rather than the stub that is about to invoke it, and + * this frame is the only description of it -- see + * x86_64_fork_syscall(), which also takes a non-NULL xcp.sregs as + * its "reached here through a system call" discriminator. + * + * It is saved and restored rather than simply set and cleared: + * x86_64_syscall_entry() has an explicit path for a nested system + * call, and when the inner one returns the outer one must still be + * described by its own frame. + * + * Note what is deliberately *not* done here. arm64 and RISC-V + * also raise TCB_FLAG_SYSCALL across the stub call, which defers + * any signal action until the system call returns. x86_64 has + * never set it, and making it do so is not free: the deferred + * action then has to be picked up by nxsig_unmask_pendingsignal() + * on the way out, and the signal dispatch path of an x86_64 kernel + * build does not survive that today -- it faults in + * x86_64_syscall_entry()'s return path with RSP == 0. That is a + * pre-existing bug in a configuration nothing has exercised, and + * fixing it does not belong to the fork/vfork work; so this + * records the frame and changes nothing else. + */ + + sregs = rtcb->xcp.sregs; + rtcb->xcp.sregs = regs; + /* Re-enable interrupts if enabled before. * Current task RFLAGS are stored in R11. */ @@ -322,6 +349,10 @@ uint64_t *x86_64_syscall(uint64_t *regs) /* Call syscall function and store return value in RAX register */ regs[REG_RAX] = stub(nbr, arg1, arg2, arg3, arg4, arg5, arg6); + + /* The system call is now done */ + + rtcb->xcp.sregs = sregs; #else svcerr("ERROR: Bad SYS call: %" PRId32 "\n", cmd); #endif diff --git a/include/nuttx/addrenv.h b/include/nuttx/addrenv.h index 8e253c88eee1b..91c2eaa28d4d2 100644 --- a/include/nuttx/addrenv.h +++ b/include/nuttx/addrenv.h @@ -394,6 +394,31 @@ int addrenv_attach(FAR struct tcb_s *tcb, FAR struct addrenv_s *addrenv); int addrenv_join(FAR struct tcb_s *ptcb, FAR struct tcb_s *tcb); +/**************************************************************************** + * Name: addrenv_fork + * + * Description: + * Duplicate the parent's address environment for a POSIX fork() child and + * attach it: the child gets its own pages holding a copy of the parent's + * contents, mapped at the same virtual addresses. Contrast + * addrenv_join(), which gives the child the parent's memory. + * + * Input Parameters: + * ptcb - The tcb of the parent process. + * tcb - The tcb of the child process. + * + * Returned Value: + * This is a NuttX internal function so it follows the convention that + * 0 (OK) is returned on success and a negated errno is returned on + * failure. -ENOMEM is returned if there is not enough free memory to + * hold a copy of the parent. + * + ****************************************************************************/ + +#ifdef CONFIG_ARCH_HAVE_FORK +int addrenv_fork(FAR struct tcb_s *ptcb, FAR struct tcb_s *tcb); +#endif + /**************************************************************************** * Name: addrenv_leave * diff --git a/include/nuttx/arch.h b/include/nuttx/arch.h index e1a598c222b2d..91492fefe9c54 100644 --- a/include/nuttx/arch.h +++ b/include/nuttx/arch.h @@ -253,8 +253,18 @@ extern initializer_t _einit[]; * Name: up_fork * * Description: - * The up_fork() function is the base of fork() function that provided in - * libc, and fork() is implemented as a wrapper of up_fork() function. + * Architecture-specific base of both cloning primitives. It snapshots the + * caller's registers and hands them to the common code, which builds the + * child from them; `vfork' says which primitive was called and so which + * memory semantics the child gets. + * + * Input Parameters: + * vfork - true for vfork(): the child shares the parent's memory and the + * parent is suspended until the child _exit()s or exec()s. + * false for POSIX fork(): the child receives its own copy of the + * parent's memory at the same virtual addresses and runs + * concurrently. Only available where CONFIG_ARCH_HAVE_FORK is + * selected. * * Returned Value: * Upon successful completion, up_fork() returns 0 to the child process @@ -264,7 +274,9 @@ extern initializer_t _einit[]; * ****************************************************************************/ -pid_t up_fork(void); +#if defined(CONFIG_ARCH_HAVE_VFORK) || defined(CONFIG_ARCH_HAVE_FORK) +pid_t up_fork(bool vfork); +#endif /**************************************************************************** * Name: up_initialize @@ -1327,6 +1339,35 @@ int up_addrenv_clone(FAR const arch_addrenv_t *src, FAR arch_addrenv_t *dest); #endif +/**************************************************************************** + * Name: up_addrenv_fork + * + * Description: + * Duplicate an address environment for POSIX fork(): allocate fresh + * pages for the destination, copy the source's contents into them, and map + * them at the same virtual addresses. Unlike up_addrenv_clone(), which + * copies only the representation and leaves both pointing at the same page + * tables, the result is independent of the source. + * + * Implemented only where CONFIG_ARCH_HAVE_FORK is selected. + * + * Input Parameters: + * src - The address environment to be duplicated. + * dest - The location to receive the duplicate. It is wiped by this + * function before anything is allocated into it. + * + * Returned Value: + * Zero (OK) on success; a negated errno value on failure. -ENOMEM is + * returned if there are not enough free pages to hold the copy, in which + * case nothing is left allocated. + * + ****************************************************************************/ + +#ifdef CONFIG_ARCH_HAVE_FORK +int up_addrenv_fork(FAR const arch_addrenv_t *src, + FAR arch_addrenv_t *dest); +#endif + /**************************************************************************** * Name: up_addrenv_attach * diff --git a/include/nuttx/sched.h b/include/nuttx/sched.h index 76630d3f10110..9778282fecd94 100644 --- a/include/nuttx/sched.h +++ b/include/nuttx/sched.h @@ -650,6 +650,14 @@ struct tcb_s /* after the frame has been */ /* removed from the stack. */ + /* vfork() Support ********************************************************/ + +#ifdef CONFIG_ARCH_HAVE_VFORK + FAR sem_t *vfork_rel; /* Non-NULL in a vfork() child: */ + /* the suspended parent to release */ + /* when this task is torn down. */ +#endif + /* External Module Support ************************************************/ #ifdef CONFIG_PIC @@ -1134,23 +1142,35 @@ void nxtask_startup(main_t entrypt, int argc, FAR char *argv[]); #endif /**************************************************************************** - * Internal fork support. The overall sequence is: - * - * 1) User code calls fork(). fork() is provided in architecture-specific - * code. - * 2) fork()and calls nxtask_setup_fork(). + * Internal support for the two cloning primitives, vfork() and fork(). The + * sequence below is common to both, and `vfork' says which one was called: + * + * vfork() the child shares the parent's memory and the parent is + * suspended until the child _exit()s or exec()s. + * fork() the child gets its own copy of the parent's memory at the same + * virtual addresses, and both run. + * + * 1) User code calls vfork() or fork(). Both are libc wrappers around + * up_fork(), which is provided in architecture-specific code. + * 2) The architecture-specific code snapshots the caller's registers and + * calls nxtask_setup_fork(). * 3) nxtask_setup_fork() allocates and configures the child task's TCB. * This consists of: * - Allocation of the child task's TCB. * - Initialization of file descriptors and streams * - Configuration of environment variables - * - Allocate and initialize the stack + * - Establishing the child's address environment: joined to the parent's + * for vfork(), duplicated from it for fork() + * - Allocating the stack, or inheriting the parent's for fork() * - Setup the input parameters for the task. * - Initialization of the TCB (including call to up_initial_state()) - * 4) fork() provides any additional operating context. fork must: + * 4) The architecture-specific code provides any additional operating + * context: * - Initialize special values in any CPU registers that were not * already configured by up_initial_state() - * 5) fork() then calls nxtask_start_fork() + * - Relocate the copied stack, unless the child shares the parent's + * 5) It then calls nxtask_start_fork(), which for vfork() additionally + * suspends the caller. * 6) nxtask_start_fork() then executes the child thread. * * nxtask_abort_fork() may be called if an error occurs between @@ -1158,8 +1178,8 @@ void nxtask_startup(main_t entrypt, int argc, FAR char *argv[]); * ****************************************************************************/ -FAR struct tcb_s *nxtask_setup_fork(start_t retaddr); -pid_t nxtask_start_fork(FAR struct tcb_s *child); +FAR struct tcb_s *nxtask_setup_fork(start_t retaddr, bool vfork); +pid_t nxtask_start_fork(FAR struct tcb_s *child, bool vfork); void nxtask_abort_fork(FAR struct tcb_s *child, int errcode); /**************************************************************************** diff --git a/include/sys/syscall_lookup.h b/include/sys/syscall_lookup.h index 97b1cee5b0532..dc85c2d76771a 100644 --- a/include/sys/syscall_lookup.h +++ b/include/sys/syscall_lookup.h @@ -116,8 +116,8 @@ SYSCALL_LOOKUP(nxsem_wait_slow, 1) /* The following can be individually enabled */ -#ifdef CONFIG_ARCH_HAVE_FORK - SYSCALL_LOOKUP(up_fork, 0) +#if defined(CONFIG_ARCH_HAVE_VFORK) || defined(CONFIG_ARCH_HAVE_FORK) + SYSCALL_LOOKUP(up_fork, 1) #endif #ifdef CONFIG_SCHED_WAITPID diff --git a/include/unistd.h b/include/unistd.h index e9d1686248c9f..885bbed12a532 100644 --- a/include/unistd.h +++ b/include/unistd.h @@ -347,8 +347,17 @@ extern "C" /* Task Control Interfaces */ +/* fork() is declared only where POSIX fork() semantics can be provided, so + * that calling it elsewhere is a build error rather than a silent change of + * meaning. + */ + +#ifdef CONFIG_ARCH_HAVE_FORK pid_t fork(void); +#endif +#ifdef CONFIG_ARCH_HAVE_VFORK pid_t vfork(void); +#endif pid_t getpid(void); pid_t getpgid(pid_t pid); pid_t getpgrp(void); diff --git a/libs/libbuiltin/libgcc/gcov.c b/libs/libbuiltin/libgcc/gcov.c index c73aa620b2301..134b6aa79ac3c 100644 --- a/libs/libbuiltin/libgcc/gcov.c +++ b/libs/libbuiltin/libgcc/gcov.c @@ -468,10 +468,16 @@ void __gcov_execle(void) { } +/* GCC redirects fork() in instrumented code to __gcov_fork(), so this is + * reachable only where unistd.h declares fork() at all. + */ + +#ifdef CONFIG_ARCH_HAVE_FORK pid_t __gcov_fork(void) { return fork(); } +#endif void __gcov_dump(void) { diff --git a/libs/libc/libc.csv b/libs/libc/libc.csv index 4600f5329f96a..2e2d13506cfc9 100644 --- a/libs/libc/libc.csv +++ b/libs/libc/libc.csv @@ -348,6 +348,7 @@ "usleep","unistd.h","","int","useconds_t" "vasprintf","stdio.h","","int","FAR char **","FAR const IPTR char *","va_list" "versionsort","dirent.h","","int","FAR const struct dirent **","FAR const struct dirent **" +"vfork","unistd.h","!defined(CONFIG_BUILD_KERNEL) && defined(CONFIG_ARCH_HAVE_VFORK)","pid_t" "vfprintf","stdio.h","defined(CONFIG_FILE_STREAM)","int","FAR FILE *","FAR const IPTR char *","va_list" "vprintf","stdio.h","","int","FAR const IPTR char *","va_list" "vscanf","stdio.h","defined(CONFIG_FILE_STREAM)","int","FAR const IPTR char *","va_list" diff --git a/libs/libc/unistd/CMakeLists.txt b/libs/libc/unistd/CMakeLists.txt index 3521d45f607eb..3bcc3a6004e49 100644 --- a/libs/libc/unistd/CMakeLists.txt +++ b/libs/libc/unistd/CMakeLists.txt @@ -104,7 +104,7 @@ if(NOT CONFIG_DISABLE_MOUNTPOINTS) list(APPEND SRCS lib_truncate.c lib_posix_fallocate.c) endif() -if(CONFIG_ARCH_HAVE_FORK) +if(CONFIG_ARCH_HAVE_VFORK OR CONFIG_ARCH_HAVE_FORK) list(APPEND SRCS lib_fork.c) endif() diff --git a/libs/libc/unistd/Make.defs b/libs/libc/unistd/Make.defs index 602887be4c33a..a2ea08697a0e2 100644 --- a/libs/libc/unistd/Make.defs +++ b/libs/libc/unistd/Make.defs @@ -54,7 +54,7 @@ ifneq ($(CONFIG_DISABLE_MOUNTPOINTS),y) CSRCS += lib_truncate.c lib_posix_fallocate.c endif -ifeq ($(CONFIG_ARCH_HAVE_FORK),y) +ifneq ($(CONFIG_ARCH_HAVE_VFORK)$(CONFIG_ARCH_HAVE_FORK),) CSRCS += lib_fork.c endif diff --git a/libs/libc/unistd/lib_fork.c b/libs/libc/unistd/lib_fork.c index 53c2bfe0e9b2c..84813db3265b5 100644 --- a/libs/libc/unistd/lib_fork.c +++ b/libs/libc/unistd/lib_fork.c @@ -34,8 +34,6 @@ #include #include -#if defined(CONFIG_ARCH_HAVE_FORK) - /**************************************************************************** * Private Functions ****************************************************************************/ @@ -137,27 +135,38 @@ static void atfork_parent(void) ****************************************************************************/ /**************************************************************************** - * Name: fork + * Name: vfork * * Description: - * The fork() function is a wrapper of up_fork() syscall + * The vfork() function is equivalent to fork(), except that the behavior + * is undefined if the process created by vfork() either modifies any data + * other than a variable of type pid_t used to store the return value from + * vfork(), or returns from the function in which vfork() was called, or + * calls any other function before successfully calling _exit() or one of + * the exec family of functions. + * + * The child shares the parent's memory and the parent is suspended until + * the child _exit()s or exec()s. The suspension lives in the kernel, so + * vfork() does not depend on CONFIG_SCHED_WAITPID. Wrapper of the + * up_fork() syscall. * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and + * Upon successful completion, vfork() returns 0 to the child process and * returns the process ID of the child process to the parent process. * Otherwise, -1 is returned to the parent, no child process is created, * and errno is set to indicate the error. * ****************************************************************************/ -pid_t fork(void) +#ifdef CONFIG_ARCH_HAVE_VFORK +pid_t vfork(void) { pid_t pid; #ifdef CONFIG_PTHREAD_ATFORK atfork_prepare(); #endif - pid = up_fork(); + pid = up_fork(true); #ifdef CONFIG_PTHREAD_ATFORK if (pid == 0) @@ -172,39 +181,38 @@ pid_t fork(void) return pid; } - -#if defined(CONFIG_SCHED_WAITPID) +#endif /* CONFIG_ARCH_HAVE_VFORK */ /**************************************************************************** - * Public Functions - ****************************************************************************/ - -/**************************************************************************** - * Name: vfork + * Name: fork * * Description: - * The vfork() function is implemented based on fork() function, on - * vfork(), the parent task need to wait until the child task is performing - * exec or running finished. + * POSIX fork(). The child receives its own copy of the parent's memory, + * at the same virtual addresses. It may modify anything, call anything, + * return from the function that called fork(), and it runs concurrently + * with the parent. None of vfork()'s restrictions apply. + * + * Provided only where CONFIG_ARCH_HAVE_FORK is selected; elsewhere fork() + * is not declared at all, so calling it is a build error. Wrapper of the + * up_fork() syscall. * * Returned Value: - * Upon successful completion, vfork() returns 0 to the child process and + * Upon successful completion, fork() returns 0 to the child process and * returns the process ID of the child process to the parent process. * Otherwise, -1 is returned to the parent, no child process is created, * and errno is set to indicate the error. * ****************************************************************************/ -pid_t vfork(void) +#ifdef CONFIG_ARCH_HAVE_FORK +pid_t fork(void) { - int status = 0; - int ret; pid_t pid; #ifdef CONFIG_PTHREAD_ATFORK atfork_prepare(); #endif - pid = up_fork(); + pid = up_fork(false); #ifdef CONFIG_PTHREAD_ATFORK if (pid == 0) @@ -217,22 +225,6 @@ pid_t vfork(void) } #endif - if (pid != 0) - { - /* we are in parent task, and we need to wait the child task - * until running finished or performing exec - */ - - ret = waitpid(pid, &status, WNOWAIT); - if (ret < 0) - { - serr("ERROR: waitpid failed: %d\n", get_errno()); - } - } - return pid; } - -#endif /* CONFIG_SCHED_WAITPID */ - #endif /* CONFIG_ARCH_HAVE_FORK */ diff --git a/sched/addrenv/addrenv.c b/sched/addrenv/addrenv.c index 9936a105c774c..32e6cba247c67 100644 --- a/sched/addrenv/addrenv.c +++ b/sched/addrenv/addrenv.c @@ -27,6 +27,7 @@ #include #include +#include #include #include @@ -292,6 +293,70 @@ int addrenv_join(FAR struct tcb_s *ptcb, FAR struct tcb_s *tcb) return OK; } +#ifdef CONFIG_ARCH_HAVE_FORK +/**************************************************************************** + * Name: addrenv_fork + * + * Description: + * Duplicate the parent process's address environment for a POSIX fork() + * child, and attach the duplicate to the child. + * + * This is the counterpart of addrenv_join(): where join gives the child + * the parent's memory, fork gives it a copy -- its own pages, holding a + * snapshot of the parent's contents, mapped at the same virtual addresses. + * Mapping at the same addresses is what lets the copy be exact: every + * pointer the parent held into its own memory remains valid in the child, + * including the pointers inside the copied heap's own metadata. + * + * The copy is eager -- there is no copy-on-write, because NuttX has no + * demand paging to build it on -- so forking a large process needs as much + * free memory as the process occupies, and fails with -ENOMEM if that is + * not available. That is the nature of the primitive on this class of + * system, not a defect of this implementation; spawn-heavy code should + * prefer posix_spawn() or vfork(). + * + * Input Parameters: + * ptcb - The tcb of the parent process + * tcb - The tcb of the child process + * + * Returned Value: + * This is a NuttX internal function so it follows the convention that + * 0 (OK) is returned on success and a negated errno is returned on + * failure. + * + ****************************************************************************/ + +int addrenv_fork(FAR struct tcb_s *ptcb, FAR struct tcb_s *tcb) +{ + FAR struct addrenv_s *addrenv; + int ret; + + DEBUGASSERT(ptcb->addrenv_own != NULL); + + addrenv = addrenv_allocate(); + if (addrenv == NULL) + { + return -ENOMEM; + } + + /* Duplicate the parent's regions into freshly allocated pages, mapped at + * the same virtual addresses. + */ + + ret = up_addrenv_fork(&ptcb->addrenv_own->addrenv, &addrenv->addrenv); + if (ret < 0) + { + berr("ERROR: up_addrenv_fork failed: %d\n", ret); + addrenv_drop(addrenv, false); + return ret; + } + + /* Hand the reference taken by addrenv_allocate() to the child */ + + return addrenv_attach(tcb, addrenv); +} +#endif /* CONFIG_ARCH_HAVE_FORK */ + /**************************************************************************** * Name: addrenv_leave * diff --git a/sched/sched/sched.h b/sched/sched/sched.h index 8c3aff1d4265d..b06cd97cbc2cd 100644 --- a/sched/sched/sched.h +++ b/sched/sched/sched.h @@ -319,6 +319,16 @@ void nxsched_remove_self(FAR struct tcb_s *rtrtcb); void nxsched_add_blocked(FAR struct tcb_s *btcb, tstate_t task_state); void nxsched_remove_blocked(FAR struct tcb_s *btcb); int nxsched_set_priority(FAR struct tcb_s *tcb, int sched_priority); + +/* Release the vfork() parent suspended on this child, if there is one. + * Called from nxsched_release_tcb(), the last point in the child's life -- + * by which time an exec()ing child has already handed its pid to the + * program it loaded. + */ + +#ifdef CONFIG_ARCH_HAVE_VFORK +void nxtask_resume_vfork(FAR struct tcb_s *child); +#endif #ifndef CONFIG_SMP bool nxsched_merge_pending(void); bool nxsched_reprioritize_rtr(FAR struct tcb_s *tcb, int priority); diff --git a/sched/sched/sched_releasetcb.c b/sched/sched/sched_releasetcb.c index 96b3e736e79cd..1f9d8fe1293f8 100644 --- a/sched/sched/sched_releasetcb.c +++ b/sched/sched/sched_releasetcb.c @@ -174,6 +174,15 @@ int nxsched_release_tcb(FAR struct tcb_s *tcb, uint8_t ttype) nxtask_joindestroy(tcb); #endif +#ifdef CONFIG_ARCH_HAVE_VFORK + /* Release a suspended vfork() parent here, the last point in the + * child's life: exec_swap() has already handed its pid to any + * program it loaded. + */ + + nxtask_resume_vfork(tcb); +#endif + /* And, finally, release the TCB itself */ if (tcb->flags & TCB_FLAG_FREE_TCB) diff --git a/sched/task/CMakeLists.txt b/sched/task/CMakeLists.txt index fdc19fdc589b1..7f6817e3dfcd9 100644 --- a/sched/task/CMakeLists.txt +++ b/sched/task/CMakeLists.txt @@ -46,7 +46,7 @@ if(CONFIG_SCHED_HAVE_PARENT) list(APPEND SRCS task_getppid.c task_reparent.c) endif() -if(CONFIG_ARCH_HAVE_FORK) +if(CONFIG_ARCH_HAVE_VFORK OR CONFIG_ARCH_HAVE_FORK) list(APPEND SRCS task_fork.c) endif() diff --git a/sched/task/Make.defs b/sched/task/Make.defs index 1fd24403b51ae..6c5857f63017e 100644 --- a/sched/task/Make.defs +++ b/sched/task/Make.defs @@ -30,7 +30,7 @@ ifeq ($(CONFIG_SCHED_HAVE_PARENT),y) CSRCS += task_getppid.c task_reparent.c endif -ifeq ($(CONFIG_ARCH_HAVE_FORK),y) +ifneq ($(CONFIG_ARCH_HAVE_VFORK)$(CONFIG_ARCH_HAVE_FORK),) CSRCS += task_fork.c endif diff --git a/sched/task/task_exit.c b/sched/task/task_exit.c index e79c4f812504f..892adfcfda942 100644 --- a/sched/task/task_exit.c +++ b/sched/task/task_exit.c @@ -157,5 +157,19 @@ int nxtask_exit(void) rtcb->lockcount--; + /* Publish anything woken while the TCB was being released. lockcount was + * raised directly rather than through sched_lock(), so the matching + * decrement above does not publish the way sched_unlock() would, and a + * vfork() parent released by nxsched_release_tcb() would be stranded -- + * in g_pendingtasks, or in g_readytorun on SMP. This mirrors what + * sched_unlock() does for each case. + */ + +#ifdef CONFIG_SMP + nxsched_deliver_task(this_cpu(), rtcb->cpu, SWITCH_HIGHER); +#else + nxsched_merge_pending(); +#endif + return ret; } diff --git a/sched/task/task_fork.c b/sched/task/task_fork.c index 5e2b133744264..ca7ff728636aa 100644 --- a/sched/task/task_fork.c +++ b/sched/task/task_fork.c @@ -34,7 +34,9 @@ #include #include +#include #include +#include #include "sched/sched.h" #include "environ/environ.h" @@ -42,9 +44,101 @@ #include "task/task.h" #include "tls/tls.h" -/* fork() requires architecture-specific support as well as waipid(). */ +/**************************************************************************** + * Private Function Prototypes + ****************************************************************************/ -#ifdef CONFIG_ARCH_HAVE_FORK +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) +static void fork_inherit_stack(FAR struct tcb_s *parent, + FAR struct tcb_s *child); +static void fork_inherit_tls(FAR struct tcb_s *child); +static void fork_restore_parent_env(void); +#endif + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) +/**************************************************************************** + * Name: fork_inherit_stack + * + * Description: + * Give the fork() child the parent's stack at the parent's virtual + * address rather than a relocated copy. The child's address environment + * is a duplicate, so the parent's stack is already there -- same contents, + * same address, its own pages -- and nothing needs allocating or copying. + * + * A relocated stack would break plain C: a pointer to a local taken + * before the fork would name the parent's copy, not the child's live + * object. + * + * TCB_FLAG_FREE_STACK is left clear: the stack belongs to the duplicated + * image and is released with it, so up_release_stack() must not free it. + * + * Input Parameters: + * parent - The parent task's TCB + * child - The child task's TCB + * + ****************************************************************************/ + +static void fork_inherit_stack(FAR struct tcb_s *parent, + FAR struct tcb_s *child) +{ + child->stack_alloc_ptr = parent->stack_alloc_ptr; + child->stack_base_ptr = parent->stack_base_ptr; + child->adj_stack_size = parent->adj_stack_size; + child->flags &= ~TCB_FLAG_FREE_STACK; +} + +/**************************************************************************** + * Name: fork_inherit_tls + * + * Description: + * Retarget the thread-local storage the fork() child inherited. + * + * tls_dup_info() cannot be used: it carves a fresh TLS block off the + * stack, which on an inherited stack would carve a second one and shift + * stack_base_ptr away from the parent's. The child's copy is already in + * place, so only the fields naming the task itself need correcting. + * + * The write lands in user memory at an address the parent also occupies, + * so the child's address environment must be current for it -- otherwise + * the parent's own TLS is what gets modified. + * + * Input Parameters: + * child - The child task's TCB + * + * Returned Value: + * Zero (OK) on success; a negated errno value on failure. + * + ****************************************************************************/ + +static void fork_inherit_tls(FAR struct tcb_s *child) +{ + FAR struct tls_info_s *info = (FAR struct tls_info_s *) + child->stack_alloc_ptr; + + info->tl_task = child->group->tg_info; + info->tl_tid = child->pid; +} + +/**************************************************************************** + * Name: fork_restore_parent_env + * + * Description: + * Undo the addrenv_select() that nxtask_setup_fork() made on the child's + * behalf, putting the caller back in its own address environment. The + * environment to go back to does not have to be remembered: the caller is + * the parent, and what was current before was the parent's own. + * + ****************************************************************************/ + +static void fork_restore_parent_env(void) +{ + addrenv_restore(this_task()->addrenv_own); +} +#endif /* CONFIG_ARCH_ADDRENV && CONFIG_ARCH_HAVE_FORK */ /**************************************************************************** * Public Functions @@ -54,37 +148,22 @@ * Name: nxtask_setup_fork * * Description: - * The fork() function has the same effect as posix fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. - * - * This function provides one step in the overall fork() sequence: It - * Allocates and initializes the child task's TCB. The overall sequence - * is: - * - * 1) User code calls fork(). fork() is provided in - * architecture-specific code. - * 2) fork()and calls nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: - * - Allocation of the child task's TCB. - * - Initialization of file descriptors and streams - * - Configuration of environment variables - * - Allocate and initialize the stack - * - Setup the input parameters for the task. - * - Initialization of the TCB (including call to up_initial_state()) - * 4) up_fork() provides any additional operating context. up_fork must: - * - Initialize special values in any CPU registers that were not - * already configured by up_initial_state() - * 5) up_fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * Allocate and initialize the child task's TCB. This is one step in the + * sequence common to vfork() and fork(); see the comment above the + * prototype in include/nuttx/sched.h for the whole sequence and for what + * the two primitives mean. + * + * Exactly two things depend on `vfork': + * + * - the address environment: vfork() joins the parent's, fork() + * duplicates it. + * - the stack: a vfork() child gets its own, which the architecture code + * fills with a relocated copy; a fork() child inherits the parent's + * address (fork_inherit_stack()). * * Input Parameters: - * retaddr - Return address - * argsize - Location to return the argument size + * retaddr - Address at which the child resumes + * vfork - true for vfork(), false for fork() * * Returned Value: * Upon successful completion, nxtask_setup_fork() returns a pointer to @@ -93,7 +172,7 @@ * ****************************************************************************/ -FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) +FAR struct tcb_s *nxtask_setup_fork(start_t retaddr, bool vfork) { FAR struct tcb_s *ptcb = this_task(); FAR struct tcb_s *parent; @@ -160,16 +239,80 @@ FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) } #if defined(CONFIG_ARCH_ADDRENV) - /* Join the parent address environment */ - if (ttype != TCB_FLAG_TTYPE_KERNEL) { - ret = addrenv_join(parent, child); + if (vfork) + { + /* vfork(): join the parent address environment, exactly as + * pthread_create() does. The child shares .data, .bss and the + * heap. + */ + + ret = addrenv_join(parent, child); + } +#ifdef CONFIG_ARCH_HAVE_FORK + else + { + /* POSIX fork(): duplicate the parent's address environment now, + * before anything else is set up. The duplicate holds a copy of + * the parent's contents -- including its stack -- at the parent's + * virtual addresses, which is what lets the child go on to inherit + * the stack address rather than be given a relocated copy. See + * fork_inherit_stack(). + */ + + ret = addrenv_fork(parent, child); + if (ret >= 0) + { + /* Make the child's address environment current for the rest of + * the setup, and for the architecture code that runs after it. + * + * From here on, everything written on the child's behalf + * has to land in the child's image rather than the parent's, + * because + * the two occupy the same virtual addresses: its thread-local + * storage, and -- on architectures that keep the register save + * area on the user stack rather than on a kernel stack -- the + * register context the child is resumed from. Writing those + * under the parent's environment corrupts the parent and + * leaves the child reading whatever the snapshot happened to + * contain. + * + * Reads are unaffected: everything the setup reads from the + * parent -- environ, the argument vector -- is legible at the + * same address in the child, precisely because it is a copy. + * + * nxtask_start_fork() puts the parent's environment back. + */ + + FAR struct addrenv_s *oldenv; + + ret = addrenv_select(child->addrenv_own, &oldenv); + } + } +#else + /* An address environment without ARCH_HAVE_FORK -- a protected build + * over an MMU, for instance. There is an address environment to join, + * but no POSIX fork() to duplicate it for, so the branch above is not + * compiled and `vfork' is always true here. + */ + + DEBUGASSERT(vfork); +#endif + if (ret < 0) { goto errout_with_tcb; } } +#else + /* Without address environments there is only one address space, so + * everything except the stack is shared no matter which primitive was + * called. POSIX fork() cannot be provided at all, and CONFIG_ARCH_HAVE_ + * FORK is not selected, so `vfork' is always true here. + */ + + DEBUGASSERT(vfork); #endif /* Duplicate the parent tasks environment */ @@ -193,12 +336,27 @@ FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) argv = nxsched_get_stackargs(parent); nxtask_setup_name(child, argv[0]); - /* Allocate the stack for the TCB */ + /* Allocate the stack for the TCB, or inherit the parent's */ + +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) + if (!vfork && ttype != TCB_FLAG_TTYPE_KERNEL) + { + /* The child's copy of the parent's stack is already in place, at the + * parent's address, courtesy of the duplication above. + */ + + fork_inherit_stack(parent, child); + ret = OK; + } + else +#endif + { + stack_size = (uintptr_t)ptcb->stack_base_ptr - + (uintptr_t)ptcb->stack_alloc_ptr + ptcb->adj_stack_size; - stack_size = (uintptr_t)ptcb->stack_base_ptr - - (uintptr_t)ptcb->stack_alloc_ptr + ptcb->adj_stack_size; + ret = up_create_stack(child, stack_size, ttype); + } - ret = up_create_stack(child, stack_size, ttype); if (ret < OK) { goto errout_with_tcb; @@ -235,20 +393,35 @@ FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) goto errout_with_tcb; } - /* Setup thread local storage */ - - ret = tls_dup_info(child, parent); - if (ret < OK) + /* Set up thread local storage and the argument vector. + * + * A fork() child that inherited its stack already has both, byte for + * byte, at the addresses the parent has them at -- they came across with + * the rest of the image. Re-creating them would carve fresh frames off a + * stack that already contains them, moving stack_base_ptr away from the + * parent's and undoing the inheritance. Only the TLS fields that name + * the task itself need correcting. + */ + +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) + if (!vfork && ttype != TCB_FLAG_TTYPE_KERNEL) { - goto errout_with_tcb; + fork_inherit_tls(child); } - - /* Setup to pass parameters to the new task */ - - ret = nxtask_setup_stackargs(child, argv[0], &argv[1]); - if (ret < OK) + else +#endif { - goto errout_with_tcb; + ret = tls_dup_info(child, parent); + if (ret < OK) + { + goto errout_with_tcb; + } + + ret = nxtask_setup_stackargs(child, argv[0], &argv[1]); + if (ret < OK) + { + goto errout_with_tcb; + } } /* Now we have enough in place that we can join the group */ @@ -258,6 +431,18 @@ FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) return child; errout_with_tcb: +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) + /* Get back into the parent's address environment before unwinding. If the + * duplication above never happened this is the environment we are already + * in, and addrenv_restore() is then a no-op. + */ + + if (!vfork && ttype != TCB_FLAG_TTYPE_KERNEL) + { + fork_restore_parent_env(); + } +#endif + nxsched_release_tcb((FAR struct tcb_s *)child, ttype); errout: set_errno(-ret); @@ -268,65 +453,129 @@ FAR struct tcb_s *nxtask_setup_fork(start_t retaddr) * Name: nxtask_start_fork * * Description: - * The fork() function has the same effect as fork(), except that the - * behavior is undefined if the process created by fork() either modifies - * any data other than a variable of type pid_t used to store the return - * value from fork(), or returns from the function in which fork() was - * called, or calls any other function before successfully calling _exit() - * or one of the exec family of functions. - * - * This function provides one step in the overall fork() sequence: It - * starts execution of the previously initialized TCB. The overall - * sequence is: - * - * 1) User code calls fork() - * 2) Architecture-specific code provides fork()and calls - * nxtask_setup_fork(). - * 3) nxtask_setup_fork() allocates and configures the child task's TCB. - * This consists of: - * - Allocation of the child task's TCB. - * - Initialization of file descriptors and streams - * - Configuration of environment variables - * - Allocate and initialize the stack - * - Setup the input parameters for the task. - * - Initialization of the TCB (including call to up_initial_state()) - * 4) fork() provides any additional operating context. fork must: - * - Initialize special values in any CPU registers that were not - * already configured by up_initial_state() - * 5) fork() then calls nxtask_start_fork() - * 6) nxtask_start_fork() then executes the child thread. + * The last step of both primitives: finish the child and run it. The + * architecture-specific code calls this once it has built the child's + * register context and stack. + * + * A vfork() additionally suspends the caller until the child calls _exit() + * or one of the exec family of functions. The suspension lives here, in + * the kernel primitive, rather than in a libc waitpid() as it once did. + * Two things follow from that. The parent is released when the child's + * TCB is torn down (see nxtask_resume_vfork()), which for an exec()ing + * child is immediately after exec_swap() has handed the child's pid to the + * program it loaded -- so the parent resumes at exec(), holding a pid that + * names the running program, as POSIX requires. And vfork() no longer + * depends on CONFIG_SCHED_WAITPID. * * Input Parameters: - * child - The tcb_s struct instance that created by - * nxtask_setup_fork() method - * wait_child - whether need to wait until the child is running finished + * child - The tcb_s struct instance created by nxtask_setup_fork() + * vfork - true for vfork(), false for fork() * * Returned Value: - * Upon successful completion, fork() returns 0 to the child process and - * returns the process ID of the child process to the parent process. - * Otherwise, -1 is returned to the parent, no child process is created, - * and errno is set to indicate the error. + * The process ID of the child, or ERROR on failure. * ****************************************************************************/ -pid_t nxtask_start_fork(FAR struct tcb_s *child) +pid_t nxtask_start_fork(FAR struct tcb_s *child, bool vfork) { +#ifdef CONFIG_ARCH_HAVE_VFORK + /* The rendezvous between the suspended parent and the child lives in this + * frame: the parent is blocked here for the whole lifetime of the child, + * so the storage is alive exactly as long as it is needed, and no + * allocation is required on a path that must not fail. + */ + + sem_t rel; + int ret; +#endif pid_t pid; - sinfo("Starting Child TCB=%p\n", child); + sinfo("Starting Child TCB=%p vfork=%d\n", child, vfork); DEBUGASSERT(child); +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) + /* The architecture code has finished writing the child's image, so put the + * parent back in its own address environment. See nxtask_setup_fork(). + */ + + if (!vfork && + (child->flags & TCB_FLAG_TTYPE_MASK) != TCB_FLAG_TTYPE_KERNEL) + { + fork_restore_parent_env(); + } +#endif + /* Get the assigned pid before we start the task */ pid = child->pid; +#ifdef CONFIG_ARCH_HAVE_VFORK + if (vfork) + { + nxsem_init(&rel, 0, 0); + child->vfork_rel = &rel; + } +#endif + /* Activate the task */ nxtask_activate(child); +#ifdef CONFIG_ARCH_HAVE_VFORK + if (vfork) + { + /* Wait for the child to _exit() or exec(). This is not a cancellation + * point and must not be interrupted by a signal: the child may be + * running on our stack, so returning early would corrupt it. + */ + + do + { + ret = nxsem_wait_uninterruptible(&rel); + } + while (ret == -EINTR); + + nxsem_destroy(&rel); + } +#endif + return pid; } +#ifdef CONFIG_ARCH_HAVE_VFORK +/**************************************************************************** + * Name: nxtask_resume_vfork + * + * Description: + * Release the vfork() parent suspended on this child, if there is one. + * + * Called from nxsched_release_tcb(), the last point in the child's life, + * by which time an exec()ing child has already handed its pid to the + * program it loaded. nxtask_abort_fork() reaches it too, so a fork that + * fails after the rendezvous also releases the parent. + * + * Input Parameters: + * child - The TCB being torn down + * + * Returned Value: + * None + * + ****************************************************************************/ + +void nxtask_resume_vfork(FAR struct tcb_s *child) +{ + FAR sem_t *rel = child->vfork_rel; + + if (rel != NULL) + { + /* Clearing the pointer first is what makes this once-only. */ + + child->vfork_rel = NULL; + nxsem_post(rel); + } +} +#endif /* CONFIG_ARCH_HAVE_VFORK */ + /**************************************************************************** * Name: nxtask_abort_fork * @@ -340,6 +589,20 @@ pid_t nxtask_start_fork(FAR struct tcb_s *child) void nxtask_abort_fork(FAR struct tcb_s *child, int errcode) { +#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_ARCH_HAVE_FORK) + /* A child holding an address environment of its own, rather than a + * reference to the caller's, is a fork() child, and nxtask_setup_fork() + * left that environment selected. Get back into the parent's before + * unwinding. See nxtask_setup_fork(). + */ + + if (child->addrenv_own != NULL && + child->addrenv_own != this_task()->addrenv_own) + { + fork_restore_parent_env(); + } +#endif + /* The TCB was added to the active task list by nxtask_setup_scheduler() */ dq_rem((FAR dq_entry_t *)child, list_inactivetasks()); @@ -349,5 +612,3 @@ void nxtask_abort_fork(FAR struct tcb_s *child, int errcode) nxsched_release_tcb(child, child->flags & TCB_FLAG_TTYPE_MASK); set_errno(errcode); } - -#endif /* CONFIG_ARCH_HAVE_FORK */ diff --git a/syscall/syscall.csv b/syscall/syscall.csv index 582deb3907f4f..98c99dff57ad4 100644 --- a/syscall/syscall.csv +++ b/syscall/syscall.csv @@ -202,7 +202,7 @@ "umount2","sys/mount.h","!defined(CONFIG_DISABLE_MOUNTPOINT)","int","FAR const char *","unsigned int" "unlink","unistd.h","!defined(CONFIG_DISABLE_MOUNTPOINT)","int","FAR const char *" "unsetenv","stdlib.h","!defined(CONFIG_DISABLE_ENVIRON)","int","FAR const char *" -"up_fork","nuttx/arch.h","defined(CONFIG_ARCH_HAVE_FORK)","pid_t" +"up_fork","nuttx/arch.h","defined(CONFIG_ARCH_HAVE_VFORK) || defined(CONFIG_ARCH_HAVE_FORK)","pid_t","bool" "utimens","sys/stat.h","","int","FAR const char *","const struct timespec [2]|FAR const struct timespec *" "wait","sys/wait.h","defined(CONFIG_SCHED_WAITPID) && defined(CONFIG_SCHED_HAVE_PARENT)","pid_t","FAR int *" "waitid","sys/wait.h","defined(CONFIG_SCHED_WAITPID) && defined(CONFIG_SCHED_HAVE_PARENT)","int","idtype_t","id_t"," FAR siginfo_t *","int"