Swift Binder client and server - #3
Merged
Merged
Conversation
The serialised payload of a transaction, in Android's layout, since the process on the other end is Android's. Three rules cover it: every write pads to 4 bytes with the padding zeroed, values go out in host byte order because a parcel is shared memory rather than a network format, and Bool/Int8/UTF16 code units are all widened to Int32 on the wire. Int64 is the trap: it takes 8 bytes but is only ever 4-byte aligned, because nothing pads to 8. Reading it as an aligned load is undefined behaviour, so the reader loads unaligned throughout. Null and empty are distinct for both strings and byte arrays - a length of -1 rather than 0 - and AIDL cares about the difference, so it is not collapsed.
Expected bytes are written out rather than produced by the encoder, so a change to the encoding fails the test instead of agreeing with it. Covers the cases that would silently desynchronise a reader: Int64 at 4-byte alignment, the String16 terminator sitting outside the length prefix, surrogate pairs counting as two code units, zeroed padding, and null versus empty.
Binder does not copy transaction payloads into a caller-supplied buffer. It writes them into a mapping established up front and hands back pointers into it, so this is not an optimisation to add later - until it exists the driver has nowhere to put a reply and every transaction fails. Read-only by design: userspace never writes here, outgoing data travels in the write buffer of a BINDER_WRITE_READ ioctl. MAP_NORESERVE because the mapping is large and almost never fully used. Size is one megabyte less two pages, matching libbinder. The subtraction is not decoration - the kernel accounts its own overhead inside the mapping.
The two-page subtraction is the part that looks droppable without consequence, so it is asserted rather than trusted.
Caps how many threads the driver may ask this process to spawn to service incoming transactions; it does not start any. Raising it past what the process can service makes the driver hand work to threads that do not exist.
Opening the device and being able to use it are different things. A bare Binder is an open file descriptor - enough to ask the driver its protocol version and nothing else. Transacting also needs a mapped receive buffer, a matching protocol version, and a thread bound the driver knows about. Keeping them apart means reading the version does not map a megabyte of address space to answer one ioctl, and a value of this type cannot exist without a buffer behind it. A version mismatch is refused at open rather than left to fail confusingly at the first transaction, since continuing would put mismatched structures on the wire.
These two need a real driver, and a Linux host without the binder module loaded has no /dev/binder - the common case for a development machine and for CI, where the suite has been failing rather than skipping. Reporting it as a failure trains people to ignore a red suite.
A BINDER_WRITE_READ ioctl does not carry one command, it carries a byte stream of them: each a 32-bit code followed immediately by its payload, with no padding and no framing. The driver takes as many as it can and reports how many bytes it consumed, which is why binder_write_read has write_consumed separate from write_size. Payload sizes come from the command codes themselves. BC_* and BR_* are full _IOC values with the size of their structure encoded in bits 16-29 by the same macros the kernel uses, so reading it back out means a length can never disagree with the kernel's idea of it - which a hand-written table eventually would.
The driver answers with a stream, not a result. A single reply is routinely preceded by BR_NOOP and BR_TRANSACTION_COMPLETE and may be interleaved with refcount commands and BR_SPAWN_LOOPER that have nothing to do with the transaction being waited on, so a caller has to walk the stream and decide which command ends its wait. An unrecognised command is a decode failure rather than something to step over: its payload length is unknown, so there is no way to find where the next one starts. Guessing would turn one unknown command into a stream of plausible nonsense.
Asserts every command's embedded size against MemoryLayout rather than assuming it, since that equivalence is what lets the stream be walked without a table of sizes. Also pins that BR_TRANSACTION and BR_TRANSACTION_SEC_CTX stay distinct: they share an ioctl number and differ only in the size of the structure they carry, so a collision would decode a security-context transaction as a plain one.
The four-character RPC marker disagreeing means the two sides were built against different binder configurations, not that the parcel is corrupt. Worth reporting distinctly because the callee rejects the entire transaction on it.
Needed to drop the prefix the driver has already consumed.
Deliberately not a flat list of errno values. A transaction fails in ways the file-descriptor layer has no vocabulary for - the peer died, the driver refused it, the reply did not decode - and collapsing those into Errno leaves a caller unable to tell 'retry later' from 'this object is gone forever'.
The driver does not answer a transaction with a reply, it answers with a stream in which the reply may be several commands in or may not have arrived yet, so the exchange is a loop rather than a call. Two things here are not optional. The reply payload lands in the mapped buffer and stays checked out until BC_FREE_BUFFER hands it back - returning without that leaks the mapping one transaction at a time until the process can no longer receive anything. And the descriptor the driver returns is bounds-checked against the mapping before it is followed, because it describes memory and arrives from another process. The request is staged in memory this function owns rather than passed through withUnsafeBufferPointer: the driver reads it through a pointer for the whole exchange, and a closure would put the blocking wait somewhere typed errors cannot propagate.
Four fields in a fixed order - strict-mode policy, work source, a four-character marker, then the descriptor - before any argument. The marker is the part that looks like decoration and is not. The receiver compares it against its own build's value and rejects the whole transaction on a mismatch, logging 'Mixing copies of libbinder?'. Which value is right depends on the process being talked to, not on where this is compiled: a client driving services inside an Android container wants SYST, while libbinder built off-Android sends UNKN and is refused by every one of them.
A handle is meaningful only to the driver and only for the process that received it. The same object has different handles in different processes, so one passed between them as an integer refers to nothing - the driver rewrites it when a flat_binder_object crosses a transaction. A null binder decodes as absence rather than as handle zero, which is the context manager and very much a real object. That is how the service manager says 'no such service'.
Binder needs exactly one fixed point to bootstrap from: looking up a service means transacting with something, and the only handle a fresh process has is zero. Transaction codes are positional - the declaration order of the methods in IServiceManager.aidl, not values anyone chose - which is why AIDL forbids reordering rather than treating them as stable identifiers. addService refuses rather than sending something wrong. Registering requires the driver to translate a flat_binder_object, which it only does when the transaction declares the offset the object sits at, and Parcel does not track offsets yet.
Pins the byte-level shape of what goes on the wire: marker packing, token field order, STRICT_MODE_PENALTY_GATHER being the sign bit, transaction numbering, and a null binder decoding as absent rather than as handle zero.
A connection is shared, not copied: several tasks transact over one device, and the driver is built for exactly that - concurrent ioctls from different threads are the normal case, which is what its thread pool is for. It also has to be a class for the async layer. A synchronous transaction blocks, so the async form hands the work to another thread through an escaping closure, and a non-copyable value cannot be captured by one. That was the blocker. The queue is concurrent rather than serial. Two transactions to different services have no reason to queue behind one another, and a synchronous binder call parks its thread until the peer answers, so serialising would let one slow service stall every other call on the connection.
Not natively asynchronous, and the docs say so: a synchronous binder transaction has no non-blocking form - the driver parks the calling thread until the peer replies - so these move the blocking call to a thread allowed to block and suspend the caller meanwhile. Running it on the cooperative pool would take a core out of circulation for as long as the peer takes to answer. Results travel back as a Result rather than through a throwing continuation, which erases to any Error and would widen every one of these signatures away from BinderError. Cancellation does not propagate, and cannot: once the driver has taken a transaction there is no way to withdraw it, and abandoning the wait would leak the reply buffer since nothing would remain to hand it back with BC_FREE_BUFFER.
Opening a connection needs a driver, so the async surface cannot run here. What can still be checked is that it type-checks from a caller's position: that the async overloads are reachable, that their errors stay typed as BinderError rather than widening to any Error, and that a connection survives a suspension point without tripping sendability. Each of those broke at least once while this was being written. The synchronous surface gets the same treatment, since adding async overloads with identical argument labels risks making the plain calls ambiguous.
The driver cannot find binder objects by scanning: a payload is opaque bytes to it, and an integer that happens to look like an object header would be indistinguishable from one. So a transaction carries an explicit array of offsets and the driver rewrites a handle only at the positions named there. An object written without its offset recorded crosses as plain data and arrives as a number addressing something else in the receiving process.
… objects Also corrects the type's documentation, which claimed nothing here holds a reference count. RemoteProxy now does; this type is the plain value, valid only while the process holds a reference, since the driver may reissue the number for a different object once the count reaches zero.
A handle is not a pointer that can be kept indefinitely. The driver keeps a per-process count for every handle it issues and reclaims the handle once it reaches zero, making the number available for reuse by a different object - so a handle held without a reference is worse than dangling, because it may silently start addressing something else. Two counts, not one: strong keeps the target alive, weak only keeps the handle from being recycled.
Fixes a real bug rather than adding a feature. The driver ties the reference it grants for each object in a transaction to the buffer and drops it in binder_transaction_buffer_release, so freeing first left every handle decoded from a reply already dead - and worse than dead, since the driver is free to reissue that number for a different object. service(named:) was returning exactly such a handle. The acquires and the free now go out in one command stream, acquires first, because the driver applies them in the order it consumes them. transact also now takes a RemoteObject rather than a bare UInt32. The typed reference exists to carry 'this number is a driver handle, valid only in this process', and unwrapping it at the one call site that matters defeated the purpose. Outgoing transactions carry their offsets array, so a parcel containing an object is now translated properly.
Makes the count accountable: one reference held for the object's lifetime, released on deinit, so the driver's count and Swift's object graph agree without anyone having to remember. Getting it wrong is not a leak in the ordinary sense. Releasing once too often lets the driver reclaim the handle and hand the same number to a different object, so a stale proxy silently calls something else. Releasing once too rarely keeps the target process alive. Neither shows up where it was caused. The initialiser adopts rather than acquires. Objects arrive from a reply already holding a reference, so acquiring again would double the count and keep the target alive forever.
A looked-up service is now a RemoteProxy, so it is callable directly with no handle unwrapping and its reference is released when it goes out of scope. The reply's object already holds a reference, so it is adopted rather than acquired again. addService works now that offsets are tracked - the driver can translate the object it carries. It still only registers a reference to an object someone else hosts; serving one needs a local binder and a thread to answer on.
Offsets are tracked now, so nothing reaches it.
The commands need a device, but what a reference count depends on is checkable here and is where the mistakes are: that objects are found only at declared offsets, that data merely resembling an object header is not mistaken for one, that an offsets array from another process cannot send the reader past the end, and that acquires are ordered ahead of the free in the stream the driver consumes.
The serving side stages and fixes up out-of-line buffers exactly as an outgoing transaction does, so a hosted HIDL-shaped service can return a vector.
Pins the 40-byte descriptor with its pointer at offset 8 - written into staged bytes by hand, so a shift would corrupt every buffer silently - and that buffers_size sums 8-aligned lengths rather than raw ones. Catches that a buffer must be recorded as an object offset too, or the driver never processes it, and that reading a received buffer returns the captured payload rather than following the freed pointer.
A binder parcel is host-order shared memory, not a network format, so a request one side writes and the other reads is byte-for-byte what would cross the kernel - the driver only copies those bytes and translates object handles and offsets. That makes a same-process round trip a faithful test of everything except the translation itself: the interface token, the argument order, the reply's exception header, the object, descriptor and buffer encodings, and how they compose. The unit tests cover each of those alone; these cover them together, which is where the integration bugs found while building this library actually lived - a field in the wrong order, an offset not recorded, a reference dropped before it was read. Covers a service-manager lookup and its null-service case, an AIDL method with mixed arguments and its exception path, dispatch through a real hosted-object handler including a rejected caller, a HIDL-shaped out-of-line buffer, an object and a descriptor crossing together, and a truncated reply.
Distinct from interfaceMismatch, which is about the four-character build marker: this is the descriptor string itself disagreeing, meaning a transaction reached an object that does not implement what the caller asked for. The serving-side token check needs to report the two apart.
Every AIDL method has the same envelope around its arguments: write the interface token, transact, read the reply's exception header, then read the results. Skipping or misordering any of it fails the same hard-to-place way - the callee rejects a request whose token is missing, and a client that forgets the exception header reads it as the first result field. These write the envelope once. RemoteProxy.call frames a request and returns a reader positioned past the checked exception header; post is the one-way form; the async call returns a whole parcel because a reader is not Sendable across the suspension, with resultReader() skipping the header on it. On the serving side, Transaction.arguments checks the token and returns a reader at the first argument, and Parcel.reply / Parcel.exception write the reply header a handler would otherwise have to remember.
Drives both halves through the helpers: a request framed by call's envelope decodes through arguments, and a reply built by reply decodes through what call leaves the caller. Covers the wrong-interface rejection being distinct from a marker mismatch, and a whole exchange where the envelope is handled entirely by the helpers on both sides.
The manual envelope is kept alongside, so the shorthand is anchored to what it expands to rather than presented as magic.
There is exactly one context manager per binder context, reached at handle zero by every process without being told where it is. Registering as it makes this process that endpoint, so a lookup service - a servicemanager - is built by becoming the context manager and then serving. It is the one binder role the library could not play: it could call the manager and serve objects, but not be the manager. The security-context form registers a node carrying FLAT_BINDER_FLAG_TXN_SECURITY_CTX, which asks the driver to deliver BR_TRANSACTION_SEC_CTX - the variant the serving loop already decodes - so the two halves meet. The plain form is the widely compatible default. Registration fails if a manager already exists for the context, the usual case on a running system, so it is for a context you own: a fresh binderfs instance or a container whose manager has not started.
remove() does not return the removed object, so the withLock result was unused and warned. DeathRegistry.remove keeps returning it, by contrast, because a cancel needs the record.
Pins that the plain and security-context registrations are distinct ioctls with the right payload sizes - a mismatch the driver rejects - and that the security-context node carries the flag that asks for BR_TRANSACTION_SEC_CTX delivery.
Completes scatter-gather. A nested buffer is one whose pointer lives inside another buffer's data, which is how HIDL lays out nested types: a hidl_vec<hidl_string> is a top-level buffer of hidl_string structs, and each string's characters are a child buffer whose pointer field sits inside that top-level buffer. The driver relocates the child, then writes its new address into the parent at parent_offset. append(buffer:) now returns the object's index for a child to name as its parent - the index into the offsets array, which is what the kernel validates bp->parent against. append(buffer:parent:parentOffset:) sets the has-parent flag and the linkage. The transaction staging is unchanged: it only writes each descriptor's own buffer pointer, so the parent fields it does not touch survive.
Pins the parent flag, the parent index into the offsets array, and the parent_offset - the three fields that route the driver's fixup, where a wrong value either misplaces the child's address or has the driver treat a nested buffer as top-level. Lays out a two-element hidl_vec<hidl_string> as the worked example.
Pins a single toolchain version across all three platforms through a SWIFT_VERSION env var. Linux takes it from the swift:6.3.3 container tag and Android from skip's --version; macOS installs it, because the macos-15 runner's bundled Swift trails the pinned version and would otherwise build against whatever Xcode ships. Replaces the previous per-platform version matrices (6.0.3/6.1.2 on Linux, 6.1/nightly-6.2 on Android), which were testing a spread of older toolchains rather than the one the project targets.
CI caught it: mmap, munmap and sysconf come from the platform C library, and the imports had no Android case - so canImport(Glibc) was false on Android and none of them resolved, failing the build on the one platform this library is actually for. Adds the Android and Bionic modules, inert on every other platform since Glibc or Darwin is matched first.
The unexpectedReturn case carries a DriverReturnProtocol, a C-imported type that auto-conforms to Hashable on newer Swift but not on 6.0.x, so requiring Hashable failed the 6.0.3 build. Nothing hashes a BinderError, so Equatable is enough and works on both.
Same cause as BinderError: Element's command is a C-imported DriverReturnProtocol, not Hashable on 6.0.x. Nothing hashes an element.
swift-actions/setup-swift does not list 6.3.3, though it is a real release - the Linux swift:6.3.3 container builds against it. SwiftyLab's action resolves versions from swift.org dynamically via swiftly, so it installs a release the older action has not catalogued.
The whole matrix now targets 6.3.3, so the tools version is raised to 6.3. This is also what makes the Hashable restore below sound: on 6.3 the C-imported DriverReturnProtocol conforms to Hashable, which it did not on 6.0.x.
Reverses the workaround from earlier in this branch. With Swift 6.3 as the floor, the C-imported DriverReturnProtocol these types embed is Hashable, so the conformance synthesises again - it was only 6.0.x that lacked it.
The kernel uapi headers (<linux/types.h>, <linux/ioctl.h>) ship with glibc and Bionic but not with musl, so a musl build could not compile CBinder - <linux/types.h> was simply not found. The shim now detects a Linux libc that is neither glibc nor Bionic (musl in practice), defines the __uN / __sN / __kernel_* types itself as it already did for Apple, and takes the ioctl macros from <sys/ioctl.h>. The two vendored kernel headers gate their own <linux/*> includes on the same condition. Verified by cross-compiling against Apple's Static Linux SDK for both aarch64-swift-linux-musl and x86_64-swift-linux-musl; the glibc and Bionic paths are unchanged.
Installs Apple's Static Linux SDK - which is musl-based - and builds for aarch64-swift-linux-musl and x86_64-swift-linux-musl. This is what exercises the musl path in CBinder. The SDK checksum is pinned and noted as needing to move with SWIFT_VERSION.
Bumps the armv7 build and test jobs from 6.1.2 / 6.0.3 to 6.3.3, and points at swift-embedded-linux/armhf-debian, the repo the old xtremekforever/swift-armv7 releases now redirect to. All twelve SDK variants the matrix uses exist at 6.3.3.
Adds ubuntu-24.04-arm to the Linux matrix, so the suite builds and runs on native arm64 rather than only x86_64 and cross-compiled targets. arm64 is the architecture Android and most binder deployments actually run on, so running the tests there is worth a runner. The swift:6.3.3 container is multi-arch, so the arm64 image is pulled automatically.
CI got further on Android once the Bionic import was added, then failed here: MAP_FAILED is not imported by Bionic's Swift overlay, and Bionic types mmap as a non-optional pointer where Glibc and Darwin use an implicitly-unwrapped optional, so the guard's optional binding was invalid there too. The result is now typed as an optional - a plain pointer coerces in, an IUO stays bindable - and the failure sentinel is compared as the bit pattern (void*)-1 rather than through the MAP_FAILED macro. Verified on glibc and cross-compiled for aarch64/x86_64 musl; the Android path is the same source.
The macOS build defines ENABLE_MOCKING, which #if'd out the try in these three ioctl wrappers - leaving a do/catch whose block throws nothing, so its catch inferred any Error and would not convert to Errno for BinderError.system. The Linux build never defines ENABLE_MOCKING, so it compiled there. The do/catch now lives inside the real (non-mocking) branch, where the inputOutput call is; the mocking branch is a bare assert, matching how MaxThreads and Version already structure it. Verified the normal and musl builds; the mocking path is macOS-only and could not be compiled on Linux because forcing the define also enables the Socket dependency's Darwin-only mocking.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements a Swift client and server for the Android Binder IPC driver
(
/dev/binder), speaking the same protocol aslibbinderso it can call, andbe called by, Android services — on Android or on desktop Linux driving a
container such as Waydroid. Pure Swift on
swift-systemandSocket; no C++,no
libbinder.The starting point implemented only the
BINDER_VERSIONioctl. This adds thewhole protocol layer on top of it.
What's included
padding,
String16, byte arrays, with object offsets tracked.mmapof the receive buffer, theBINDER_WRITE_READcommandcodec (
BC_*/BR_*streams), synchronous andasynctransactions.service/checkService/services/addService,plus becoming the context manager (handle 0) to be a service registry.
RemoteProxy(strong) and
WeakProxy(weak, with promotion).transactions,
BR_TRANSACTION_SEC_CTXhandling,BINDER_THREAD_EXITon exit.pingliveness.(
hidl_vec<hidl_string>-shaped data).call/post/arguments/replywrite theinterface-token-and-exception-header envelope so a method supplies only its
code and arguments.
Testing
161 tests across 17 suites, covering the wire format, the codec, reference
counting, dispatch decisions, and full request→reply integration round trips.
The two tests that need a device skip cleanly when
/dev/binderis absent, andthe Linux CI job now runs the suite.
Not yet done
This has not run against a real binder driver. The test host has no binder
kernel module, so every ioctl path is reasoned from the kernel source and
libbinderrather than observed; the tests validate everything up to thesyscall boundary, not past it. Remaining gaps: fd arrays (
BINDER_TYPE_FDA),the freeze/node-debug maintenance ioctls, and an interruptible
serve().