TransportMachPort wrote to its event-loop notification pipe for every
received message. Large bursts could spend significant time in the IO
thread just waking the main loop, even when a previous read notification
was still pending and would already drain the queued messages.
Track whether a read notification is pending while holding the incoming
queue mutex, and only write a new pipe byte when the queue needs a wake.
EOF still schedules a notification, and synchronous waiters continue to
use the condition variable for every arrival.
The Mach transport always sent the serialized IPC bytes as an
out-of-line descriptor. That asks Mach to transfer the payload through a
VM region even when the message is only a small control message.
Use a second data message ID for inline bodies whose full Mach message
fits within 4 KiB. Inline messages carry the unpadded payload length
before the serialized bytes so the receiver can ignore Mach message-size
padding, while larger messages keep using the existing out-of-line
descriptor path.
Mach transport already sends payloads as out-of-line virtual-copy
regions, but the receive path immediately copied each payload into a new
Vector and deallocated the kernel mapping. That made the IPC IO thread
touch every byte before the main thread could decode the message.
Add ReceivedMessageBytes as the raw-message byte storage and let the
Mach transport adopt the OOL region directly. The mapping now lives
until the raw message storage is destroyed, so invalid descriptor paths
and normal queue teardown both release it through the same destructor.
Socket transports keep their existing receive copy path by wrapping
vectors in the same storage type, and the direct raw-message consumers
now decode from its ReadonlyBytes view.
MessageBuffer::transfer_message() handed a MessageDataType to transport
APIs that accepted Vector<u8> const&. That forced the inline-capacity
vector to be materialized as a plain Vector<u8>, and the Mach transport
then copied the same payload again into its pending-send queue.
Make the transport API take MessageDataType by value and pass the
encoded buffer with take_data(). The Mach pending queue now stores the
same type so the payload can move directly to the IO thread. The socket
transport keeps queued messages as owned headers plus moved payloads
instead of copying the payload into an AllocatingMemoryStream, while
preserving the existing chunked send and fd acknowledgement behavior.
Problem: When the browser is closed during startup, we crash inside
PageHost::attach_compositor_ui_client() — on what looks like a normal
null check, but which is actually reading uninitialized memory.
Cause: A sync allocate_compositor_context_id IPC call is issued by the
PageHost constructor before ConnectionFromClient::m_page_host has been
assigned. The sync call runs inside the initializer that’s still
computing m_page_host’s value. If a peer disconnects before responding
(e.g., during shutdown), wait_for_specific_endpoint_message_impl’s
failure path drains any still-queued messages via handle_messages().
One of those is the initial ConnectToCompositor, which dispatches to
m_page_host->attach_compositor_ui_client() — but m_page_host is
uninitialized garbage at that point, so the call segfaults.
Fix: On peer EOF, stop draining queued messages from the failure path.
Re-entering arbitrary handlers from any sync IPC wait is unsafe — since
that wait can be reached from a constructor whose members are still
being initialized. Instead, call shutdown() to close the transport and
invoke die(). That exits processes cleanly via _exit(0) — achieving the
same effect the queued close_server message would’ve had.
Fixes https://github.com/LadybirdBrowser/ladybird/issues/9582
Display-list resource transactions need to carry fonts across an IPC
boundary without making LibWeb know how each typeface stores its bytes.
Add a LibGfx-owned Typeface IPC representation so callers can encode
and decode typefaces directly.
Anonymous-buffer and Core::Resource-backed typefaces serialize through
their retained backing. System typefaces serialize as family and style
data, with macOS system UI typefaces carrying their SystemUIFontKind
from creation so the receiver can rematch them through CoreText.
This is preparatory work required to add IPC between the main and
compositor threads.
IPC currently needs each distinct numeric wrapper to provide its own
serialization specialization. That makes compositor resource ids and
pixel-unit wrappers grow one-off glue as soon as they cross an endpoint.
Teach LibIPC to serialize AK::DistinctNumeric wrappers through their
underlying value type and remove the redundant Web-side specializations
for UniqueNodeID and DevicePixels. Existing compound pixel-unit
serializers now rely on generic wrapper support for their components.
This is preparatory work required to add IPC between the main and
compositor threads.
Treat invalid port descriptors in received Mach messages as an IPC
peer failure instead of a process invariant. The kernel can hand the
receiver a dead or null descriptor when a transferred endpoint dies in
flight, and the receiver must not abort before endpoint dispatch can
shut the connection down.
Close the transport after dropping the malformed message and release
the temporary out-of-line payload mapping before returning.
This commit splits out synchronization primitives from LibThreading into
LibSync. This is because LibThreading depends on LibCore, while LibCore
needs the synchronization primitives from LibThreading. This worked
while they were header only, but when I tried to add an implementation
file it ran into the circular dependency. To abstract away the pthread
implementation using cpp files is necessary so the synchronization
primitives were moved to a separate library.
Empty transport handles can be generated in a few places in Ladybird
sources, notably in WebContentClient::request_worker_agent when
view_for_page_id finds nothing.
If those handles reach encode, a VERIFY is triggered in the broker
process. An page lookup failure should not be fatal to the browser, so
I'll boldly assert it is better to return an error here.
This failure was observed during large runs of origin and IndexedDB
heavy wpt tests in test-web.
A MessagePort can be transferred while it already has local queued
state such as incoming messages drained from its transport,
outgoing messages posted before a transport exists, and a pending
shutdown to apply once the port is enabled.
Serialize and restore that state as part of transfer so it moves with
the port instead of being left behind on the old transport.
Also mark transports that are being transferred so shutdown of the old
endpoint during handoff is not reported as peer EOF. That shutdown is
part of moving the transport to the new owner, not peer disconnected.
Co-Authored-By: Alexander Kalenik <kalenik.aliaksandr@gmail.com>
Remove the 5-second send timeout from mach_msg() to align Mach port
transport behavior with the Unix domain socket transport, which blocks
indefinitely on send.
The timeout also made it impossible to attach a debugger to a child
process on macOS: if you didn't attach within the 5-second window, the
send would time out and the connection would be marked as EOF.
Move handle serialization and deserialization entirely into
TransportSocketWindows so that Windows can share the common Message and
File implementations with other platforms.
Move MachPortServer from LibWebView into LibIPC as MachBootstrapListener
and move the Mach message structs from MachMessageTypes.h into LibIPC.
These types are IPC infrastructure, not UI or platform concerns.
Consolidating them in LibIPC keeps the Mach bootstrap handshake
self-contained in a single library and removes LibWebView's dependency
on LibThreading.
Previously, the bootstrap handshake used a two-state machine
(WaitingForPorts / WaitingForReplyPort) to handle a race: the parent
registering transport ports and the child sending a bootstrap request
could arrive in either order, so whichever came first stored its half
and the second completed the handshake.
Eliminate the race by holding a mutex across spawn() and
register_child_transport(). Since the child cannot send a bootstrap
request before it exists, and the lock isn't released until its
transport is registered, handle_bootstrap_request() is guaranteed to
find the entry. This reduces the pending map to a simple pid-to-ports
lookup and collapses the two-variant state into two straightforward
branches: known child, or on-demand (non-child) caller like WebDriver.
Registering multiple Mach port names with the bootstrap server at
runtime is not how macOS expects it to be used — the bootstrap server
is meant for static services, and the only reason we used it originally
was so child processes could reach back to the UI process.
Remove bootstrap_transport_over_socket(), which had both sides register
dynamic names with the bootstrap server and exchange them over a socket.
Instead, WebDriver and BrowserProcess connections now go through
MachPortServer instances directly. When a non-child process contacts a
MachPortServer, the server creates a port pair on demand (detected via
sysctl ppid check) and returns the local half immediately. This keeps
bootstrap server usage limited to the one original case: child processes
looking up their parent's MachPortServer.
WebDriver Session now runs its own MachPortServer per session.
--webdriver-content-path becomes --webdriver-mach-server-name on macOS.
Spare WebContent launches are skipped when a WebDriver session is active
to avoid bootstrap races.
On macOS, use Mach port messaging instead of Unix domain sockets for
all IPC transport. This makes the transport capable of carrying Mach
port rights as message attachments, which is a prerequisite for sending
IOSurface handles over the main IPC channel (currently sent via a
separate out-of-band path). It also avoids the need for the FD
acknowledgement protocol that TransportSocket requires, since Mach port
right transfers are atomic in the kernel.
Three connection establishment patterns:
- Spawned helper processes (WebContent, RequestServer, etc.) use the
existing MachPortServer: the child sends its task port with a reply
port, and the parent responds with a pre-created port pair.
- Socket-bootstrapped connections (WebDriver, BrowserProcess) exchange
Mach port names over the socket, then drop the socket.
- Pre-created pairs for IPC tests and in-message transport transfer.
Attachment on macOS now wraps a MachPort instead of a file descriptor,
converting between the two via fileport_makeport()/fileport_makefd().
The LibIPC socket transport tests are disabled on macOS since they are
socket-specific.
The #pragma once was placed after the #include directives instead of
immediately after the copyright comment, inconsistent with every other
header file
Encode transfer-data attachments as raw IPC attachments instead of first
rewrapping them as IPC::File values.
This is preparatory refactoring for the upcoming Mach-port transport
introduction on macOS, where attachments should remain transport-native
rather than being normalized through file descriptors.
MultiServer was inherited from SerenityOS where it was used in many
places. Now that BrowserProcess is its only consumer, inline the
connection acceptance logic directly into BrowserProcess and remove
the abstraction.
TransportSocket uses its own pipe-based notification mechanism on the IO
thread, making LocalSocket's built-in Core::Notifier redundant. When the
socket reaches EOF, this notifier is disabled from the IO thread. Since
the QSocketNotifier lives on the main thread, the its destruction is
deferred. If the socket is closed before the deferred destruction runs,
Qt detects the invalid socket on the next poll and prints:
QSocketNotifier: Invalid socket 50 and type 'Read', disabling...
Fix this by disabling the redundant socket-level notifier upfront in the
TransportSocket constructor.
Previously, `create_paired()` returned two full Transport objects, and
callers would immediately call `from_transport()` on the remote side to
extract its underlying fd. This wasted resources: the remote
Transport's IO thread, wakeup pipes, and send queue were initialized
only to be torn down without ever sending or receiving a message.
Now `create_paired()` returns `{Transport, TransportHandle}` — the
remote side is born as a lightweight handle containing just the raw fd,
skipping all unnecessary initialization.
Also replace `release_underlying_transport_for_transfer()` (which
returned a raw int fd) with `release_for_transfer()` (which returns a
TransportHandle directly), hiding the socket implementation detail
from callers including MessagePort.
Replace IPC::File / AutoCloseFileDescriptor / MessageFileType in
the IPC message pipeline with a new IPC::Attachment class. This
wraps a file descriptor transferred alongside IPC messages, and
provides a clean extension point for platform-specific transport
mechanisms (e.g., Mach ports on macOS) that will be introduced later.
Replace clone_from_transport() (which dup()s the FD) with
from_transport() (which releases the FD) in the WebWorkerClient
call site. The UI process never uses the WebWorkerClient connection
after spawning — it only passes the transport to WebContent — so
releasing instead of cloning is safe and simpler.
This removes clone_from_transport() from TransportHandle, and
clone_for_transfer() from TransportSocket/TransportSocketWindows,
as they no longer have any callers.
Now that auxiliary service sockets are sent over IPC rather than passed
as command-line arguments, TransportHandle no longer needs to expose raw
file descriptors or manage close-on-exec flags. Remove fd() and
clear_close_on_exec(), and simplify the connect helpers accordingly.
Add IPC::TransportHandle as an abstraction for passing IPC
transports through .ipc messages. This replaces IPC::File at
all sites where a transport (not a generic file) is being
transferred between processes.
TransportHandle provides from_transport(),
clone_from_transport(), and create_transport() methods that
encapsulate the fd-to-socket-to-transport conversion in one
place. This is preparatory work for Mach port support on
macOS -- when that lands, only TransportHandle's internals
need to change while all .ipc definitions and call sites
remain untouched.
Consolidate the repeated socketpair + adopt + configure pattern from
4 call sites into a single Transport::create_paired() factory method.
This fixes inconsistent error handling and socket configuration across
call sites, and prepares for future mach port support on macOS.
We were allocating vectors on the heap and copying the message header's
and payload's bytes to it before passing them on to
`::enqueue_message()`.
Remove these allocations and just pass `ReadonlyBytes` views into the
message header and payload directly. On my machine, this reduces the
time spent on the send-side queuing path by 13% to 42%, depending on the
message size.
While we're processing received messages, we can end up with unprocessed
bytes after the last message. Instead of copying the data into a new
ByteBuffer, just move the bytes inside the existing buffer and resize
it.
This does mean that as long as there are unprocessed bytes after reading
incoming messages, the buffer does not shrink. But as soon as there's
nothing left, we clear this buffer again.
Change post_message(MessageBuffer) to post_message(MessageBuffer&)
to avoid copying the MessageBuffer onto the stack. MessageBuffer
contains a Vector<u8, 1024> with a 1024-byte inline buffer, so
passing by value was adding over 1 KiB to the stack frame of
handle_messages().
This reduces the handle_messages() stack frame from 1328 bytes to
224 bytes, which matters because handle_messages() sits near the
base of the call stack when GC runs its conservative stack scan
in response to IPC requests.
This commit stops using deprecated WSA functions. While the ANSI
versions are most likely not going anywhere, Windows is natively UTF-16
so it has to convert to ANSI internally. All the ANSI functions in
Winsock are marked as deprecated. The macro suppressing the warnings is
no longer defined.
Previously, IPC messages were decoded on the main thread:
1. I/O thread received raw bytes and file descriptors
2. I/O thread stored them in a queue and notified main thread
3. Main thread decoded bytes into Message objects
4. Main thread processed the messages
Now, decoding happens on the I/O thread:
1. I/O thread receives raw bytes and file descriptors
2. I/O thread decodes them using a configurable MessageDecoder
3. I/O thread calls MessageHandler which stores decoded messages
4. I/O thread signals condition variable (for sync waiters)
5. I/O thread wakes main event loop via deferred_invoke()
6. Main thread processes already-decoded messages
This is achieved by:
- Adding MessageDecoder and MessageHandler callbacks to TransportSocket
- Connection template sets up the decoder (tries both endpoints)
- ConnectionBase::initialize_messaging() sets up the handler
- Storing a WeakEventLoopReference to wake the main thread
- Using mutex + condition variable for thread-safe queue access
- Sync message waiting now uses the CV directly instead of polling
The raw message API (read_as_many_messages_as_possible_without_blocking)
is preserved for MessagePort which uses its own decoding logic.
This architecture prepares for future multi-thread dispatch where
different message types could be routed to different handler threads
(e.g., scrolling messages to a dedicated scroll thread).
Previously, when an IPC message failed to parse, we only logged
"Failed to parse IPC message" followed by a hex dump, making it
difficult to diagnose the actual cause.
Now we log the specific error from each endpoint's decode attempt,
making it much easier to identify issues like size limit violations
or invalid field values.
AnonymousBuffer is backed by shared memory, not heap allocation.
The MAX_DECODED_SIZE limit in decode_size() is meant to prevent OOM
from malicious peers claiming huge sizes that would cause heap
allocations, but this doesn't apply to AnonymousBuffer since the
memory is already allocated by the sender.
This fixes decoding of large animated images (e.g. 300 frames at
240x240) where the total bitmap data exceeds 64 MiB.
