dmaax/ladybird
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ladybird/Services/RequestServer/RequestServer.ipc

50 lines
2.8 KiB
Text
Raw Permalink Normal View History

RequestServer: Expose cache sidecars over IPC Add RequestServer messages and LibRequests helpers for storing and retrieving HTTP cache sidecar data. Transfer sidecar payloads as anonymous buffers so large JavaScript bytecode blobs do not have to be serialized into IPC message bodies.
2026-05-03 14:12:09 -03:00
#include <LibCore/AnonymousBuffer.h>
RequestServer: Remove now-unused HTTP related code
2024-09-19 02:54:24 -03:00
#include <LibCore/Proxy.h>
LibHTTP+LibWeb+RequestServer: Handle the Fetch API's cache mode If the cache mode is no-store, we must not interact with the cache at all. If the cache mode is reload, we must not use any cached response. If the cache-mode is only-if-cached or force-cache, we are permitted to respond with stale cache responses. Note that we currently cannot test only-if-cached in test-web. Setting this mode also requires setting the cors mode to same-origin, but our http-test-server infra requires setting the cors mode to cors.
2026-01-20 19:12:36 -03:00
#include <LibHTTP/Cache/CacheMode.h>
LibHTTP+LibWebView+RequestServer: Allow users to set disk cache limits This adds a settings box to about:settings to allow users to limit the disk cache size. This will override the default 5 GiB limit. We do not automatically delete cache data if the new limit is suddenly less than the used disk space; this will happen on the next request. This allows multiple changes to the settings in a row without thrashing the cache. In the future, we can add more toggles, such as disabling the disk cache altogether.
2026-02-05 14:25:14 -03:00
#include <LibHTTP/Cache/DiskCacheSettings.h>
RequestServer: Expose cache sidecars over IPC Add RequestServer messages and LibRequests helpers for storing and retrieving HTTP cache sidecar data. Transfer sidecar payloads as anonymous buffers so large JavaScript bytecode blobs do not have to be serialized into IPC message bodies.
2026-05-03 14:12:09 -03:00
#include <LibHTTP/Cache/Utilities.h>
LibWeb+RequestServer: Attach HTTP cookie headers from RequestServer We currently attach HTTP cookie headers from LibWeb within Fetch. This has the downside that the cookie IPC, and the infrastructure around it, are all synchronous. This blocks the WebContent process entirely while the cookie is being retrieved, for every request on a page. We now attach cookie headers from RequestServer. The state machine in RequestServer::Request allows us to easily do this work asynchronously. We can also skip this work entirely when the response is served from disk cache. Note that we will continue to parse cookies in the WebContent process. If something goes awry during parsing. we limit the damage to that process, instead of the UI or RequestServer. Also note that WebSocket requests still have cookie headers attached attached from LibWeb. This will be handled in a future patch. In the future, we may want to introduce a memory cache for cookies in RequestServer to avoid IPC altogether as able.
2026-02-07 16:06:22 -03:00
#include <LibHTTP/Cookie/IncludeCredentials.h>
LibHTTP+LibWeb+RequestServer: Move Fetch's HTTP header infra to LibHTTP The end goal here is for LibHTTP to be the home of our RFC 9111 (HTTP caching) implementation. We currently have one implementation in LibWeb for our in-memory cache and another in RequestServer for our disk cache. The implementations both largely revolve around interacting with HTTP headers. But in LibWeb, we are using Fetch's header infra, and in RS we are using are home-grown header infra from LibHTTP. So to give these a common denominator, this patch replaces the LibHTTP implementation with Fetch's infra. Our existing LibHTTP implementation was not particularly compliant with any spec, so this at least gives us a standards-based common implementation. This migration also required moving a handful of other Fetch AOs over to LibHTTP. (It turns out these AOs were all from the Fetch/Infra/HTTP folder, so perhaps it makes sense for LibHTTP to be the implementation of that entire set of facilities.)
2025-11-26 16:13:23 -03:00
#include <LibHTTP/Header.h>
LibIPC+LibWeb+LibWebView+Services: Add IPC::TransportHandle Add IPC::TransportHandle as an abstraction for passing IPC transports through .ipc messages. This replaces IPC::File at all sites where a transport (not a generic file) is being transferred between processes. TransportHandle provides from_transport(), clone_from_transport(), and create_transport() methods that encapsulate the fd-to-socket-to-transport conversion in one place. This is preparatory work for Mach port support on macOS -- when that lands, only TransportHandle's internals need to change while all .ipc definitions and call sites remain untouched.
2026-03-11 16:10:08 -03:00
#include <LibIPC/TransportHandle.h>
AK+LibURL: Move AK::URL into a new URL library This URL library ends up being a relatively fundamental base library of the system, as LibCore depends on LibURL. This change has two main benefits: * Moving AK back more towards being an agnostic library that can be used between the kernel and userspace. URL has never really fit that description - and is not used in the kernel. * URL _should_ depend on LibUnicode, as it needs punnycode support. However, it's not really possible to do this inside of AK as it can't depend on any external library. This change brings us a little closer to being able to do that, but unfortunately we aren't there quite yet, as the code generators depend on LibCore.
2024-03-18 00:22:27 -03:00
#include <LibURL/URL.h>
RequestServer: Remove now-unused HTTP related code
2024-09-19 02:54:24 -03:00
#include <RequestServer/CacheLevel.h>
RequestServer: Use correct request map for stale-while-revalidate cookie When we request the HTTP cookie for a SWR request, we were providing the cookie to the standard request corresponding to the SWR request's ID. This had two effects: 1. The SWR request would never finish. 2. If the corresponding standard request happened to be a connect-only request, this would result in a crash as we were expecting it to have gone through the normal fetch process. This was seen on some articles on news.google.com.
2026-04-13 17:53:29 -03:00
#include <RequestServer/RequestType.h>
Everywhere: Use IPC include syntax Remove superfluous includes from IPCCompiler's generated output and add include directives in IPC definitions where appropriate.
2021-07-03 01:40:44 -03:00
Services: Rename ProtocolServer to RequestServer The current ProtocolServer was really only used for requests, and with the recent introduction of the WebSocket service, long-lasting connections with another server are not part of it. To better reflect this, this commit renames it to RequestServer. This commit also changes the existing 'protocol' portal to 'request', the existing 'protocol' user and group to 'request', and most mentions of the 'download' aspect of the request to 'request' when relevant, to make everything consistent across the system. Note that LibProtocol still exists as-is, but the more generic Client class and the more specific Download class have both been renamed to a more accurate RequestClient and Request to match the new names. This commit only change names, not behaviors.
2021-04-23 17:45:52 -03:00
endpoint RequestServer
{
LibWebView+ImageDecoder+RequestServer+WebContent: Add init_transport
2025-01-03 13:19:46 -03:00
init_transport(int peer_pid) => (int peer_pid)
LibIPC+LibWeb+LibWebView+Services: Add IPC::TransportHandle Add IPC::TransportHandle as an abstraction for passing IPC transports through .ipc messages. This replaces IPC::File at all sites where a transport (not a generic file) is being transferred between processes. TransportHandle provides from_transport(), clone_from_transport(), and create_transport() methods that encapsulate the fd-to-socket-to-transport conversion in one place. This is preparatory work for Mach port support on macOS -- when that lands, only TransportHandle's internals need to change while all .ipc definitions and call sites remain untouched.
2026-03-11 16:10:08 -03:00
connect_new_client() => (IPC::TransportHandle handle)
connect_new_clients(size_t count) => (Vector<IPC::TransportHandle> handles)
RequestServer: Add IPC to create a new client and return the sockets We'll need this to implement single-instance RS for the Browser.
2024-04-10 19:02:40 -03:00
LibHTTP+LibWebView+RequestServer: Allow users to set disk cache limits This adds a settings box to about:settings to allow users to limit the disk cache size. This will override the default 5 GiB limit. We do not automatically delete cache data if the new limit is suddenly less than the used disk space; this will happen on the next request. This allows multiple changes to the settings in a row without thrashing the cache. In the future, we can add more toggles, such as disabling the disk cache altogether.
2026-02-05 14:25:14 -03:00
set_disk_cache_settings(HTTP::DiskCacheSettings disk_cache_settings) =|
LibDNS+LibWeb+Ladybird+RequestServer: Let there be DNS over TLS This commit adds our own DNS resolver, with the aim of implementing DoT (and eventually DoH, maybe even DNSSEC etc.)
2024-11-01 19:53:43 -03:00
// use_tls: enable DNS over TLS
LibWebView+RequestSever: Wire up a validate-DNSSEC setting option to RS
2025-05-13 07:34:55 -03:00
set_dns_server(ByteString host_or_address, u16 port, bool use_tls, bool validate_dnssec_locally) =|
LibWebView+RequestServer: Add some UI for DNS settings
2025-04-07 22:56:35 -03:00
set_use_system_dns() =|
LibDNS+LibWeb+Ladybird+RequestServer: Let there be DNS over TLS This commit adds our own DNS resolver, with the aim of implementing DoT (and eventually DoH, maybe even DNSSEC etc.)
2024-11-01 19:53:43 -03:00
Services: Rename ProtocolServer to RequestServer The current ProtocolServer was really only used for requests, and with the recent introduction of the WebSocket service, long-lasting connections with another server are not part of it. To better reflect this, this commit renames it to RequestServer. This commit also changes the existing 'protocol' portal to 'request', the existing 'protocol' user and group to 'request', and most mentions of the 'download' aspect of the request to 'request' when relevant, to make everything consistent across the system. Note that LibProtocol still exists as-is, but the more generic Client class and the more specific Download class have both been renamed to a more accurate RequestClient and Request to match the new names. This commit only change names, not behaviors.
2021-04-23 17:45:52 -03:00
// Test if a specific protocol is supported, e.g "http"
Everywhere: Rename {Deprecated => Byte}String This commit un-deprecates DeprecatedString, and repurposes it as a byte string. As the null state has already been removed, there are no other particularly hairy blockers in repurposing this type as a byte string (what it _really_ is). This commit is auto-generated: $ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \ Meta Ports Ladybird Tests Kernel) $ perl -pie 's/\bDeprecatedString\b/ByteString/g; s/deprecated_string/byte_string/g' $xs $ clang-format --style=file -i \ $(git diff --name-only | grep \.cpp\|\.h) $ gn format $(git ls-files '*.gn' '*.gni')
2023-12-16 11:19:34 -03:00
is_supported_protocol(ByteString protocol) => (bool supported)
Services: Rename ProtocolServer to RequestServer The current ProtocolServer was really only used for requests, and with the recent introduction of the WebSocket service, long-lasting connections with another server are not part of it. To better reflect this, this commit renames it to RequestServer. This commit also changes the existing 'protocol' portal to 'request', the existing 'protocol' user and group to 'request', and most mentions of the 'download' aspect of the request to 'request' when relevant, to make everything consistent across the system. Note that LibProtocol still exists as-is, but the more generic Client class and the more specific Download class have both been renamed to a more accurate RequestClient and Request to match the new names. This commit only change names, not behaviors.
2021-04-23 17:45:52 -03:00
LibWeb+RequestServer: Attach HTTP cookie headers from RequestServer We currently attach HTTP cookie headers from LibWeb within Fetch. This has the downside that the cookie IPC, and the infrastructure around it, are all synchronous. This blocks the WebContent process entirely while the cookie is being retrieved, for every request on a page. We now attach cookie headers from RequestServer. The state machine in RequestServer::Request allows us to easily do this work asynchronously. We can also skip this work entirely when the response is served from disk cache. Note that we will continue to parse cookies in the WebContent process. If something goes awry during parsing. we limit the damage to that process, instead of the UI or RequestServer. Also note that WebSocket requests still have cookie headers attached attached from LibWeb. This will be handled in a future patch. In the future, we may want to introduce a memory cache for cookies in RequestServer to avoid IPC altogether as able.
2026-02-07 16:06:22 -03:00
start_request(u64 request_id, ByteString method, URL::URL url, Vector<HTTP::Header> request_headers, ByteBuffer request_body, HTTP::CacheMode cache_mode, HTTP::Cookie::IncludeCredentials include_credentials, Core::ProxyData proxy_data) =|
LibRequests+RequestServer: Store request IDs as u64 Not super important, but this will match an upcoming ID for stale-while- revalidate requests. It also would techinically be UB if this ID had ever overflowed. Let's make it 64-bit while we are here to also avoid the possibility that it ever will overflow.
2025-12-12 11:56:53 -03:00
stop_request(u64 request_id) => (bool success)
set_certificate(u64 request_id, ByteString certificate, ByteString key) => (bool success)
RequestServer+LibProtocol: Add an 'EnsureConnection' IPC endpoint This will allow LibWeb (and other components) to request a connection to be premade and cached, to make subsequent loads faster.
2021-09-27 17:36:52 -03:00
RequestServer: Do not randomly assign preconnect request IDs Let's avoid the possibility that we randomly generate a request ID that already exists and is in use.
2025-12-11 16:31:12 -03:00
ensure_connection(u64 request_id, URL::URL url, ::RequestServer::CacheLevel cache_level) =|
Everywhere: Merge the WebSocket service into RequestServer This keeps the APIs separate as they are wildly different, a future improvement could be to somehow unify the APIs (if possible). Closes #23080.
2024-03-05 21:50:52 -03:00
RequestServer: Use correct request map for stale-while-revalidate cookie When we request the HTTP cookie for a SWR request, we were providing the cookie to the standard request corresponding to the SWR request's ID. This had two effects: 1. The SWR request would never finish. 2. If the corresponding standard request happened to be a connect-only request, this would result in a crash as we were expecting it to have gone through the normal fetch process. This was seen on some articles on news.google.com.
2026-04-13 17:53:29 -03:00
retrieved_http_cookie(int client_id, u64 request_id, ::RequestServer::RequestType request_type, String cookie) =|
LibWeb+RequestServer: Attach HTTP cookie headers from RequestServer We currently attach HTTP cookie headers from LibWeb within Fetch. This has the downside that the cookie IPC, and the infrastructure around it, are all synchronous. This blocks the WebContent process entirely while the cookie is being retrieved, for every request on a page. We now attach cookie headers from RequestServer. The state machine in RequestServer::Request allows us to easily do this work asynchronously. We can also skip this work entirely when the response is served from disk cache. Note that we will continue to parse cookies in the WebContent process. If something goes awry during parsing. we limit the damage to that process, instead of the UI or RequestServer. Also note that WebSocket requests still have cookie headers attached attached from LibWeb. This will be handled in a future patch. In the future, we may want to introduce a memory cache for cookies in RequestServer to avoid IPC altogether as able.
2026-02-07 16:06:22 -03:00
LibRequests+RequestServer: Add a method to estimate disk cache size This allows estimating the cache size stored on disk since a provided time stamp, and in total.
2025-11-02 15:10:27 -03:00
estimate_cache_size_accessed_since(u64 cache_size_estimation_id, UnixDateTime since) =|
RequestServer: Add a time parameter to the clear cache endpoint This allows removing cache entries last accessed since a provided timestamp.
2025-11-02 18:44:30 -03:00
remove_cache_entries_accessed_since(UnixDateTime since) =|
LibWeb+RequestServer: Send cached bytecode with responses Attach cached JavaScript bytecode sidecars to HTTP response headers so WebContent can materialize classic and module scripts directly from a decoded cache blob on cache hits. Carry the disk cache vary key with the sidecar and reuse it when storing fresh bytecode, avoiding mismatches against the augmented network request headers used to create the cache entry. Keep CORS-filtered module responses intact for status, MIME, and script creation checks. Read bytecode sidecar data only from the internal response, and treat decode or materialization failure as a cache miss that falls back to normal source compilation.
2026-05-03 15:04:35 -03:00
store_cache_associated_data(URL::URL url, ByteString method, Vector<HTTP::Header> request_headers, Optional<u64> vary_key, HTTP::CacheEntryAssociatedData associated_data, Core::AnonymousBuffer data) => (bool stored)
retrieve_cache_associated_data(URL::URL url, ByteString method, Vector<HTTP::Header> request_headers, Optional<u64> vary_key, HTTP::CacheEntryAssociatedData associated_data) => (Optional<Core::AnonymousBuffer> data)
LibHTTP+RequestServer: Add support for synthetic disk-cache entries This exposes the operation through RequestServer.ipc and RequestClient so the client can create the stub before accessing a content-keyed side-data file.
2026-05-11 10:57:16 -03:00
create_synthetic_cache_entry(URL::URL url, ByteString method) => (bool created)
LibWebView+RequestServer: Support clearing the HTTP disk cache This is a bit of a blunt hammer, but this hooks an action to clear the HTTP disk cache into the existing Clear Cache action. Upon invocation, it stops all existing cache entries from making further progress, and then deletes the entire cache index and all cache files. In the future, we will of course want more fine-grained control over cache deletion, e.g. via an about:history page.
2025-10-09 15:24:47 -03:00
Everywhere: Merge the WebSocket service into RequestServer This keeps the APIs separate as they are wildly different, a future improvement could be to somehow unify the APIs (if possible). Closes #23080.
2024-03-05 21:50:52 -03:00
// Websocket Connection API
LibRequests+RequestServer: Store websocket IDs as u64 The bug fixed here is that in RequestServer's ConnectionFromClient, we stored the websocket ID as i32, despite it being i64 everywhere else. Let's make it unsigned while we are here to be consistent with how all other RS-related IDs will be soon.
2025-12-11 13:58:50 -03:00
websocket_connect(u64 websocket_id, URL::URL url, ByteString origin, Vector<ByteString> protocols, Vector<ByteString> extensions, Vector<HTTP::Header> additional_request_headers) =|
websocket_send(u64 websocket_id, bool is_text, ByteBuffer data) =|
LibRequests+RequestServer: Send oversized ws frames via shared memory
2026-05-14 09:39:07 -03:00
websocket_send_shared(u64 websocket_id, bool is_text, Core::AnonymousBuffer data) =|
LibRequests+RequestServer: Store websocket IDs as u64 The bug fixed here is that in RequestServer's ConnectionFromClient, we stored the websocket ID as i32, despite it being i64 everywhere else. Let's make it unsigned while we are here to be consistent with how all other RS-related IDs will be soon.
2025-12-11 13:58:50 -03:00
websocket_close(u64 websocket_id, u16 code, ByteString reason) =|
websocket_set_certificate(u64 request_id, ByteString certificate, ByteString key) => (bool success)
LibProtocol+RequestServer: Add IPC to dump RequestServer connection info This was available on Serenity by sending SIGINFO to RequestServer. Add an IPC so we may see this info while running the browser.
2024-08-03 14:24:56 -03:00
Services: Rename ProtocolServer to RequestServer The current ProtocolServer was really only used for requests, and with the recent introduction of the WebSocket service, long-lasting connections with another server are not part of it. To better reflect this, this commit renames it to RequestServer. This commit also changes the existing 'protocol' portal to 'request', the existing 'protocol' user and group to 'request', and most mentions of the 'download' aspect of the request to 'request' when relevant, to make everything consistent across the system. Note that LibProtocol still exists as-is, but the more generic Client class and the more specific Download class have both been renamed to a more accurate RequestClient and Request to match the new names. This commit only change names, not behaviors.
2021-04-23 17:45:52 -03:00
}
Reference in a new issue Copy permalink