It's now possible to block until another thread in the same process has
exited. We can also retrieve its exit value, which is whatever value it
passed to pthread_exit(). :^)
While executing in the kernel, a thread can acquire various resources
that need cleanup, such as locks and references to RefCounted objects.
This cleanup normally happens on the exit path, such as in destructors
for various RAII guards. But we weren't calling those exit paths when
killing threads that have been executing in the kernel, such as threads
blocked on reading or sleeping, thus causing leaks.
This commit changes how killing threads works. Now, instead of killing
a thread directly, one is supposed to call thread->set_should_die(),
which will unblock it and make it unwind the stack if it is blocked
in the kernel. Then, just before returning to the userspace, the thread
will automatically die.
This patch adds pthread_create() and pthread_exit(), which currently
simply wrap our existing create_thread() and exit_thread() syscalls.
LibThread is also ported to using LibPthread.
Some syscalls have to pass parameters through a struct, since we can
only fit 3 parameters with our calling convention.
This patch makes use of C++ structured binding to clean up the places
where we expand those parameters structs into local variables.
POSIX's openat() is very similar to open(), except you also provide a
file descriptor referring to a directory from which relative paths
should be resolved.
Passing it the magical fd number AT_FDCWD means "resolve from current
directory" (which is indeed also what open() normally does.)
This fixes libarchive's bsdtar, since it was trying to do something
extremely wrong in the absence of openat() support. The issue has
recently been fixed upstream in libarchive:
https://github.com/libarchive/libarchive/issues/1239
However, we should have openat() support anyway, so I went ahead and
implemented it. :^)
Fixes#748.
Don't wait for someone to wait() on a dead process before releasing its
TTY object. This fixes the child process death detection used by the
Terminal application, which relies on getting an EOF on the master PTY
in order to know it's time to wait() on the child process. :^)
Instead of the big ugly switch statement, build a lookup table using
the syscall enumeration macro.
This greatly simplifies the syscall implementation. :^)
Scheduling priority is now set at the thread level instead of at the
process level.
This is a step towards allowing processes to set different priorities
for threads. There's no userspace API for that yet, since only the main
thread's priority is affected by sched_setparam().
This patch changes the parameter to Region::map() to be a PageDirectory
since that matches how we think about the memory model:
Regions are views onto VMObjects, and are mapped into PageDirectories.
Each Process has a PageDirectory. The kernel also has a PageDirectory.
Add the ability to both pass arguments to scripts with shebangs
(./script argument1 argument2) and to specify them in the shebang line
(#!/usr/local/bin/bash -x -e)
Fixes#585
We now return EISDIR whenever a program attempts to call sys$read
on a directory. Previously, attempting to read a directory could
either return junk data or, in the case of /proc/, cause a kernel
panic.
Thread::make_userspace_stack_for_main_thread is only ever called from
Process::do_exec, after all the fun ELF loading and TSS setup has
occured.
The calculations in there that check if the combined argv + envp
size will exceed the default stack size are not used in the rest of
the stack setup. So, it should be safe to move this to the beginning
of do_exec and bail early with -E2BIG, just like the man pages say.
Additionally, advertise this limit in limits.h to be a good POSIX.1
citizen. :)
ELFLoader::layout() had a "failed" variable that was never set. This
patch checks the return value of each hook (alloc/map section and tls)
and fails the load if they return null.
I also needed to patch Process so that the alloc_section_hook and
map_section_hook actually return nullptr when allocating a region fails.
Fixes#664 :)
The way it gets the entropy and blasts it to the buffer is pretty
ugly IMHO, but it does work for now. (It should be replaced, by
not truncating a u32.)
It implements an (unused for now) flags argument, like Linux but
instead of OpenBSD's. This is in case we want to distinguish
between entropy sources or any other reason and have to implement
a new syscall later. Of course, learn from Linux's struggles with
entropy sourcing too.
Make sure we don't move accepted sockets to the Completed setup state
until we've actually constructed a FileDescription for them.
This is important, since this state transition will trigger connect()
to unblock on the client side, and the client may try writing to the
socket right away.
This makes DNS lookups way more reliable since we don't just fail to
write() right after connect()ing to LookupServer sometimes. :^)
Added the exception_code field to RegisterDump, removing the need
for RegisterDumpWithExceptionCode. To accomplish this, I had to
push a dummy exception code during some interrupt entries to properly
pad out the RegisterDump. Note that we also needed to change some code
in sys$sigreturn to deal with the new RegisterDump layout.
When splitting an Region that's already the result of an earlier split,
we have to take the Region's offset-in-VMObject into account since it
may be non-zero.
We were always returning the full VM range of the partially-unmapped
Region to the range allocator. This caused us to re-use those addresses
for subsequent VM allocations.
This patch also skips creating a new VMObject in partial munmap().
Instead we just make split regions that point into the same VMObject.
This fixes the mysterious GCC ICE on large C++ programs.
This simplifies the ownership model and makes Region easier to reason
about. Userspace Regions are now primarily kept by Process::m_regions.
Kernel Regions are kept in various OwnPtr<Regions>'s.
Regions now only ever get unmapped when they are destroyed.
We now no longer hardcode the sigreturn syscall in
the signal trampoline. Because of the way inline asm inputs
work, I've had to enclose the trampoline in the function
signal_trampoline_dummy.
This patch makes it possible to *run* text files that start with the
characters "#!" followed by an interpreter.
I've tested this with both the Serenity built-in shell and the Bash
shell, and it works as expected. :^)
The fchdir() function is equivalent to chdir() except that the
directory that is to be the new current working directory is
specified by a file descriptor.
Due to the changes in signal handling m_kernel_stack_for_signal_handler_region
and m_signal_stack_user_region are no longer necessary, and so, have been
removed. I've also removed the similarly reduntant m_tss_to_resume_kernel.
This patch adds support for TLS according to the x86 System V ABI.
Each thread gets a thread-specific memory region, and the GS segment
register always points _to a pointer_ to the thread-specific memory.
In other words, to access thread-local variables, userspace programs
start by dereferencing the pointer at [gs:0].
The Process keeps a master copy of the TLS segment that new threads
should use, and when a new thread is created, they get a copy of it.
It's basically whatever the PT_TLS program header in the ELF says.
This commit drastically changes how signals are handled.
In the case that an unblocked thread is signaled it works much
in the same way as previously. However, when a blocking syscall
is interrupted, we set up the signal trampoline on the user
stack, complete the blocking syscall, return down the kernel
stack and then jump to the handler. This means that from the
kernel stack's perspective, we only ever get one system call deep.
The signal trampoline has also been changed in order to properly
store the return value from system calls. This is necessary due
to the new way we exit from signaled system calls.
You can now munmap() a part of a region. The kernel will then create
one or two new regions around the "hole" and re-map them using the same
physical pages as before.
This goes towards fixing #175, but not all the way since we don't yet
do munmap() across multiple mappings.
We were doing this for the initial kernel-spawned userspace process(es)
to work around instability in the page fault handler. Now that the page
fault handler is more robust, we can stop worrying about this.
Specifically, the page fault handler was previous not able to handle
getting a page fault in anything but the currently executing task's
page directory.
It is now possible to unmount file systems from the VFS via `umount`.
It works via looking up the `fsid` of the filesystem from the `Inode`'s
metatdata so I'm not sure how fragile it is. It seems to work for now
though as something to get us going.
We were forced to do this because the page fault code would fall apart
when trying to generate a backtrace for a non-current thread.
This issue has been fixed for a while now, so let's go back to lazily
loading executable pages which should make everything a little better.
This patch adds the mprotect() syscall to allow changing the protection
flags for memory regions. We don't do any region splitting/merging yet,
so this only works on whole mmap() regions.
Added a "crash -r" flag to verify that we crash when you attempt to
write to read-only memory. :^)
This is more logical and allows us to solve the problem of
non-blocking TCP sockets getting stuck in SocketRole::None.
The only complication is that a single LocalSocket may be shared
between two file descriptions (on the connect and accept sides),
and should have two different roles depending from which side
you look at it. To deal with it, Socket::role() is made a
virtual method that accepts a file description, and LocalSocket
internally tracks which FileDescription is the which one and
returns a correct role.
After a fork, the parent and the child are supposed to share
the same file description. For example, modifying the current
offset of a file description is visible in both of them.
We were only doing this in Process::deallocate_region(), which meant
that kernel-only Regions never gave back their VM.
With this patch, we can start reusing freed-up address space! :^)
This is not perfect as it uses a lot of VM, but since the buffers are
supposed to be temporary it's not super terrible.
This could be improved by giving back the unused VM to the kernel's
RangeAllocator after finishing the buffer building.
InodeVMObject is a VMObject with an underlying Inode in the filesystem.
AnonymousVMObject has no Inode.
I'm happy that InodeVMObject::inode() can now return Inode& instead of
VMObject::inode() return Inode*. :^)
The VMObject name was always either the owning region's name, or the
absolute path of the underlying inode.
We can reconstitute this information if wanted, no need to keep copies
of these strings around.
This allows the page fault code to find the owning PageDirectory and
corresponding process for faulting regions.
The mapping is implemented as a global hash map right now, which is
definitely not optimal. We can come up with something better when it
becomes necessary.
In the userspace, this mimics the Linux pipe2() syscall;
in the kernel, the Process::sys$pipe() now always accepts
a flags argument, the no-argument pipe() syscall is now a
userspace wrapper over pipe2().
In the future, we should allow mounting any block device. At the moment
there is too much filesystem code that depends on the underlying device
being a DiskDevice.
- You must now have superuser privileges to use mount().
- We now verify that the mount point is a valid path first, before
trying to find a filesystem on the specified device.
- Convert some dbgprintf() to dbg().
It is now possible to mount ext2 `DiskDevice` devices under Serenity on
any folder in the root filesystem. Currently any user can do this with
any permissions. There's a fair amount of assumptions made here too,
that might not be too good, but can be worked on in the future. This is
a good start to allow more dynamic operation under the OS itself.
It is also currently impossible to unmount and such, and devices will
fail to mount in Linux as the FS 'needs to be cleaned'. I'll work on
getting `umount` done ASAP to rectify this (as well as working on less
assumption-making in the mount syscall. We don't want to just be able
to mount DiskDevices!). This could probably be fixed with some `-t`
flag or something similar.
Processes can now have an icon assigned, which is essentially a 16x16 RGBA32
bitmap exposed as a shared buffer ID.
You set the icon ID by calling set_process_icon(int) and the icon ID will be
exposed through /proc/all.
To make this work, I added a mechanism for making shared buffers globally
accessible. For safety reasons, each app seals the icon buffer before making
it global.
Right now the first call to GWindow::set_icon() is what determines the
process icon. We'll probably change this in the future. :^)
This is expensive because we have to page in the entire executable for every
process up front for this to work. This is due to the page fault code not
being strong enough to run while another process is active.
Note that we already had userspace symbols in *crash* stacks. This patch
adds them generally, so they show up in /proc, Process Manager, etc.
There's room for improvement here, but the debugging benefits way overshadow
the performance penalty right now. :^)
This makes assertion failures generate backtraces again. Sorry to everyone
who suffered from the lack of backtraces lately. :^)
We share code with the /proc/PID/stack implementation. You can now get the
current backtrace for a Thread via Thread::backtrace(), and all the traces
for a Process via Process::backtrace().
The syscall is quite simple:
int watch_file(const char* path, int path_length);
It returns a file descriptor referring to a "InodeWatcher" object in the
kernel. It becomes readable whenever something changes about the inode.
Currently this is implemented by hooking the "metadata dirty bit" in
Inode which isn't perfect, but it's a start. :^)
The "stddbg" stream was a cute idea but we never ended up using it in
practice, so let's simplify this and implement userspace dbgprintf() on top
of a simple dbgputch() syscall instead.
This makes debugging LibC startup a little bit easier. :^)
This is very simple but already very useful. Now you're able to call to
dump_backtrace() from anywhere userspace to get a nice symbolicated
backtrace in the debugger output. :^)
And use this to return EINTR in various places; some of which we were
not handling properly before.
This might expose a few bugs in userspace, but should be more compatible
with other POSIX systems, and is certainly a little cleaner.
It's a very bad idea to increment the refcount on behalf of another
process. That process may (for either benign or evil reasons) not
reference the SharedBuffer, and then we'll be stuck with loads of
SharedBuffers until we OOM.
Instead, increment the refcount when the buffer is mapped. That way, a
buffer is only kept if *someone* has explicitly requested it via
get_shared_buffer.
Fixes#341
Generate a special page containing the "return from signal" trampoline code
on startup and then route signalled threads to it. This avoids a page
allocation in every process that ever receives a signal.
Region now has is_user_accessible(), which informs the memory manager how
to map these pages. Previously, we were just passing a "bool user_allowed"
to various functions and I'm not at all sure that any of that was correct.
All the Region constructors are now hidden, and you must go through one of
these helpers to construct a region:
- Region::create_user_accessible(...)
- Region::create_kernel_only(...)
That ensures that we don't accidentally create a Region without specifying
user accessibility. :^)
"Blocking" is not terribly informative, but now that everything is
ported over, we can force the blocker to provide us with a reason.
This does mean that to_string(State) needed to become a member, but
that's OK.
Rolling with the theme of adding a dialog to shutdown the machine, it is
probably nice to have a way to reboot the machine without performing a full
system powerdown.
A reboot program has been added to `/bin/` as well as a corresponding
`syscall` (SC_reboot). This syscall works by attempting to pulse the 8042
keyboard controller. Note that this is NOT supported on new machines, and
should only be a fallback until we have proper ACPI support.
The implementation causes a triple fault in QEMU, which then restarts the
system. The filesystems are locked and synchronized before this occurs,
so there shouldn't be any corruption etctera.
This allows us to seal a buffer *before* anyone else has access to it
(well, ok, the creating process still does, but you can't win them all).
It also means that a SharedBuffer can be shared with multiple clients:
all you need is to have access to it to share it on again.
Exec doesn't leave through the syscall handler, so it didn't unlock the
big_lock. This means that reentering can lock it again, and then another
thread could endlessly yield waiting to acquire the lock (futilely).
This fixes AudioServer using 100% CPU.
We were locking the list of references, and then destroying the
reference, which made things go a little crazy.
It's more straightforward to just remove the per-reference lock: the
syscalls all have to lock the full list anyway, so let's just do that
and avoid the hassle.
While I'm at it, also move the SharedBuffer code out to its own file as it's
getting a little long and unwieldly, and Process.cpp is already huge.
Rather than limiting it to two shared processes, store a Vector of
references, so we can add more if we want. Makes the code a little
more generic.
No actual change to the syscall interface yet, so nothing takes
advantage of this yet.
This makes waitpid() return when a child process is stopped via a signal.
Use this in Shell to catch stopped children and return control to the
command line. :^)
Fixes#298.
This is obviously more readable. If we ever run into a situation where
ref count churn is actually causing trouble in the future, we can deal with
it then. For now, let's keep it simple. :^)
Instead of computing the path length inside the syscall handler, let the
caller do that work. This allows us to implement to new variants of open()
and creat(), called open_with_path_length() and creat_with_path_length().
These are suitable for use with e.g StringView.
String&& is just not very practical. Also return const String& when the
returned string is a member variable. The call site is free to make a copy
if he wants, but otherwise we can avoid the retain count churn.
After reading a bunch of POSIX specs, I've learned that a file descriptor
is the number that refers to a file description, not the description itself.
So this patch renames FileDescriptor to FileDescription, and Process now has
FileDescription* file_description(int fd).
Define the multiboot info struct properly so we don't have to grab at byte
offsets in the memory access checker code. Also print kernel command line
in init().
We definitely need to replace m_executable before clearing interrupts, since
otherwise we might call ~Custody() which would make it assert in locking.
Also avoid calling FileDescriptor::metadata() repeatedly and just cache the
result from the first call.
I also added a comment at the point where we've decided to commit to the new
executable and follow through with the swap.
The current working directory is now stored as a custody. Likewise for a
process executable file. This unbreaks /proc/PID/fd which has not been
working since we made the filesystem bigger.
This still needs a bunch of work, for instance when renaming or removing
a file somewhere, we have to update the relevant custody links.
Right now, we allow anything inside a user to raise or lower any other process's
priority. This feels simple enough to me. Linux disallows raising, but
that's annoying in practice.
By moving the sending (and receiving) thread into the BlockedSignal state,
we ensure that the thread doesn't continue executing until the signal
has been dispatched.
This is in preparation for eventually using it in userspace.
LinearAddress.h has not been moved for the time being (as it seems to be
only used by a very small part of the code).
Since we transition to a new PageDirectory on exec(), we need a matching
RangeAllocator to go with the new directory. Instead of juggling this in
Process and MemoryManager, simply attach the RangeAllocator to the
PageDirectory instead.
Fixes#61.
Passing this flag to recv() temporarily puts the file descriptor into
non-blocking mode.
Also implement LocalSocket::recv() as a simple forwarding to read().
select essentially has 3 modes (which is presumably why we're finding it
so hard to get this right in a reliable way :)).
1. NULL timeout -- no timeout on blocking
2. non-NULL timeout that is not zero'd -- timeout on blocking
3. non-NULL but zero timeout -- no blocking at all (immediate poll)
For cases 1 and 2, we want to block the thread. We have a timeout set
only for case 2, though.
Case 3 should not block the thread, and does not have a timeout set.
The scheduler expects m_select_timeout to act as a deadline. That is, it
should contain the time that a task should wake at -- but we were
directly copying the time from userspace, which meant that it always
returned virtually immediately.
At the same time, fix CEventLoop to not rely on the broken select behavior
by subtracting the current time from the time of the nearest timer.
Based on the description I read, this syscall doesn't seem completely
reasonable, but let's at least return a number that is likely to change
between invocations in case somebody depends on that happening.
These functions were doing exactly the same thing for range allocation, so
share that code in an allocate_range() helper.
Region allocation will now also fail if range allocation fails, which means
that mmap() can actually fail without falling apart. Exciting times!
This replaces the previous virtual address allocator which was basically
just "m_next_address += size;"
With this in place, virtual addresses can get reused, which cuts down on
the number of page tables created. When we implement ASLR some day, we'll
probably have to do page table deallocation, but for now page tables are
only deallocated once the process dies.
Stash away the ELFLoader used to load an executable in Process so we can use
it for symbolicating userspace addresses later on. This will make debugging
userspace programs a lot nicer. :^)
This patch moves away from using kmalloc memory for thread kernel stacks.
This reduces pressure on kmalloc (16 KB per thread adds up fast) and
prevents kernel stack overflow from scribbling all over random unrelated
kernel memory.
Make the Socket functions take a FileDescriptor& rather than a socket role
throughout the code. Also change threads to block on a FileDescriptor,
rather than either an fd index or a Socket.
We were only destroying the main thread when a process died, leaving any
secondary threads around. They couldn't run, but because they were still
in the global thread list, strange things could happen since they had some
now-stale pointers to their old process.
Calling systrace(pid) gives you a file descriptor with a stream of the
syscalls made by a peer process. The process must be owned by the same
UID who calls systrace(). :^)
I was originally implementing signals by looking at some man page about
sigaction() to see how it works. It seems like the restorer thingy is
system-specific and not required by POSIX, so let's get rid of it.
If connect() is called on a non-blocking socket, it will "fail" immediately
with -EINPROGRESS. After that, you select() on the socket and wait for it to
become writable.
We can't have multiple threads in the same process running in the kernel
at the same time, so let's have a per-process lock that threads have to
acquire on syscall entry/exit (and yield while blocked.)
This currently only works for "normal" processes created by fork().
It does not work for create_user_process() processes spawned by the
kernel, as those are a bit special during construction.
It's basically a userspace port of the kernel's Lock class.
Added gettid() and donate() syscalls to support the timeslice donation
feature we already enjoyed in the kernel.
This introduces a tiny amount of timer drift which I will have to fix
somehow eventually, but it's a huge improvement in timing consistency
as we no longer suddenly jump from e.g 10:45:49.123 to 10:45:50.000.
The scheduler now operates on threads, rather than on processes.
Each process has a main thread, and can have any number of additional
threads. The process exits when the main thread exits.
This patch doesn't actually spawn any additional threads, it merely
does all the plumbing needed to make it possible. :^)
Now that everything is nice and mature, the WindowServer can just use the
client PID it receives in the Greeting message, and we can get rid of this
hacky ioctl. :^)
This is accomplished using a new Alarm class and a BlockedSnoozing state.
Basically, you call Process::snooze_until(some_alarm) and then the scheduler
won't wake up the process until some_alarm.is_ringing() returns true.
We had a bug where an accepted socket would appear to be EOF/disconnected
on the accepting side until the connecting side had called attach_fd().
Fix this by adding a new SocketRole::Connecting state.
Only the receive timeout is hooked up yet. You can change the timeout by
calling setsockopt(..., SOL_SOCKET, SO_RCVTIMEO, ...).
Use this mechanism to make /bin/ping report timeouts.
Finally fixed the weird flaky crashing when resizing Terminal windows.
It was because we were dispatching a signal to "current" from the scheduler.
Yet another thing I dislike about even having a "current" process while
we're in the scheduler. Not sure yet how to fix this.
Let the signal handler's kernel stack be a kmalloc() allocation for now.
Once we can do allocation of consecutive physical pages in the supervisor
memory region, we can use that for all types of kernel stacks.
It's now possible to create symbolic links! :^)
This exposed an issue in Ext2FS where we'd write uninitialized data past
the end of an inode's content. Fix this by zeroing out the tail end of
the last block in a file.
The idea here is to combine a potential syscall error code with an arbitrary
type in the case of success. I feel like this will end up much less error
prone than returning some arbitrary type that kinda sorta has bool semantics
(but sometimes not really) and passing the error through an out-param.
This patch only converts a few syscalls to using it. More to come.
It was very confusing that you had to open a FileDescriptor in order to stat
a file. This patch gives VFS a separate stat() function and uses it to
implement the stat() and lstat() syscalls.
I set it up so that TIOCSWINSZ on a master PTY gets forwarded to the slave.
This feels intuitively right. Terminal can then use that to inform the shell
or whoever is inside the slave that the window size has changed.
TIOCSWINSZ also triggers the generation of a SIGWINCH signal. :^)
Turns out FD_CLOEXEC and O_CLOEXEC are different values. Silly mistake.
I noticed that Terminal's shell process still had the Terminal's window
server connection open, albeit in a broken state.
When the kernel performs a successful exec(), whatever was on the kernel
stack for that process before goes away. For this reason, we need to make
sure we don't have any stack objects holding onto kmalloc memory.
The mismatch between the two was causing some trouble if you'd mmap e.g 1KB
and then try to munmap() it. The kernel would whine that it couldn't find
any such mapping (because mmap() actually rounded the 1KB to a 4KB page.)
This is a monster patch that required changing a whole bunch of things.
There are performance and stability issues all over the place, but it works.
Pretty cool, I have to admit :^)
Currently you can only mmap the entire framebuffer.
Using this when starting up the WindowServer gets us yet another step
closer towards it moving into userspace. :^)
We were reading one client message per client per event loop iteration.
That was not very snappy. Make the sockets non-blocking and read() until
there are no messages left.
It would be even better to make as few calls to read() as possible to
reduce context switching, but this is already a huge improvement.
This is really cool! :^)
Apps currently refuse to start if the WindowServer isn't listening on the
socket in /wsportal. This makes sense, but I guess it would also be nice
to have some sort of "wait for server on startup" mode.
This has performance issues, and I'll work on those, but this stuff seems
to actually work and I'm very happy with that.
Since we know who's holding the lock, and we're gonna have to yield anyway,
we can just ask the scheduler to donate any remaining ticks to that process.
Instead of processes themselves getting scheduled to finish dying,
let's have a Finalizer process that wakes up whenever someone is dying.
This way we can do all kinds of lock-taking in process cleanup without
risking reentering the scheduler.
- Don't cli() in Process::do_exec() unless current is execing.
Eventually this should go away once the scheduler is less retarded
in the face of interrupts.
- Improved memory access validation for ring0 processes.
We now look at the kernel ELF header to determine if an access
is appropriate. :^) It's very hackish but also kinda neat.
- Have Process::die() put the process into a new "Dying" state where
it can still get scheduled but no signals will be dispatched.
This way we can keep executing in die() but won't get our EIP
hijacked by signal dispatch. The main problem here was that die()
wanted to take various locks.
Also add assertion in Lock that the scheduler isn't currently active.
I've been seeing occasional fuckups that I suspect might be someone called
by the scheduler trying to take a busy lock.
Instead of cowboy-calling the VESA BIOS in the bootloader, find the emulator
VGA adapter by scanning the PCI bus. Then set up the desired video mode by
sending device commands.
The current strategy is simply to nuke all physical pages and force
reload them from disk. This is obviously not optimal and should eventually
be optimized. It should be fairly straightforward.
Font now uses the same in-memory format as the font files we have on disk.
This allows us to simply mmap() the font files and not use any additional
memory for them. Very cool! :^)
Hacking on this exposed a bug in file-backed VMObjects where the first client
to instantiate a VMObject for a specific inode also got to decide its size.
Since file-backed VMObjects always have the same size as the underlying file,
this made no sense, so I removed the ability to even set a size in that case.
Also use an enum for the rather-confusing return value in dispatch_signal().
I will go through the rest of the signals and set them up with the
appropriate default dispositions at some other point.
Now the filesystem is generated on-the-fly instead of manually adding and
removing inodes as processes spawn and die.
The code is convoluted and bloated as I wrote it while sleepless. However,
it's still vastly better than the old ProcFS, so I'm committing it.
I also added /proc/PID/fd/N symlinks for each of a process's open fd's.
GObjects can now register a timer with the GEventLoop. This will eventually
cause GTimerEvents to be dispatched to the GObject.
This needed a few supporting changes in the kernel:
- The PIT now ticks 1000 times/sec.
- select() now supports an arbitrary timeout.
- gettimeofday() now returns something in the tv_usec field.
With these changes, the clock window in guitest2 finally ticks on its own.
FileDescriptor will now keep a pointer to the original inode even after
opening it resolves to a character device.
Fixed up /bin/ls to display major and minor device numbers instead of size
for device files.
This required a fair bit of plumbing. The CharacterDevice::close() virtual
will now be closed by ~FileDescriptor(), allowing device implementations to
do custom cleanup at that point.
One big problem remains: if the master PTY is closed before the slave PTY,
we go into crashy land.
Only raw octal modes are supported right now.
This patch also changes mode_t from 32-bit to 16-bit to match the on-disk
type used by Ext2FS.
I also ran into EPERM being errno=0 which was confusing, so I inserted an
ESUCCESS in its place.
It's really only supported in Ext2FS since SynthFS doesn't really want you
mucking around with its files. This is pretty neat though :^)
I ran into some trouble with HashMap while working on this but opted to work
around it and leave that for a separate investigation.
This means we only have to do one fill_rect() per line and the whole process
ends up being ~10% faster than before.
Also added a read_tsc() syscall to give userspace access to the TSC.
This patch adds most of the plumbing for working file deletion in Ext2FS.
Directory entries are removed and inode link counts updated.
We don't yet update the inode or block bitmaps, I will do that separately.
Make PageDirectory retainable and have each Region co-own the PageDirectory
they're mapped into. When unmapped, Region has no associated PageDirectory.
This allows Region to automatically unmap itself when destroyed.
It's a bit confusing that the "current" process is not actually running
while we're inside the scheduler. Perhaps the scheduler should redirect
"current" to its own dummy Process. I'm not sure.
Regardless, this patch improves responsiveness by allowing the scheduler
to unblock a process right after it calls select() in case it already has
a pending wakeup request.
The system can finally idle without burning CPU. :^)
There are some issues with scheduling making the mouse cursor sloppy
and unresponsive that need to be dealt with.
When you open /dev/ptmx, you get a file descriptor pointing to one of the
available MasterPTY's. If none are available, you get an EBUSY.
This makes it possible to open multiple (up to 4) Terminals. :^)
To support this, I also added a CharacterDevice::open() that gets control
when VFS is opening a CharacterDevice. This is useful when we want to return
a custom FileDescriptor like we do here.
Userspace programs can now open /dev/gui_events and read a stream of GUI_Event
structs one at a time.
I was stuck on a stupid problem where we'd reenter Scheduler::yield() due to
having one of the has_data_available_for_reading() implementations using locks.
This is a lot better than having them in kmalloc memory. I'm gonna need
a way to keep track of which process owns which bitmap eventually,
maybe through some sort of resource keying system. We'll see.
The old approach only worked because of an overpermissive accident.
There's now a concept of supervisor physical pages that can be allocated.
They all sit in the low 4 MB of physical memory and are identity mapped,
shared between all processes, and only ring 0 can access them.
Also use a simple array of { dword, const char* } for the KSyms and put the
whole shebang in kmalloc_eternal() memory. This was a fugly source of
kmalloc perma-frag.
It walks all the live Inode objects and flushes pending metadata changes
wherever needed.
This could be optimized by keeping a separate list of dirty Inodes,
but let's not get ahead of ourselves.
Use a little template magic to have Retainable::release() call out to
T::will_be_destroyed() if such a function exists before actually calling
the destructor. This gives us full access to virtual functions in the
pre-destruction code.
This synchronous approach to inodes is silly, obviously. I need to rework
it so that the in-memory CoreInode object is the canonical inode, and then
we just need a sync() that flushes pending changes to disk.
The kernel now bills processes for time spent in kernelspace and userspace
separately. The accounting is forwarded to the parent process in reap().
This makes the "time" builtin in bash work.
This way the scheduler doesn't need to plumb the exit status into the waiter.
We still plumb the waitee pid though, I don't love it but it can be fixed.
mmap() will now map uncommitted pages that get allocated and zeroed upon the
first access. I also made /proc/PID/vm show number of "committed" bytes in
each region. This is so cool! :^)
I was surprised to find that dup()'ed fds don't share the close-on-exec flag.
That means it has to be stored separately from the FileDescriptor object.
Instead of memcpy'ing the entire screen every time we press enter at the
bottom, use the VGA start address register to make a "view" onto the
underlying memory that moves downward as we scroll.
Eventually we run out of memory and have to reset to the start of the
buffer. That's when we memcpy everything. It would be cool if there was
some way to get the hardware to act like a ring buffer with automatic
wrapping here but I don't know how to do that.
I didn't even put the { } properly around everything that would leak.
Let's make sure this works correctly by splitting out the work into a
helper called do_exec().
- Process::exec() needs to restore the original paging scope when called
on a non-current process.
- Add missing InterruptDisabler guards around g_processes access.
- Only flush the TLB when modifying the active page tables.
This is really sweet! :^) The four instances of /bin/sh spawned at
startup now share their read-only text pages.
There are problems and limitations here, and plenty of room for
improvement. But it kinda works.
All right, we can now mmap() a file and it gets magically paged in from fs
in response to an NP page fault. This is really cool :^)
I need to refactor this to support sharing of read-only file-backed pages,
but it's cool to just have something working.
First of all, change sys$mmap to take a struct SC_mmap_params since our
sycsall calling convention can't handle more than 3 arguments.
This exposed a bug in Syscall::invoke() needing to use clobber lists.
It was a bit confusing to debug. :^)
This is dirty but pretty cool! If we have a pending, unmasked signal for
a process that's blocked inside the kernel, we set up alternate stacks
for that process and unblock it to execute the signal handler.
A slightly different return trampoline is used here: since we need to
get back into the kernel, a dedicated syscall is used (sys$sigreturn.)
This restores the TSS contents of the process to the state it was in
while we were originally blocking in the kernel.
NOTE: There's currently only one "kernel resume TSS" so signal nesting
definitely won't work.
Processes are either alive (with many substates), dead or forgiven.
A dead process is forgiven when the parent waitpid()s on it.
Dead orphans are also forgiven.
There's a lot of work to be done around this.
It only works for sending a signal to a process that's in userspace code.
We implement reception by synthesizing a PUSHA+PUSHF in the receiving process
(operating on values in the TSS.)
The TSS CS:EIP is then rerouted to the signal handler and a tiny return
trampoline is constructed in a dedicated region in the receiving process.
Also hacked up /bin/kill to be able to send arbitrary signals (kill -N PID)
Implemented some syscalls: dup(), dup2(), getdtablesize().
FileHandle is now a retainable, since that's needed for dup()'ed fd's.
I didn't really test any of this beyond a basic smoke check.
sys$fork() now clones all writable regions with per-page COW bits.
The pages are then mapped read-only and we handle a PF by COWing the pages.
This is quite delightful. Obviously there's lots of work to do still,
and it needs better data structures, but the general concept works.
This turned out way better than the old code. ELF loading is now quite
straightforward, and we don't need the weird concept of subregions anymore.
Next step is to respect the is_writable flag.
This is quite cool! The syscall entry point plumbs the register dump
down to sys$fork(), which uses it to set up the child process's TSS
in order to resume execution right after the int 0x80 fork() call. :^)
This works pretty well, although there is some problem with the kernel
alias mappings used to clone the parent process's regions. If I disable
the MM::release_page_directory() code, there's no problem. Probably there's
a premature freeing of a physical page somehow.
We no longer disable interrupts around the whole affair.
Since MM manages per-process data structures, this works quite smoothly now.
Only procfs had to be tweaked with an InterruptDisabler.
