I think this is a good discussion for Friday, but I wanted to send out an advance email to give time for people to gather their thoughts in advance The current set of return codes include a type. e.g. calls like allow/brk currently return SyscallReturnU32U32 or SyscallReturnU32. Clearly this is not true for 64-bit platforms. Or for CHERI where the size of an address is not the size of a pointer.I think there are a few options: 1. 1) They should return different codes on different platforms: SyscallReturnu32 on 32-bit, SyscallReturnu64on 64-bit, SyscallReturnMetaPtr on CHERI etc with some meta constructors to help choose. Userspace should expect the same and be If-Defed appropriately. 2. 2) They should return a new more specific code SyscallReturnPtr / SyscallReturnUsize on all platforms, and user space should check for just that one.e 3. 3) These calls are already made type safe because of the wrappers in libtock-c / libtock-rs. Just use SyscallSuccess for these built in syscalls. Wrappers know what they are getting. 4. 4) A true evil one. Option 3, but SyscallReturnu32 is the code for legacy reasons. You know what you are actually getting. What I actually did on my fork was just 3 (because all these different codes were getting in the way), but this does break compatibility unless there is a matching change in libtock-c/libtock-rs and it would probably be better not to do so. In fact, both 2 and 3 are breaking API changes. This means that possibly we should prefer (1), simply for backwards compatibility? This is roughly what the consttructor I added for SyscallReturnUsize does: selects one of SyscallReturnu32 or SyscallReturnu64 as appropriate. But possibly we will need a few more as soon as CHERI comes into play.As for types for commands:Currently in the kernel commands are:fn command(&self, cmd_num: usize, arg1: usize, arg2: usize, appid: ProcessId) Which is already usize-ified.In libtock-c: syscall_return_t command(uint32_t driver, uint32_t command, size_t arg1, size_t arg2); There is clearly a mismatch there on command number. I am of the mind that the driver/command being u32 is fine, and args should really be usize. For instance, a command might realistically take an offset to a previously allowed buffer. This is usize, not u32.Upcalls: in the kernel were all u32, I changed most of the arguments usize apart from data/fn pointer which are MetaPtr. i.e., the type in C should be:typedef void (subscribe_upcall)(size_t, size_t, size_t, void*);where currently it istypedef void (subscribe_upcall)(int, int, int, void*);And I think its worth some discussion whether people are happy with this. It means nothing for 32-bit libtock-c because the ABI for passing usize is the same as int on 32-bit platforms. In libtock-rs I have the patches ready to go to define the upcall trait on both u32 and usize so it won't break any code there. Lawrence
At least some of the complexity results from system call arguments/return values which are sometimes arbitrary bit-limited values, and sometimes pointers. For example, memop 6 returns a pointer, whereas memop 7 returns a quantity. In contrast, allow only has pointers as arguments and returns pointers. One idea to consider is restructuring the system call interface to remove this ambiguity; arguments per system call are either pointers or values, but never both. - Brad On Wed, Oct 2, 2024 at 5:53 PM Lawrence Esswood via Devel < devel@lists.tockos.org> wrote:
I think this is a good discussion for Friday, but I wanted to send out an advance email to give time for people to gather their thoughts in advance
The current set of return codes include a type. e.g. calls like allow/brk currently return SyscallReturnU32U32 or SyscallReturnU32. Clearly this is not true for 64-bit platforms. Or for CHERI where the size of an address is not the size of a pointer.I think there are a few options:
1. 1) They should return different codes on different platforms: SyscallReturnu32 on 32-bit, SyscallReturnu64on 64-bit, SyscallReturnMetaPtr on CHERI etc with some meta constructors to help choose. Userspace should expect the same and be If-Defed appropriately. 2. 2) They should return a new more specific code SyscallReturnPtr / SyscallReturnUsize on all platforms, and user space should check for just that one.e 3. 3) These calls are already made type safe because of the wrappers in libtock-c / libtock-rs. Just use SyscallSuccess for these built in syscalls. Wrappers know what they are getting. 4. 4) A true evil one. Option 3, but SyscallReturnu32 is the code for legacy reasons. You know what you are actually getting.
What I actually did on my fork was just 3 (because all these different codes were getting in the way), but this does break compatibility unless there is a matching change in libtock-c/libtock-rs and it would probably be better not to do so. In fact, both 2 and 3 are breaking API changes. This means that possibly we should prefer (1), simply for backwards compatibility? This is roughly what the consttructor I added for SyscallReturnUsize does: selects one of SyscallReturnu32 or SyscallReturnu64 as appropriate. But possibly we will need a few more as soon as CHERI comes into play.As for types for commands:Currently in the kernel commands are:fn command(&self, cmd_num: usize, arg1: usize, arg2: usize, appid: ProcessId) Which is already usize-ified.In libtock-c: syscall_return_t command(uint32_t driver, uint32_t command, size_t arg1, size_t arg2); There is clearly a mismatch there on command number. I am of the mind that the driver/command being u32 is fine, and args should really be usize. For instance, a command might realistically take an offset to a previously allowed buffer. This is usize, not u32.Upcalls: in the kernel were all u32, I changed most of the arguments usize apart from data/fn pointer which are MetaPtr. i.e., the type in C should be:typedef void (subscribe_upcall)(size_t, size_t, size_t, void*);where currently it istypedef void (subscribe_upcall)(int, int, int, void*);And I think its worth some discussion whether people are happy with this. It means nothing for 32-bit libtock-c because the ABI for passing usize is the same as int on 32-bit platforms. In libtock-rs I have the patches ready to go to define the upcall trait on both u32 and usize so it won't break any code there.
Lawrence
_______________________________________________ Devel mailing list -- devel@lists.tockos.org To unsubscribe send an email to devel-leave@lists.tockos.org
The current set of return codes include a type. e.g. calls like allow/brk currently return SyscallReturnU32U32 or SyscallReturnU32. Clearly this is not true for 64-bit platforms. Or for CHERI where the size of an address is not the size of a pointer.
Excellent point, and this is definitely something to address. Thanks for noticing and bringing this up. I think given this is in the context of switching from only 32 support to adding 64 bit and/or CHERI, I don't think we should be overly concerned about breaking backwards compatibility. Instead, this is potentially a fine reason to think towards a 3.0 system call interface---one which shouldn't be overly difficult to port to, but wouldn't be binary compatible with applications compiled for 2.0. (we may lump in fully relocatable binaries and/or first-class non-XIP support if changing some expectations about binaries helps us accomplish that, as as potentially changes to core system interfaces, such as IPC) From that perspective, I think (1) is untenable. I think we should be much more concerned with portability of, e.g., capsules and a unified system call interface than with backwards compatibility. I don't think there is any advantage to (1) other than backwards compatibility. I think (2) is the answer. It follows the general idea in the MetaPtr PR (#4174) of converting naming things what they actually are. (3) I understand the expediency for a downstream port, but I don't see an advantage to this if we can do 2 (which I think we can). (4) Similar to 3. I think it would be _reasonable_, if we went with 3, to bump the major version number and redefine "success" to have the current value of successu32, such that it just happened to be the case that 2.0 binaries also turned out to work unmodified. -Amit
I don't think (2) is acceptable in Tock 2, because it would be a breaking change. However, I don't think we need to add as many new return variants as (1) proposes. I don't think we care about any platforms where address offsets may exceed 2 GiB (correct me if I am wrong), so we can pack offsets into i32/u32 even on our 64-bit platforms. Therefore, I think the only return variants we need to add are: 1. SuccessWithMetaPtr: Used by Memops 1-9. 2. SuccessWith2MetaPtr: Used by Subscribe and all three Allows. 3. FailureWith2MetaPtr Used by Subscribe and all three Allows. We would keep the current return variants for all currently-stabilized system calls on 32-bit non-CHERI platforms, and use the new return variants on all other platforms (including all 64-bit and CHERI). That keeps backwards-compatibility on platforms that Tock 2 currently supports while providing better semantics on 64-bit and CHERI platforms. For Tock 3, it might make sense to go somewhat further than just u32, u64, usize, and MetaPtr. How about signed integer, floats, bool, etc? It would be nice if the system call layer could indicate that the button upcall <https://github.com/tock/tock/blob/65dda58c893e5b1fb49da7e4389552584758d5e1/d...> receives a (u32, bool) and the temperature reading upcall <https://github.com/tock/tock/blob/65dda58c893e5b1fb49da7e4389552584758d5e1/d...> receives an i32 (this isn't in the documentation), rather than pretending that all upcalls receive (u32, u32, u32). On Thu, Oct 3, 2024 at 3:51 PM Amit Levy via Devel <devel@lists.tockos.org> wrote:
The current set of return codes include a type. e.g. calls like allow/brk currently return SyscallReturnU32U32 or SyscallReturnU32. Clearly this is not true for 64-bit platforms. Or for CHERI where the size of an address is not the size of a pointer.
Excellent point, and this is definitely something to address. Thanks for noticing and bringing this up.
I think given this is in the context of switching from only 32 support to adding 64 bit and/or CHERI, I don't think we should be overly concerned about breaking backwards compatibility. Instead, this is potentially a fine reason to think towards a 3.0 system call interface---one which shouldn't be overly difficult to port to, but wouldn't be binary compatible with applications compiled for 2.0. (we may lump in fully relocatable binaries and/or first-class non-XIP support if changing some expectations about binaries helps us accomplish that, as as potentially changes to core system interfaces, such as IPC)
From that perspective, I think (1) is untenable. I think we should be much more concerned with portability of, e.g., capsules and a unified system call interface than with backwards compatibility. I don't think there is any advantage to (1) other than backwards compatibility.
I think (2) is the answer. It follows the general idea in the MetaPtr PR (#4174) of converting naming things what they actually are.
(3) I understand the expediency for a downstream port, but I don't see an advantage to this if we can do 2 (which I think we can).
(4) Similar to 3. I think it would be _reasonable_, if we went with 3, to bump the major version number and redefine "success" to have the current value of successu32, such that it just happened to be the case that 2.0 binaries also turned out to work unmodified.
-Amit _______________________________________________ Devel mailing list -- devel@lists.tockos.org To unsubscribe send an email to devel-leave@lists.tockos.org
Hi Lawrence, Thanks for posting this! Apart from the other posts, I wanted to briefly summarize the next steps we discussed on the core call on Friday, the notes of which are up in a PR [1]. There are three types of changes to the system call interface that we'll have to think about for supporting a CHERI and/or 64-bit platform: 1. The system call driver return values. Possible changes are to - extend the `SyscallReturn` enum with additional variants. We'll have to make sure that we did not promise this enumeration to be exhaustive in our documentation (likely TRD104) and ensure that none of our current userspace libraries rely on the current enum definitions to be exhaustive. We can likely implement this in a TRD that extends TRD 104, and as part of a minor version bump. - change certain system call's return values, to better match the semantic meaning of their return value and to avoid switching between variants on different platforms. For this change to be rolled out on 32-bit platforms, we'll need to wait for a major (breaking) release. Depending on where this is documented, we probably want a full revision of TRD 104 and other applicable reference documents for this. 2. System call arguments. Driver and subdriver (command number) arguments should always be 32-bit. We should check if we state something different anywhere, and then manifest these changes in the kernel code base. Notably, libtock-rs already assumes them to be u32s. For the two command arguments, we need to decide whether we want to encode more typing information into a driver's interface. Currently, these arguments are `usize`. We very much intended these to be only `u32`s (such that driver interfaces are identical across platforms). The kernel still uses `usize` for these as we didn't want to change interfaces when practically, given we only supported 32-bit platforms, these types were identical. It is clear that restricting these to `u32`s will be prohibitive of some applications, such as passing length parameters over this interface. However, we'd probably still want to document and enforce which type a driver actually expects to be passed as a parameter. One such soltion would leave the raw interface types as `usize`, but extend a proposal like https://github.com/tock/tock/pull/4112 to add type information per command-number and implicitly cast this `usize` per sub-command. 3. Upcall parameters. Upcall parameters should be able to return buffer lengths, etc. Therefore, they also should be `usize` wide. TBD: do we want more typing information here? It seems harder to enforce the relation between upcall-number and types passed. Maybe we can extend the Grant type to encode this information? In summary, we'll likely want to review these interface changes as part of either a minor extension or revision of TRD 104, to ensure that we stay backwards compatible. A full interface revision can then be proposed as a full TRD. Finally, I want to echo Hudson's and Johnathan's statements from the call: as long as we don't change the existing 32-bit non-CHERI interface, it should be fine to introduce a new ABI for these new platforms. However, we want to make sure that this doesn't cause much overhead or churn for kernel & capsule developers. This interface should probably also be proposed in a TRD. It seems best to continue figuring out these open questions and details on the mailing list. Hope this summary is useful! I hope I didn't mischaracterize anyone's opinion, please let me know if you disagree with anything. -Leon [1]: https://github.com/tock/tock/pull/4189
participants (5)
-
Amit Levy -
Brad Campbell -
Johnathan Van Why -
Lawrence Esswood -
Leon Schuermann