October 2024 - Devel - lists.tockos.org

Re: [Spam/Phising] Notification:- Password Update

by Leon Schuermann

"lists.tockos.org or Email Admin via Devel" <devel(a)lists.tockos.org> writes: > The password to your mailbox devel(a)lists.tockos.org has expired. Apologies for letting this email through, this is obviously phishing. The "Accept" and "Reject" buttons in the webinterface were too close together. :/

Oct. 14, 2024

Re: Looking for Insights on Tock's Context Switch Latency

by Samir Rashid

Hello Zhiyao, Unfortunately, I also do not have a great explanation for you. At 64MHz (on the NRF52840DK), your benchmark takes 4846 ms. I would like to point out some things. Your context switch diagram is underestimating the amount of context switches. The sequence you present is not correct because If a syscall does not immediately error (which IPC is not doing in this case), then the kernel triggers a context switch (ContextSwitchReason::SyscallFired). To be more precise, the kernel's process loop is going to mark that a syscall has fired and ask the scheduler whether it should immediately handle the syscall. The default round robin scheduler will always choose to switch to the other process. To revise your diagram: On the "client" side: - calls command() -> context switch to the kernel - (NEW) kernel defers processing the command() -> kernel round robin scheduler context switches to server's main() - (NEW) server app main loop immediately yields -> server context switches to kernel - kernel processes the client's command -> kernel context switch to the client app - client immediately runs yield_for -> context switch to kernel I empirically verified that the `loop { yield(); }` runs 10,000 times in the server. It is possible there is more overhead from the server calling `ipc_notify_client(pid);`, but I did not trace further in your timeline. I think the best method to get more info about this bug would be to do some debugging inside the kernel scheduler (`round_robin.rs`). Let me know if I got anything wrong, Professor Levy. Samir Fingerprint: DE65 F61B 7AD6 69C8 3972 1530 A81C A0CA 8957 AC94 On 2024-10-11 18:23, zhiyao.ma.98--- via Devel <devel(a)lists.tockos.org> wrote: > Hi Professor Levy, > > Thank you so much for looking into the problem. Please let me know if there is anything I can help on my side. > > Best regards, > Zhiyao > _______________________________________________ > Devel mailing list -- devel(a)lists.tockos.org > To unsubscribe send an email to devel-leave(a)lists.tockos.org

Oct. 14, 2024

Re: Looking for Insights on Tock's Context Switch Latency

by zhiyao.ma.98＠gmail.com

Hi Professor Levy, Thank you so much for looking into the problem. Please let me know if there is anything I can help on my side. Best regards, Zhiyao

Oct. 12, 2024

Re: Looking for Insights on Tock's Context Switch Latency

by Amit Levy

> current RTC counter) takes about 42uS (10,000 in ~2.1 seconds), so about > 21uS each way or 1680 instructions at 64MHz. That's about 13x off of Correction, I meant 50,000 here, not 10,000.

Oct. 11, 2024

Tock meeting agenda 2024-10-11

by Amit Levy

Meeting Link: https://meetings.dialpad.com/room/tock/1vnk9r3e1b No agenda items today. We can add some on the call. Otherwise, I'll suggest briefly talking about clarifying invariants of the alarm stack, followed by pull request triage. Agenda - Updates - TBD

Oct. 11, 2024

Call for Tock meeting agenda items 2024-10-11

by Amit Levy

Please response with proposed agenda items and +1s for tomorrows core meeting

Oct. 11, 2024

Re: Looking for Insights on Tock's Context Switch Latency

by Amit Levy

Hi Zhiyao, Thank you for including code. I haven't done so just yet, but will replicate this soon to both validate and try to debug/explain in more detail. Meanwhile, there area few things going on here, I think. First, to confirm some general numbers. At 64MHz (on the NRF52840DK), an almost no-op system-call (a context switch back and forth that reads the current RTC counter) takes about 42uS (10,000 in ~2.1 seconds), so about 21uS each way or 1680 instructions at 64MHz. That's about 13x off of what you're seeing _if_ no work is being done between those context switches, though of course there is. So on average, there is something like 12k instructions left to explain, which is indeed a lot. Second, it's worth noting that IPC in Tock is so unoptimized its not even funny. I added IPC in 2016 originally in service of a paper deadline, when there was no real use case for IPC for any Tock users, but as a proof of concept that, yes, this kind of interaction could also be supported. Since then it's hardly been touched. Recently there are some actual use cases that use and/or require IPC, so it's a good time to design a half-way decent system... Finally, IPC in Tock is quite different from FreeRTOS task notifications (as I understand them anyway), and as a result does quite a bit more. FreeRTOS task notifications directly pass control from the sender to the receiver rather than queue a notification to be handled asynchronously, which is what Tock IPC does. So there are a bunch of subsystems that get traversed in Tock (process scheduler, systick reset, hardware event loop, MPU reconfiguration, etc) that shouldn't need to be traversed in FreeRTOS. MPU reconfiguration is expensive (not _that_ expensive), but should only be happening twice here. Because the IPC is asynchronous, other things _could_ be happening between yieling in the client and receiving the notification in the server (or vice versa), such as handling hardware interupts, that that seems unlikely. In summary, while I don't have an off-the-cuff explanation for the high latency, I'm not _that_ surprised that IPC is very slow. To be continued... -Amit "zhiyao.ma.98--- via Devel" <devel(a)lists.tockos.org> writes: > Hello Tock developers, > > I am recently testing Tock's context switch speed with an STM32F412G-Discovery board, and I noticed that the latency is longer than expected. I hope to confirm that my settings are correct and gain some insights behind the data. > > To test the context switch speed, I implemented two apps called "server" and "client." The server sets up an IPC service that simply notifies completion back to the client. The client invokes the IPC service 10,000 times in a loop and then calculates the average ping-pong time. > > On my board, the 10,000 ping-pong takes 10,748 ms. That is more than 1ms for each ping-pong. > > I believe the CPU clock is set to 96 MHz by the default kernel configuration for the board. With a similar clock speed, a ping-pong implemented through task notification on FreeRTOS takes around 13 μs, which is almost two orders of magnitude faster than Tock. > > Professor Campbell pointed out to me that there are multiple context switches for each ping-pong: > > On the "client" side: > - calls command() -> context switch to the kernel > - command returns -> context switch back to the client app > - client app calls yield -> context switch back to the kernel > > Next, on the “server” side: > - upcall scheduled -> context switch to the server app > - calls ipc_notify_client -> context switch to the kernel to schedule the upcall > - command returns -> context switch back to the server app > - yield -> context switch to the kernel > > Finally, on the “client” side again: > - upcall scheduled -> context switch to the client app > > However, even if there are 8 context switches per ping-pong, on average each context switch still takes around 135 μs, which is around 13k instructions executed per context switch on a 96 MHz CPU. The number still seems to be high. > > Below is the source code for the “server” and “client” app. > > Server: > > ```c > #include <libtock/kernel/ipc.h> > #include <libtock/tock.h> > > static void ipc_callback( > int pid, > __attribute__ ((unused)) int len, > __attribute__ ((unused)) int buf, > __attribute__ ((unused)) void* ud) { > ipc_notify_client(pid); > } > > int main(void) { > ipc_register_service_callback("server", ipc_callback, > NULL); > > while (1) { > yield(); > } > } > ``` > > Client: > > ```c > #include <stdio.h> > #include <libtock-sync/services/alarm.h> > #include <libtock/kernel/ipc.h> > > size_t service_id = -1; > > char buf[64] __attribute__((aligned(64))); > > bool done = false; > > static void ipc_callback(__attribute__ ((unused)) int pid, > __attribute__ ((unused)) int len, > __attribute__ ((unused)) int arg2, > __attribute__ ((unused)) void* ud) { > done = true; > } > > static uint32_t get_time_ms(void) { > struct timeval tv; > libtock_alarm_gettimeasticks(&tv, NULL); > return (tv.tv_sec * 1000) + (tv.tv_usec / 1000); > } > > int main(void) { > int ret; > > ret = ipc_discover("server", &service_id); > while (ret != RETURNCODE_SUCCESS) { > printf("No server, retry in 1 second.\n"); > libtocksync_alarm_delay_ms(1000); > } > > printf("Server discovered.\n"); > > ipc_register_client_callback(service_id, ipc_callback, NULL); > ipc_share(service_id, buf, 64); > > uint32_t start_time = get_time_ms(); > > for (int i = 0; i < 10000; ++i) { > done = false; > ipc_notify_service(service_id); > yield_for(&done); > } > > uint32_t end_time = get_time_ms(); > > printf("Ping-pong 10000 times take %lu ms.\n", end_time - start_time); > > return 0; > } > ``` > _______________________________________________ > Devel mailing list -- devel(a)lists.tockos.org > To unsubscribe send an email to devel-leave(a)lists.tockos.org

Oct. 11, 2024

Looking for Insights on Tock's Context Switch Latency

by zhiyao.ma.98＠gmail.com

Hello Tock developers, I am recently testing Tock's context switch speed with an STM32F412G-Discovery board, and I noticed that the latency is longer than expected. I hope to confirm that my settings are correct and gain some insights behind the data. To test the context switch speed, I implemented two apps called "server" and "client." The server sets up an IPC service that simply notifies completion back to the client. The client invokes the IPC service 10,000 times in a loop and then calculates the average ping-pong time. On my board, the 10,000 ping-pong takes 10,748 ms. That is more than 1ms for each ping-pong. I believe the CPU clock is set to 96 MHz by the default kernel configuration for the board. With a similar clock speed, a ping-pong implemented through task notification on FreeRTOS takes around 13 μs, which is almost two orders of magnitude faster than Tock. Professor Campbell pointed out to me that there are multiple context switches for each ping-pong: On the "client" side: - calls command() -> context switch to the kernel - command returns -> context switch back to the client app - client app calls yield -> context switch back to the kernel Next, on the “server” side: - upcall scheduled -> context switch to the server app - calls ipc_notify_client -> context switch to the kernel to schedule the upcall - command returns -> context switch back to the server app - yield -> context switch to the kernel Finally, on the “client” side again: - upcall scheduled -> context switch to the client app However, even if there are 8 context switches per ping-pong, on average each context switch still takes around 135 μs, which is around 13k instructions executed per context switch on a 96 MHz CPU. The number still seems to be high. Below is the source code for the “server” and “client” app. Server: ```c #include <libtock/kernel/ipc.h> #include <libtock/tock.h> static void ipc_callback( int pid, __attribute__ ((unused)) int len, __attribute__ ((unused)) int buf, __attribute__ ((unused)) void* ud) { ipc_notify_client(pid); } int main(void) { ipc_register_service_callback("server", ipc_callback, NULL); while (1) { yield(); } } ``` Client: ```c #include <stdio.h> #include <libtock-sync/services/alarm.h> #include <libtock/kernel/ipc.h> size_t service_id = -1; char buf[64] __attribute__((aligned(64))); bool done = false; static void ipc_callback(__attribute__ ((unused)) int pid, __attribute__ ((unused)) int len, __attribute__ ((unused)) int arg2, __attribute__ ((unused)) void* ud) { done = true; } static uint32_t get_time_ms(void) { struct timeval tv; libtock_alarm_gettimeasticks(&tv, NULL); return (tv.tv_sec * 1000) + (tv.tv_usec / 1000); } int main(void) { int ret; ret = ipc_discover("server", &service_id); while (ret != RETURNCODE_SUCCESS) { printf("No server, retry in 1 second.\n"); libtocksync_alarm_delay_ms(1000); } printf("Server discovered.\n"); ipc_register_client_callback(service_id, ipc_callback, NULL); ipc_share(service_id, buf, 64); uint32_t start_time = get_time_ms(); for (int i = 0; i < 10000; ++i) { done = false; ipc_notify_service(service_id); yield_for(&done); } uint32_t end_time = get_time_ms(); printf("Ping-pong 10000 times take %lu ms.\n", end_time - start_time); return 0; } ```

Oct. 10, 2024

Re: Discussion on u32

by Leon Schuermann

Hi Lawrence, Thanks for posting this! Apart from the other posts, I wanted to briefly summarize the next steps we discussed on the core call on Friday, the notes of which are up in a PR [1]. There are three types of changes to the system call interface that we'll have to think about for supporting a CHERI and/or 64-bit platform: 1. The system call driver return values. Possible changes are to - extend the `SyscallReturn` enum with additional variants. We'll have to make sure that we did not promise this enumeration to be exhaustive in our documentation (likely TRD104) and ensure that none of our current userspace libraries rely on the current enum definitions to be exhaustive. We can likely implement this in a TRD that extends TRD 104, and as part of a minor version bump. - change certain system call's return values, to better match the semantic meaning of their return value and to avoid switching between variants on different platforms. For this change to be rolled out on 32-bit platforms, we'll need to wait for a major (breaking) release. Depending on where this is documented, we probably want a full revision of TRD 104 and other applicable reference documents for this. 2. System call arguments. Driver and subdriver (command number) arguments should always be 32-bit. We should check if we state something different anywhere, and then manifest these changes in the kernel code base. Notably, libtock-rs already assumes them to be u32s. For the two command arguments, we need to decide whether we want to encode more typing information into a driver's interface. Currently, these arguments are `usize`. We very much intended these to be only `u32`s (such that driver interfaces are identical across platforms). The kernel still uses `usize` for these as we didn't want to change interfaces when practically, given we only supported 32-bit platforms, these types were identical. It is clear that restricting these to `u32`s will be prohibitive of some applications, such as passing length parameters over this interface. However, we'd probably still want to document and enforce which type a driver actually expects to be passed as a parameter. One such soltion would leave the raw interface types as `usize`, but extend a proposal like https://github.com/tock/tock/pull/4112 to add type information per command-number and implicitly cast this `usize` per sub-command. 3. Upcall parameters. Upcall parameters should be able to return buffer lengths, etc. Therefore, they also should be `usize` wide. TBD: do we want more typing information here? It seems harder to enforce the relation between upcall-number and types passed. Maybe we can extend the Grant type to encode this information? In summary, we'll likely want to review these interface changes as part of either a minor extension or revision of TRD 104, to ensure that we stay backwards compatible. A full interface revision can then be proposed as a full TRD. Finally, I want to echo Hudson's and Johnathan's statements from the call: as long as we don't change the existing 32-bit non-CHERI interface, it should be fine to introduce a new ABI for these new platforms. However, we want to make sure that this doesn't cause much overhead or churn for kernel & capsule developers. This interface should probably also be proposed in a TRD. It seems best to continue figuring out these open questions and details on the mailing list. Hope this summary is useful! I hope I didn't mischaracterize anyone's opinion, please let me know if you disagree with anything. -Leon [1]: https://github.com/tock/tock/pull/4189

Oct. 7, 2024

Re: Discussion on u32

by Johnathan Van Why

I don't think (2) is acceptable in Tock 2, because it would be a breaking change. However, I don't think we need to add as many new return variants as (1) proposes. I don't think we care about any platforms where address offsets may exceed 2 GiB (correct me if I am wrong), so we can pack offsets into i32/u32 even on our 64-bit platforms. Therefore, I think the only return variants we need to add are: 1. SuccessWithMetaPtr: Used by Memops 1-9. 2. SuccessWith2MetaPtr: Used by Subscribe and all three Allows. 3. FailureWith2MetaPtr Used by Subscribe and all three Allows. We would keep the current return variants for all currently-stabilized system calls on 32-bit non-CHERI platforms, and use the new return variants on all other platforms (including all 64-bit and CHERI). That keeps backwards-compatibility on platforms that Tock 2 currently supports while providing better semantics on 64-bit and CHERI platforms. For Tock 3, it might make sense to go somewhat further than just u32, u64, usize, and MetaPtr. How about signed integer, floats, bool, etc? It would be nice if the system call layer could indicate that the button upcall <https://github.com/tock/tock/blob/65dda58c893e5b1fb49da7e4389552584758d5e1/…> receives a (u32, bool) and the temperature reading upcall <https://github.com/tock/tock/blob/65dda58c893e5b1fb49da7e4389552584758d5e1/…> receives an i32 (this isn't in the documentation), rather than pretending that all upcalls receive (u32, u32, u32). On Thu, Oct 3, 2024 at 3:51 PM Amit Levy via Devel <devel(a)lists.tockos.org> wrote: > > The current set of return codes include a type. e.g. calls like allow/brk > currently return SyscallReturnU32U32 or SyscallReturnU32. Clearly this is > not true for 64-bit platforms. Or for CHERI where the size of an address is > not the size of a pointer. > > Excellent point, and this is definitely something to address. Thanks for > noticing and bringing this up. > > I think given this is in the context of switching from only 32 support to > adding 64 bit and/or CHERI, I don't think we should be overly concerned > about breaking backwards compatibility. Instead, this is potentially a fine > reason to think towards a 3.0 system call interface---one which shouldn't > be overly difficult to port to, but wouldn't be binary compatible with > applications compiled for 2.0. (we may lump in fully relocatable binaries > and/or first-class non-XIP support if changing some expectations about > binaries helps us accomplish that, as as potentially changes to core system > interfaces, such as IPC) > > >From that perspective, I think (1) is untenable. I think we should be > much more concerned with portability of, e.g., capsules and a unified > system call interface than with backwards compatibility. I don't think > there is any advantage to (1) other than backwards compatibility. > > I think (2) is the answer. It follows the general idea in the MetaPtr PR > (#4174) of converting naming things what they actually are. > > (3) I understand the expediency for a downstream port, but I don't see an > advantage to this if we can do 2 (which I think we can). > > (4) Similar to 3. I think it would be _reasonable_, if we went with 3, to > bump the major version number and redefine "success" to have the current > value of successu32, such that it just happened to be the case that 2.0 > binaries also turned out to work unmodified. > > -Amit > _______________________________________________ > Devel mailing list -- devel(a)lists.tockos.org > To unsubscribe send an email to devel-leave(a)lists.tockos.org >

Oct. 5, 2024