Skip to content

Conversation

@segfauIt
Copy link
Contributor

@segfauIt segfauIt commented Oct 9, 2025

It is intuitive to allow each CPU have its own tick counter.

This commit provides the key concepts of per-CPU variable and serves as a warm-up for the Lock Lab: memory allocator.

(*) Preemption

Accessing per-CPU variable must disable preemption to avoid data races.

For example, a process tries to update the tick counter of CPU0, but is switched to another CPU before store instruction, and the subsequent process running on CPU0 also wants to update the tick counter of CPU0.

tick++ => load    a5, (tick)
          addi    a5, a5, 1
          store   a5, (tick)

In xv6, disabling and enabling preemption can be controlled by push_off() and pop_off().

(*) Remove lock contention and deadlock

Previously, interrupts had to be disabled before holding the tickslock in sys_uptime() to avoid deadlock, as this lock was also used in the interrupt context, i.e., clockintr().

Note: however, in xv6, invoking sleep() requires holding a lock, and we need to adjust this restriction.

(*) False sharing

+-----------------+  +-----------------+
|            CPU0 |  |            CPU1 |
| L1 cache        |  | L1 cache        |
| +-------------+ |  | +-------------+ |
| | tick0 tick1 | |  | | tick0 tick1 | |
| +-------------+ |  | +-------------+ |
+-----------------+  +-----------------+

L2 cache
+-------------+
| tick0 tick1 |
+-------------+

Per-CPU variables should be placed in different cache lines to avoid false sharing, that is,

+-----------------+  +-----------------+
|            CPU0 |  |            CPU1 |
| L1 cache        |  | L1 cache        |
| +-------------+ |  | +-------------+ |
| | tick0       | |  | | tick1       | |
| +-------------+ |  | +-------------+ |
+-----------------+  +-----------------+

L2 cache
+-------------+
| tick0       |
+-------------+
| tick1       |
+-------------+

In addition, per-CPU tick counter also improves fairness. Previously, wakeup() was handled only by CPU0 in clockintr(). Ideally, each CPU should have its own sleep queue.
https://github.com/segfauIt/xv6-riscv/blob/e60f18e/kernel/proc.c#L555-L626

It is intuitive to allow each CPU have its own tick counter.

This commit provides the key concepts of per-CPU variable and serves
as a warm-up for the Lock Lab: memory allocator.

(*) Preemption

    Accessing per-CPU variable must disable preemption to avoid data
    races.

    For example, a process tries to update the tick counter of CPU0,
    but is switched to another CPU before store instruction, and the
    subsequent process running on CPU0 also wants to update the tick
    counter of CPU0.

    tick++ => load    a5, (tick)
              addi    a5, a5, 1
              store   a5, (tick)

    In xv6, disabling and enabling preemption can be controlled by
    push_off() and pop_off().

(*) Remove lock contention and deadlock

    Previously, interrupts had to be disabled before holding the
    tickslock in sys_uptime() to avoid deadlock, as this lock was
    also used in the interrupt context, i.e., clockintr().

    Note: however, in xv6, invoking sleep() requires holding a lock,
    and we need to adjust this restriction.

(*) False sharing

    +-----------------+  +-----------------+
    |            CPU0 |  |            CPU1 |
    | L1 cache        |  | L1 cache        |
    | +-------------+ |  | +-------------+ |
    | | tick0 tick1 | |  | | tick0 tick1 | |
    | +-------------+ |  | +-------------+ |
    +-----------------+  +-----------------+

    L2 cache
    +-------------+
    | tick0 tick1 |
    +-------------+

    Per-CPU variables should be placed in different cache lines to
    avoid false sharing, that is,

    +-----------------+  +-----------------+
    |            CPU0 |  |            CPU1 |
    | L1 cache        |  | L1 cache        |
    | +-------------+ |  | +-------------+ |
    | | tick0       | |  | | tick1       | |
    | +-------------+ |  | +-------------+ |
    +-----------------+  +-----------------+

    L2 cache
    +-------------+
    | tick0       |
    +-------------+
    | tick1       |
    +-------------+

In addition, per-CPU tick counter also improves fairness. Previously,
wakeup() was handled only by CPU0 in clockintr(). Ideally, each CPU
should have its own sleep queue.
https://github.com/segfauIt/xv6-riscv/blob/e60f18e/kernel/proc.c#L555-L626
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant