Question about RISCV cpu_start

[Giri2801]

Since RISCV spec allows WFI to be implemented as a NOP, shouldn't asm("WFI"); be in a while loop, as is commonly done?

https://github.com/apache/nuttx/blob/master/arch/risc-v/src/common/riscv_cpustart.c#L78

void riscv_cpu_boot(int cpu)
{
  ....
  /* Enable machine software interrupt for IPI to boot */

  up_enable_irq(RISCV_IRQ_SOFT);

  /* Wait interrupt */

  asm("WFI");

#ifdef CONFIG_BUILD_KERNEL
  /* Initialize the per CPU areas */
 ...
}

[acassis]

I don't understand your point. Maybe @lupyuen @xiaoxiang781216 or @tmedicci could have a better understanding, because they are more used to RISC-V than I.

[lupyuen]

Hi @Giri2801 I believe you're referring to this in the RISC-V Privileged Spec (Page 47):

The Wait for Interrupt instruction (WFI) provides a hint to the implementation that the current
hart can be stalled until an interrupt might need servicing ... The purpose of the WFI instruction is to provide a hint to the implementation, and so a legal implementation is to simply implement WFI as a NOP.

So far on the RISC-V Platforms supported by NuttX, WFI has been tested OK as a "Loop Forever" instruction. According to the spec, it's potentially possible that a Future RISC-V Platform might implement WFI as NOP, and affect the NuttX Implementation.

Before we propose the change (and do Regression Test across all RISC-V Platforms), are you aware of any platforms that might already implement WFI as NOP? Thanks.

[lupyuen]

Hi @xiaoxiang781216 and @tmedicci : It might be good to update the WFI code in NuttX. Right now we assume that WFI will loop forever, but it's possible that the behaviour will change for future RISC-V SoCs. We could follow the pattern used by Linux Boot Code, with WFI inside a Loop: arch/riscv/kernel/head.S

/* This will loop forever */
.Lloop:
	wfi
	j .Lloop

Why WFI inside a Loop? I suppose WFI might be good for reducing Power Consumption, for the SoCs that actually implement WFI as Loop Forever. For other SoCs, it will simply loop forever while consuming extra power. Thanks!

[Giri2801]

Hi @lupyuen , I'm working in Shakti Microprocessor team, and our hardware team had recently changed the implementation of WFI to wait for 1024 cycles, and then go through. I had observed that Nuttx was not booting up after that, so just put in this query. Thanks for the quick response!

[lupyuen]

@Giri2801 Your project sounds cool! Hope I understand you correctly:

1. At Startup: NuttX calls qemu_rv_start
2. Which calls riscv_cpu_boot start each CPU
3. riscv_cpu_boot calls WFI to wait for an interrupt (IPI) before it proceeds
4. Your implementation of WFI waits 1024 cycles. (Because the RISC-V Spec says WFI is a Hint, not an actual Implementation)
5. So riscv_cpu_boot is stuck waiting for the interrupt. Is this correct?

Are we suggesting that we should use a Spin-Wait for the CPU to boot? Thanks!

[acassis]

Amazing @Giri2801 !!!! Please submit support to Shakti microprocessor to mainline, this way more people in India could test it!

[Giri2801]

@lupyuen Your step 5 alone is slightly different I guess. The next line is

struct tcb_s *tcb = this_task();

https://github.com/apache/nuttx/blob/master/arch/risc-v/src/common/riscv_cpustart.c#L94.

So this_task() will return valid thread block, if core 0 has initialized it properly. But core 0 initialization is complete only after the IPI is received. If we proceed before actually getting interrupt, this_task() returns invalid pointer, and I'm getting an access fault. Hence we need a loop here, because we need to wait till IPI comes to signal all initialization by core 0 is done. This is what I have understood, might not be fully right.

@acassis Yes, we are working on submitting support for RISCV based Shakti Quadcore.

[lupyuen]

If we proceed before actually getting interrupt, this_task() returns invalid pointer, and I'm getting an access fault. Hence we need a loop here, because we need to wait till IPI comes to signla all initializations by core 0 is done

Hi @xiaoxiang781216 and @tmedicci : Would you know how we can fix this? Thanks.

[Giri2801]

Hi,
I had found this peace of code in riscv-pk. I think this would help us here. The loop should be something like :

while(1) {
    asm("wfi");
    # check if interrupt pending 
    # is set for IPI, that is,
    # software interrupt pending
    if (pending) {
         break;
    }
}

[lupyuen]

@Giri2801 Sorry I'm not so great at RISC-V Assembly, wonder if you could implement this in your fork of NuttX Kernel, and verify if it works? Thanks

[inochisa]

Hi, I had found this peace of code in riscv-pk. I think this would help us here. The loop should be something like :

while(1) {
    asm("wfi");
    # check if interrupt pending 
    # is set for IPI, that is,
    # software interrupt pending
    if (pending) {
         break;
    }
}

I think wfi loop is necessary, As AIA allow wfi to respond interrupt from another privileged mode (sec 5.5), this will break if we use a S-mode wfi but have a M-mode interrupt.