RISC-V Ox64 BL808 SBC: UART Interrupt and Platform-Level Interrupt Controller (PLIC)

đź“ť 3 Dec 2023

Platform-Level Interrupt Controller for Pine64 Ox64 64-bit RISC-V SBC (Bouffalo Lab BL808)

“It’s time for the little red chicken’s bedtime story - and a reminder from Papa to try not to interrupt. But the chicken can’t help herself!”

— “Interrupting Chicken”

Our Story today is all about RISC-V Interrupts on the tiny adorable Pine64 Ox64 BL808 64-bit Single-Board Computer (pic below)…

We’ll walk through the steps with a simple barebones operating system: Apache NuttX RTOS. (Real-Time Operating System)

Though we’ll hit a bumpy journey with our work-in-progress NuttX on Ox64…

We begin our story…

Pine64 Ox64 64-bit RISC-V SBC (Bouffalo Lab BL808)

1 Platform-Level Interrupt Controller

What’s this PLIC?

Platform-Level Interrupt Controller (PLIC) is the hardware inside our BL808 SoC that controls the forwarding of Peripheral Interrupts to our 64-bit RISC-V CPU.

(Like the Interrupts for UART, I2C, SPI, …)

Why should we bother with PLIC?

Suppose we’re using the Serial Console on Ox64 SBC (pic below)…

That’s why it’s good to understand how PLIC works with an Operating System. (Like Linux or NuttX)

PLIC handles all kinds of Interrupts?

Yep plenty! We identify each Interrupt by its RISC-V IRQ Number. (IRQ means Interrupt Request Number)

NuttX uses its own NuttX IRQ Number…

That’s because NuttX reserves a bunch of IRQ Numbers for Internal Use. (Hence the Offset of 25)

Let’s figure out the IRQ Number for Serial Console…

(PLIC is documented in C906 User Manual, Page 74)

(See the Official PLIC Spec)

BL808 Platform-Level Interrupt Controller

2 UART Interrupt

What’s the Interrupt Number for the Serial Console?

To enable Text Input in the Ox64 Serial Console, we need the Interrupt Number for the UART Controller…

Therefore the RISC-V IRQ Number for our Serial Console (UART3) is 20.

Remember that NuttX uses its own NuttX IRQ Number…

Later we’ll handle NuttX IRQ Number 45 in our code. And our Ox64 Serial Console will support Text Input!

How did we get the UART Driver for Ox64 BL808?

We copied the NuttX UART Driver from BL602 to BL808, since the UART Controllers are similar…

BL808 Reference Manual (Page 44)

BL808 Reference Manual (Page 44)

3 Initialise the Interrupts

How shall we get started with PLIC?

We walk through the steps to prepare the Platform-Level Interrupt Controller (PLIC) at startup…

  1. Disable all Interrupts

    (Because we’re about to configure them)

  2. Clear the Outstanding Interrupts

    (So we won’t get stuck at startup)

  3. Set the Interrupt Priority

    (To the Lowest Priority)

  4. Set the Interrupt Threshold

    (Allowing Interrupts to be fired later)

Disable Interrupts

3.1 Disable all Interrupts

We begin by disabling all Interrupts in PLIC.

Writing 0 to the Interrupt Enable Register (pic above) will disable all PLIC Interrupts: bl808_irq.c

// Init the Platform-Level Interrupt Controller
void up_irqinitialize(void) {

  // Disable Supervisor-Mode Interrupts (SIE Register)
  up_irq_save();

  // Attach the Common RISC-V Exception Handlers
  riscv_exception_attach();

  // Disable all External Interrupts
  // PLIC_ENABLE1 is 0xE000_2080
  // PLIC_ENABLE2 is 0xE000_2084
  // putreg32(V, A) writes 32-bit value V to address A
  putreg32(0x0, PLIC_ENABLE1);  // RISC-V IRQ 1  to 31
  putreg32(0x0, PLIC_ENABLE2);  // RISC-V IRQ 32 to 63

Hence at startup, all PLIC Interrupts are disabled until we enable them later (in PLIC).

(up_irq_save is defined here)

(putreg32 is defined here)

(PLIC_ENABLE and other PLIC Offsets)

(NuttX calls up_irqinitialize at startup)

Clear Interrupts

3.2 Clear the Interrupts

Next we Claim and Complete the Outstanding Interrupts, so they won’t bother us at startup (pic above): bl808_irq.c

  // Claim and Complete the Outstanding Interrupts
  // PLIC_CLAIM is 0xE020_1004
  // getreg32(A) reads a 32-bit value from address A
  uintptr_t val = getreg32(PLIC_CLAIM);
  putreg32(val, PLIC_CLAIM);

(getreg32 is defined here)

(More about Claim and Complete in a while)

Set Interrupt Priority

3.3 Set the Interrupt Priority

We initialise the Interrupt Priority of all Interrupts to 1 (pic above): bl808_irq.c

  // Set Priority for all External Interrupts to 1 (Lowest)
  // NR_IRQS is 83 (TODO: BL808 only supports 82 Peripheral Interrupts)
  // PLIC_PRIORITY is 0xE000_0000
  for (int id = 1; id <= NR_IRQS; id++) {
    putreg32(
      1,  // Value
      (uintptr_t)(PLIC_PRIORITY + 4 * id)  // Address
    );
  }

Why set Interrupt Priority to 1?

Set Interrupt Threshold

3.4 Set the Interrupt Threshold

Finally we set the Interrupt Threshold to 0 (pic above): bl808_irq.c

  // Set Interrupt Threshold to 0
  // (Permits all External Interrupts)
  // PLIC_THRESHOLD is 0xE020_1000
  putreg32(0, PLIC_THRESHOLD);

  // Enable Supervisor-Mode Interrupts (SIE Register)
  up_irq_enable();
}

(riscv_exception_attach is here)

(up_irq_enable is here)

Why set Interrupt Threshold to 0?

And we’re done initing the PLIC at startup!

Enable Interrupt

4 Enable the Interrupt

Our Platform-Level Interrupt Controller (PLIC) is all ready for action…

How will we enable Interrupts in PLIC?

Suppose we’re enabling RISC-V IRQ 20 for UART3 Interrupt.

All we need to do is to flip Bit 20 to 1 in the Interrupt Enable Register (pic above). Like so: bl808_irq.c

// Enable the NuttX IRQ specified by `irq`
// UART3 Interrupt is RISC-V IRQ 20
// Which is NuttX IRQ 45 (Offset by 25)
void up_enable_irq(int irq) {

  // Omitted: Enable Inter-CPU Interrupts (SIE Register)
  // Omitted: Enable Timer Interrupts (TIE Register)

  // If this is an External Interrupt...
  if (irq > RISCV_IRQ_EXT) {

    // Subtract 25 from NuttX IRQ to get the RISC-V IRQ
    int extirq = irq - RISCV_IRQ_EXT;

    // Set the Interrupt Enable Bit for `extirq` in PLIC
    // PLIC_ENABLE1 is 0xE000_2080
    // PLIC_ENABLE2 is 0xE000_2084
    if (0 <= extirq && extirq <= 63) {
      modifyreg32(
        PLIC_ENABLE1 + (4 * (extirq / 32)),  // Address
        0,  // Clear Bits
        1 << (extirq % 32)  // Set Bits
      );
    }
    else { PANIC(); }  // IRQ not supported (for now)
  }
}

(modifyreg32 is here)

And PLIC will happily accept RISC-V IRQ 20 whenever we press a key!

(On the Serial Console, pic above)

Who calls up_enable_irq?

At startup, NuttX calls bl808_attach to attach the UART Interrupt Handler…

// Attach UART Interrupt Handler
static int bl808_attach(struct uart_dev_s *dev) {
  ...
  // Enable Interrupt for UART3.
  // `irq` is NuttX IRQ 45
  up_enable_irq(priv->irq);

Which will call up_enable_irq to enable the UART3 Interrupt.

We’re halfway through our Grand Plan of PLIC Interrupts! (Steps 1, 2 and 3, pic below)

We pause a moment to talk about Harts…

Registers for Platform-Level Interrupt Controller

5 Hart 0, Supervisor Mode

The pic above: Why does it say “Hart 0, Supervisor Mode”?

“Hart” is a RISC-V CPU Core.

(“Hardware Thread”)

“Hart 0” refers to the (one and only) 64-bit RISC-V Core inside the BL808 SoC…

Inside the BL808 SoC

That runs our NuttX RTOS.

Does the Hart Number matter?

Most certainly! Inside the StarFive JH7110 SoC (for Star64 SBC), there are 5 Harts…

Inside the StarFive JH7110

NuttX boots on Hart 1. So the PLIC Settings will use Hart 1. (Not Hart 0)

And the PLIC Register Offsets are different for Hart 0 vs Hart 1. Thus the Hart Number really matters!

Why “Supervisor Mode”?

  1. RISC-V Machine Mode is the most powerful mode in our RISC-V SBC.

    OpenSBI Supervisor Binary Interface runs in Machine Mode.

    (It’s like BIOS for RISC-V)

  2. RISC-V Supervisor Mode is less powerful than Machine Mode.

    NuttX Kernel runs in Supervisor Mode.

    (Linux too!)

  3. RISC-V User Mode is the least powerful mode.

    NuttX Apps run in User Mode.

    (Same for Linux Apps)

PLIC has a different set of registers for Machine Mode vs Supervisor Mode.

That’s why we specify Supervisor Mode for the PLIC Registers.

What about the registers WITHOUT “Hart 0, Supervisor Mode”?

These are the Common PLIC Registers, shared across all Harts and RISC-V Modes.

Heading back to our (interrupted) story…

Handle Interrupt

6 Handle the Interrupt

What happens when we press a key on the Serial Console? (Pic above)

How will PLIC handle the UART Interrupt?

This is how we handle an Interrupt with the Platform-Level Interrupt Controller (PLIC)…

  1. Claim the Interrupt

    (To acknowledge the Interrupt)

  2. Dispatch the Interrupt

    (Call the Interrupt Handler)

  3. Complete the Interrupt

    (Tell PLIC we’re done)

  4. Optional: Inspect and reset the Pending Interrupts

    (In case we’re really curious)

Interrupt Claim Register

6.1 Claim the Interrupt

How will we know which RISC-V Interrupt has been fired?

That’s why we have the Interrupt Claim Register! (Pic above)

We read the Interrupt Claim Register to get the RISC-V IRQ Number that has been fired (20 for UART3): bl808_irq.c

// Dispatch the RISC-V Interrupt
void *riscv_dispatch_irq(uintptr_t vector, uintptr_t *regs) {

  // Compute the (Interim) NuttX IRQ Number
  // Based on the Interrupt Vector Number
  int irq = (vector >> RV_IRQ_MASK) | (vector & 0xf);

  // If this is an External Interrupt...
  if (RISCV_IRQ_EXT == irq) {

    // Read the RISC-V IRQ Number
    // From PLIC Claim Register
    // Which also Claims the Interrupt
    // PLIC_CLAIM is 0xE020_1004
    uintptr_t val = getreg32(PLIC_CLAIM);

    // Compute the Actual NuttX IRQ Number:
    // RISC-V IRQ Number + 25 (RISCV_IRQ_EXT)
    irq += val;
  }
  // For UART3: `val` is 20 and `irq` is 45
  // Up Next: Dispatch and Complete the Interrupt

What exactly are we “claiming”?

When we Claim an Interrupt (by reading the Interrupt Claim Register)…

We’re telling the PLIC: “Yes we acknowledge the Interrupt, but we’re not done yet!”

In a while we shall Complete the Interrupt. (To tell PLIC we’re done)

(riscv_dispatch_irq is called by the RISC-V Common Exception Handler)

6.2 Dispatch the Interrupt

We have Claimed the Interrupt. It’s time to do some work: bl808_irq.c

  // Omitted: Claim the Interrupt
  ...
  // Remember: `irq` is now the ACTUAL NuttX IRQ Number:
  // RISC-V IRQ Number + 25 (RISCV_IRQ_EXT)
  // For UART3: `irq` is 45

  // If the RISC-V IRQ Number is valid (non-zero)...
  if (RISCV_IRQ_EXT != irq) {

    // Call the Interrupt Handler
    regs = riscv_doirq(irq, regs);
  }
  // Up Next: Complete the Interrupt

For UART Interrupts: riscv_doirq will call uart_interrupt to handle the keypress.

(That’s because at startup, bl808_attach has registered uart_interrupt as the UART Interrupt Handler)

Interrupt Claim Register

6.3 Complete the Interrupt

To tell PLIC we’re done, we write the RISC-V IRQ Number (20) back to the Interrupt Claim Register.

(Yep the same one we read earlier! Pic above)

This will Complete the Interrupt, so PLIC can fire the next one: bl808_irq.c

  // Omitted: Claim and Dispatch the Interrupt
  ...
  // Remember: `irq` is now the ACTUAL NuttX IRQ Number:
  // RISC-V IRQ Number + 25 (RISCV_IRQ_EXT)
  // For UART3: `irq` is 45

  // If this is an External Interrupt (RISCV_IRQ_EXT = 25)...
  if (RISCV_IRQ_EXT <= irq) {

    // Compute the RISC-V IRQ Number (20 for UART3)
    // and Complete the Interrupt.
    // PLIC_CLAIM is 0xE020_1004
    putreg32(               // We write the...
      irq - RISCV_IRQ_EXT,  // RISC-V IRQ Number (RISCV_IRQ_EXT = 25)
      PLIC_CLAIM            // To PLIC Claim (Complete) Register
    );
  }

  // Return the Registers to the Caller
  return regs;
}

And that’s how we handle a PLIC Interrupt!

Interrupt Pending Register

6.4 Pending Interrupts

What’s with the Pending Interrupts?

Normally the Interrupt Claim Register is perfectly adequate for handling Interrupts.

But if we’re really curious: PLIC has an Interrupt Pending Register (pic above) that will tell us which Interrupts are awaiting Claiming or Completion: jh7110_irq.c

// Check the Pending Interrupts...
// Read PLIC_IP0: Interrupt Pending for interrupts 1 to 31
uintptr_t ip0 = getreg32(0xe0001000);

// If Bit 20 is set...
if (ip0 & (1 << 20)) {
  // Then UART3 Interrupt was fired (RISC-V IRQ 20)
  val = 20;
}

To tell PLIC we’re done: We clear the Individual Bits in the Interrupt Pending Register: jh7110_irq.c

// Clear the Pending Interrupts...
// Set PLIC_IP0: Interrupt Pending for interrupts 1 to 31
putreg32(0, 0xe0001000);

// TODO: Clear the Individual Bits instead of wiping out the Entire Register

Once again, we don’t need really need this. We’ll stash this as our Backup Plan in case things go wrong.

(Oh yes, things will go wrong in a while)

Set Interrupt Priority

7 Trouble with Interrupt Priority

I sense a twist in our story…

Earlier we initialised the Interrupt Priorities to 1 at startup (pic above): bl808_irq.c

// Init the Platform-Level Interrupt Controller
void up_irqinitialize(void) {
  ...
  // Set Priority for all External Interrupts to 1 (Lowest)
  // NR_IRQS is 83 (TODO: BL808 only supports 82 Peripheral Interrupts)
  // PLIC_PRIORITY is 0xE000_0000
  for (int id = 1; id <= NR_IRQS; id++) {
    putreg32(
      1,  // Value
      (uintptr_t)(PLIC_PRIORITY + 4 * id)  // Address
    );
  }

  // Dump the Interrupt Priorities
  infodumpbuffer("PLIC Interrupt Priority: After", 0xe0000004, 0x50 * 4);

When we boot NuttX on Ox64, something strange happens…

PLIC Interrupt Priority: After (0xe0000004):
0000  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................

(See the Complete Log)

Everything becomes zero! Why???

Yeah this is totally baffling! And no Interrupts get fired, because Interrupt Priority 0 is NOT valid.

Let’s set the Interrupt Priority specifically for RISC-V IRQ 20 (UART3 Interrupt): bl602_serial.c

// Test the setting of PLIC Interrupt Priority
// For RISC-V IRQ 20 only
void test_interrupt_priority(void) {
  // Read the values before setting Interrupt Priority
  uint32_t before50 = *(volatile uint32_t *) 0xe0000050UL;  // RISC-V IRQ 20
  uint32_t before54 = *(volatile uint32_t *) 0xe0000054UL;  // RISC-V IRQ 21

  // Set the Interrupt Priority
  // for 0x50 (IRQ 20) but NOT 0x54 (IRQ 21)
  *(volatile uint32_t *) 0xe0000050UL = 1;

  // Read the values after setting Interrupt Priority
  uint32_t after50 = *(volatile uint32_t *) 0xe0000050UL;  // RISC-V IRQ 20
  uint32_t after54 = *(volatile uint32_t *) 0xe0000054UL;  // RISC-V IRQ 21

  // Dump before and after values:
  _info("before50=%u, before54=%u, after50=%u, after54=%u\n",
    before50, before54, after50, after54);
}

Again we get odd results (pic below)…

before50=0, before54=0
after50=1,  after54=1

(See the Complete Log)

IRQ 20 is set correctly: “after50=1”

However IRQ 21 is also set! “after54=1”

Hmmm… Our writing seems to have leaked over to the next 32-bit word?

Yeah we see the Leaky Write again when we set the Interrupt Enable Register…

// Before setting Interrupt Enable: Everything is 0
PLIC Hart 0 S-Mode Interrupt Enable: Before (0xe0002080):
0000  00 00 00 00 00 00 00 00                          ........        

// Set Interrupt Enable for RISC-V IRQ 20 (Bit 20)
up_enable_irq: extirq=20, addr=0xe0002080, val=0x1048576

// After setting Interrupt Enable:
// Bit 20 is also set in the next word!
PLIC Hart 0 S-Mode Interrupt Enable: After (0xe0002080):
0000  00 00 10 00 00 00 10 00                          ........  

(See the Complete Log)

Interrupt Enable has leaked over from 0xE000 2080 to 0xE000 2084!

Thus we have an unexplained problem of Leaky Writes, affecting the Interrupt Priority and Interrupt Enable Registers.

Up Next: More worries…

Leaky Write for PLIC Interrupt Priority

8 More Trouble with Interrupt Claim

We talked earlier about Handling Interrupts…

Claim Interrupt

And how we fetch the RISC-V IRQ Number from the Interrupt Claim Register: bl808_irq.c

// Dispatch the RISC-V Interrupt
void *riscv_dispatch_irq(uintptr_t vector, uintptr_t *regs) {

  // Compute the (Interim) NuttX IRQ Number
  int irq = (vector >> RV_IRQ_MASK) | (vector & 0xf);

  // If this is an External Interrupt...
  if (RISCV_IRQ_EXT == irq) {

    // Read the RISC-V IRQ Number
    // From PLIC Claim Register
    // Which also Claims the Interrupt
    // PLIC_CLAIM is 0xE020_1004
    uintptr_t val = getreg32(PLIC_CLAIM);

What happens when we run this?

On Ox64 we see NuttX booting normally to the NuttX Shell…

NuttShell (NSH) NuttX-12.0.3
nsh>

(See the Complete Log)

When we press a key on the Serial Console (to trigger a UART Interrupt)…

riscv_dispatch_irq:
  claim=0

Our Interrupt Handler says that the Interrupt Claim Register is 0…

Which means we can’t read the RISC-V IRQ Number!

We activate our Backup Plan…

Pending Interrupts

9 Backup Plan

What’s our Backup Plan for Handling Interrupts?

We can get the RISC-V IRQ Number by reading the Interrupt Pending Register (pic above): jh7110_irq.c

// If Interrupt Claimed is 0...
if (val == 0) {
  // Check the Pending Interrupts...
  // Read PLIC_IP0: Interrupt Pending for interrupts 1 to 31
  uintptr_t ip0 = getreg32(0xe0001000);

  // If Bit 20 is set...
  if (ip0 & (1 << 20)) {
    // Then UART3 Interrupt was fired (RISC-V IRQ 20)
    val = 20;
  }
}

// Compute the Actual NuttX IRQ Number:
// RISC-V IRQ Number + 25 (RISCV_IRQ_EXT)
irq += val;

// Omitted: Call the Interrupt Handler
// and Complete the Interrupt

Which tells us the correct RISC-V IRQ Number for UART3 yay!

riscv_dispatch_irq:
  irq=45

(NuttX IRQ 45 means RISC-V IRQ 20)

Don’t forget to clear the Pending Interrupts: jh7110_irq.c

// Clear the Pending Interrupts
// TODO: Clear the Individual Bits instead of wiping out the Entire Register
putreg32(0, 0xe0001000);  // PLIC_IP0: Interrupt Pending for interrupts 1 to 31
putreg32(0, 0xe0001004);  // PLIC_IP1: Interrupt Pending for interrupts 32 to 63

// Dump the Pending Interrupts
infodumpbuffer("PLIC Interrupt Pending", 0xe0001000, 2 * 4);

// Yep works great, Pending Interrupts have been cleared...
// PLIC Interrupt Pending (0xe0001000):
// 0000  00 00 00 00 00 00 00 00                          ........        

Does it work for UART Input?

Since we’ve correctly identified the IRQ Number, riscv_dispatch_irq will (eventually) call bl808_receive to read the UART Input (pic below)…

bl808_receive: rxdata=-1
bl808_receive: rxdata=0x0

But the UART Input is empty! We need to troubleshoot our UART Driver some more.

Meanwhile we wrap up our story for today…

(See the Complete Log)

(Watch the Demo on YouTube)

(See the Build Outputs)

NuttX boots OK on Ox64 BL808! But UART Input is null

10 All Things Considered

Feels like we’re wading into murky greyish territory… Like Jaws meets Twilight Zone on the Beach?

Yeah we said this last time, and it’s happening now…

“If RISC-V ain’t RISC-V on SiFive vs T-Head: We’ll find out!”

The PLIC Code in this article was originally tested OK with…

Today we’re hitting 2 Strange Issues in the BL808 (C906) PLIC…

Which shouldn’t happen because PLIC is in the Official RISC-V Spec! So many questions…

  1. Any clue what’s causing this?

    Leaky Writes don’t seem to happen before enabling the MMU (Memory Management Unit)…

    // Before enabling Memory Mgmt Unit...
    bl808_mm_init: Test Interrupt Priority
    
    // No Leaky Writes!
    test_interrupt_priority:
      before50=0, before54=0
      after50=1,  after54=0
    
    // Leaky Writes after enabling Memory Mgmt Unit
    bl808_kernel_mppings: map I/O regions
    

    (See the Complete Log)

    So it might be a problem with our MMU Settings.

    (More about Memory Management Unit)

    (U-Boot Bootloader doesn’t have Leaky Writes)

  2. What if we configure the MMU differently?

    We moved the PLIC from Level 2 Page Tables up to Level 1…

    Same problem.

  3. Something special about the C906 MMU?

    According to the C906 User Manual (Page 53), the C906 MMU supports Extended Page Attributes.

    Is the MMU Caching / Buffering / Strong Ordering causing issues? What’s sysmap.h?

    (More about C906 Extended Page Attributes)

  4. What about the C906 PLIC?

    According to the Linux PLIC Driver…

    “The T-HEAD C9xx SoC implements a modified/custom T-HEAD PLIC specification which will mask current IRQ upon read to CLAIM register and will unmask the IRQ upon write to CLAIM register”

    Will this affect our Interrupt Claim?

    (More about C906 PLIC)

  5. Maybe the GCC Compiler didn’t generate the right code?

    We wrote RISC-V Assembly, disabling DCACHE / ICACHE and with SFENCE.

    Still the same.

  6. Perhaps our problem is Leaky Reads? Not Leaky Writes?

    Hmmm… Perhaps!

  7. So RISC-V ain’t RISC-V on SiFive vs T-Head?

    It feels… Very different? Compare the docs…

    SiFive U74 Manual

    T-Head C906 User Manual

    T-Head C906 Integration Manual (Chinese)

Can we rewrite our Sad Story with a Happier Conclusion? Find out in the next article…

Pine64 Ox64 64-bit RISC-V SBC (Sorry for my substandard soldering)

11 What’s Next

Today we talked about Interrupting Chicken, Oxen and Ox64 BL808 RISC-V SBC…

We have plenty to fix for NuttX on Ox64 BL808. Stay tuned for updates!

Many Thanks to my GitHub Sponsors (and the awesome NuttX Community) for supporting my work! This article wouldn’t have been possible without your support.

Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…

lupyuen.github.io/src/plic2.md

Ox64 Serial Console

12 Appendix: UART Driver for Ox64

How did we create the NuttX UART Driver for Ox64 BL808?

Today NuttX supports the 32-bit predecessor of BL808: Bouffalo Lab BL602.

When we compare these UARTs…

We discover that BL808 UART works the same way as BL602!

Thus we’ll simply copy the NuttX Driver for BL602 UART to Ox64.

Here’s the UART Driver ported to BL808: bl808_serial.c

What did we change?

We hardcoded the UART3 Base Address: bl808_uart.h

// UART3 Base Address
#define BL808_UART3_BASE   0x30002000ul
#define BL808_UART_BASE(n) (UNUSED(n), BL808_UART3_BASE)
// We call `UNUSED` to suppress warnings about unused `uart_idx`

Then we set the UART3 Interrupt Number: irq.h

// From BL808 Manual: UART3 Interrupt = (IRQ_NUM_BASE + 4)
// Where IRQ_NUM_BASE = 16
// So RISC-V IRQ = 20
// And NuttX IRQ = 45 (Offset by 25)
#define BL808_IRQ_UART3 45

And we modified the NuttX Start Code to call our new UART Driver: bl808_start.c

// At Startup, init the new UART Driver
void riscv_earlyserialinit(void) {
  bl808_earlyserialinit();
}

// Same here
void riscv_serialinit(void) {
  bl808_serialinit();
}

Finally we enabled and handled the UART Interrupt…

What about the UART Configuration?

We enable UART3 in the BL808 Peripheral Configuration: Kconfig

comment "BL808 Configuration Options"
menu "BL808 Peripheral Support"
config BL808_UART3
	bool "UART 3"
	default n
	select UART3_SERIALDRIVER
	select ARCH_HAVE_SERIAL_TERMIOS
endmenu

And we select UART3 as the Serial Console in the NuttX Build Configuration: nsh/defconfig

CONFIG_BL808_UART3=y
CONFIG_UART3_BAUD=2000000
CONFIG_UART3_SERIAL_CONSOLE=y

For now, the UART3 Baud Rate isn’t used. We assume that U-Boot Bootloader has already configured the UART3 Port: bl808_serial.c

// Configure the UART baud, bits, parity, etc.
static void bl808_uart_configure(const struct uart_config_s *config)
{
  // Assume that U-Boot Bootloader has already configured the UART
}

(Comments on UART Configuration)

Does it work?

After making these changes, the UART Driver works OK for Serial Console Input and Output! We need to enable Strongly-Ordered Access to I/O Memory, as explained here…

(See the Complete Log)

(Watch the Demo on YouTube)

NuttX boots OK on Ox64 BL808! But UART Input is null

13 Appendix: Build and Run NuttX

In this article, we ran a Work-In-Progress Version of Apache NuttX RTOS for Ox64, with PLIC partially working.

(Console Input is not yet fixed)

This is how we download and build NuttX for Ox64 BL808 SBC…

## Download the WIP NuttX Source Code
git clone \
  --branch ox64b \
  https://github.com/lupyuen2/wip-pinephone-nuttx \
  nuttx
git clone \
  --branch ox64b \
  https://github.com/lupyuen2/wip-pinephone-nuttx-apps \
  apps

## Build NuttX
cd nuttx
tools/configure.sh star64:nsh
make

## Export the NuttX Kernel
## to `nuttx.bin`
riscv64-unknown-elf-objcopy \
  -O binary \
  nuttx \
  nuttx.bin

## Dump the disassembly to nuttx.S
riscv64-unknown-elf-objdump \
  --syms --source --reloc --demangle --line-numbers --wide \
  --debugging \
  nuttx \
  >nuttx.S \
  2>&1

(Remember to install the Build Prerequisites and Toolchain)

(And enable Scheduler Info Output)

Then we build the Initial RAM Disk that contains NuttX Shell and NuttX Apps…

## Build the Apps Filesystem
make -j 8 export
pushd ../apps
./tools/mkimport.sh -z -x ../nuttx/nuttx-export-*.tar.gz
make -j 8 import
popd

## Generate the Initial RAM Disk `initrd`
## in ROMFS Filesystem Format
## from the Apps Filesystem `../apps/bin`
## and label it `NuttXBootVol`
genromfs \
  -f initrd \
  -d ../apps/bin \
  -V "NuttXBootVol"

## Prepare a Padding with 64 KB of zeroes
head -c 65536 /dev/zero >/tmp/nuttx.pad

## Append Padding and Initial RAM Disk to NuttX Kernel
cat nuttx.bin /tmp/nuttx.pad initrd \
  >Image

(See the Build Script)

(See the Build Outputs)

(Why the 64 KB Padding)

Next we prepare a Linux microSD for Ox64 as described in the previous article.

(Remember to flash OpenSBI and U-Boot Bootloader)

Then we do the Linux-To-NuttX Switcheroo: Overwrite the microSD Linux Image by the NuttX Kernel…

## Overwrite the Linux Image
## on Ox64 microSD
cp Image \
  "/Volumes/NO NAME/Image"
diskutil unmountDisk /dev/disk2

Insert the microSD into Ox64 and power up Ox64.

Ox64 boots OpenSBI, which starts U-Boot Bootloader, which starts NuttX Kernel and the NuttX Shell (NSH).

What happens when we press a key?

NuttX will respond to our keypress. (Because we configured the PLIC)

But the UART Input reads as null right now. (Pic above)

(See the NuttX Log)

(Watch the Demo on YouTube)

(See the Build Outputs)

Drawing the Platform-Level Interrupt Controller for Pine64 Ox64 64-bit RISC-V SBC (Bouffalo Lab BL808)