NVME is not functional #23
Comments
|
Others have reported problems with Western Digital NVMe's as well: #7 So far the Samsung NVMe are the only known working devices with this build. |
|
@inindev Is the problem in kernel or u-boot? |
|
Do you have a serial console to test u-boot? As I recall the previous report did not. I would like to see the u-boot output of this: |
|
@inindev Yes, I'll test and report this evening |
|
Image was build from master branch (commit 1c61cfb) |
|
|
@inindev boot of debian 11 from friendlyarm where NVME function properly: |
|
Have you tried with |
|
@ajayramaswamy Yes, I've tried - no success |
|
I am currently running u-boot from https://github.com/Kwiboo/u-boot-rockchip/commits/rk3568-2023.10 at commit |
|
Also it is interesting that booting kernel from friendlyarm (I've build it from their sources) using stock u-boot (from @inindev) results in partial working hardware:
It seems that friendlyarm's u-boot do a lot of work for hardware initialization. But is it old (2017) and intended mostly for android (a lot of crap android features like ten emmc partitions, etc) |
|
@ajayramaswamy How to configure, build and install this version of u-boot for R5S? Do any additional patches required? |
|
#!/bin/bash
#
cd ~/scratch/kwiboo/u-boot-rockchip/ || exit 99
git clean -fdx
#git switch rk3568-2023.07
export CROSS_COMPILE=aarch64-linux-gnu-
export BL31=~/scratch/kwiboo/rkbin-2/bin/rk35/rk3568_bl31_v1.43.elf
export
ROCKCHIP_TPL=~/scratch/kwiboo/rkbin-2/bin/rk35/rk3568_ddr_1560MHz_v1.18.bin
#git am ../patches/0*.patch
make -j"$(nproc)" distclean
make -j"$(nproc)" nanopi-r5s-rk3568_defconfig
make -j"$(nproc)" olddefconfig
make -j"$(nproc)" savedefconfig
make -j"$(nproc)"
This is the shell script I am using to build on my fedora x86_64 laptop
…On Fri, 22 Sept 2023 at 18:28, Vasily Evseenko ***@***.***> wrote:
@ajayramaswamy <https://github.com/ajayramaswamy> How to configure, build
and install this version of u-boot for R5S? Do any additional patches
required?
—
Reply to this email directly, view it on GitHub
<#23 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAPZLF65OXQ4ZR22BQBYOO3X3WDPRANCNFSM6AAAAAA5BNJ2VY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Thanks & best regards
Ajay
|
|
@ajayramaswamy I've tried following branches: rk3568-2023.10, rk3568-2023.07.02 and tag rk3568-2023.07. No success. Behavior still the same - No NVMe detect on boot and device freeze after pci rescan. Also I've tried to enable pcie and nvme in u-boot config. Also no success. What device tree do you use for fedora kernel and what kernel version is? |
|
@inindev Do you use debian 12 kernel 6.1 or build custom 6.4? I ask this because dtb is from 6.4 and 6.1 doesn't have dtb files for R5S |
|
I've tried linux-6.5.0 from debian-sid. Also no success. It seems the problem is in kernel (or dtb) because u-boot see NVMe drive and can read a partition table. |
The issue is not u-boot, we see it is properly being detected. ☝️ The issue looks to be a timeout with linux: I have increased the timeout in this dtb, can you replace the one in |
|
I've tried and behavior was the same - no nvme device during boot and stuck device after pci rescan
|
Also note that Jonas Karlman has builds for his repositories here: https://github.com/Kwiboo/u-boot-build/tree/main Here is the list of targets: https://github.com/Kwiboo/u-boot-build/actions And the outputs from the most recent u-boot rk3568-2023.07.02 for RK356x: https://github.com/Kwiboo/u-boot-build/actions/runs/6100093756 |
I have ordered a WD NVMe to see if I can reproduce this issue locally. |
|
I've tried his u-boot builds and no success for nvme support in subsequent linux kernel boot (see my replies to @ajayramaswamy above)
|
|
My drive is wd red sn700 2TB
|
As I noted above, the issue is with linux, not with u-boot. The reference to the build artifacts were for anyone reading this to know that local u-boot builds were unnecessary. With some luck, maybe multiple WD NVMEs have problems and I might be able to figure out why. |
|
My WD Blue SN570 came in today and failed in the same way you report. Placing These are the steps I followed: Note: replace /dev/sdX with your MMC device name, then boot using this MMC Once booted, edit the kernel command line: |
|
To boot to Western Digital NVMe: While booted from MMC: Remove the MMC boot partition so it boots to NVMe: After reboot: Note: You can also write u-boot to the internal eMMC which is: |
|
@inindev Unfortunately this doesn't help with WD Red: linux from frienlyarm: It is interesting that PCI bridge has different address |
Any idea if it expected that it identifies itself as a WD Black under frienlyarm?
|
|
It seems that only differs by firmware. WD Red designed for write-intensive tasks like CCTV cameras or NAS
|
|
I am afraid I probably will not be much help in working out NVMe issues with PCI in the debian kernel. Looking around at others experiencing such issues, using |
|
@inindev I think the root of problem is invalid PCIe bridge setup, because device is not visible by |
|
One difference is the Note: I had to remove the |
|
No change in case of NVMe, but with
|
|
I am unsure if this is even a valid node, but another difference is the https://drive.google.com/file/d/1akXUibWhxobbKVMGOJMBi3WI2LhvvalB/view?usp=sharing Here are the additions at this point: |
|
NVMe still not visible. Also no changes in dmesg compared to previous verison |
|
Comparing the friendlyarm device tree to the upstream device tree I noticed a large difference between the https://drive.google.com/file/d/1XyMA-GEzNKGhbSCLRw7tOEGv8c93E7XM/view?usp=sharing |
|
NVMe still not visible. Also no changes in dmesg compared to previous verison. After pci rescan nvme detected and hangs as before. |
|
Hi, I just wanted to thank for your work and report that, Crucial P2 boots from NVMe. 
|
|
Just want to report that SK Hynix Gold P31 2TB also works fine. |
|
not all NVMe devices appear to work with the nanopi r5s this topic lists several known working options known working
known not working
|
|
Just noting the following here in case it helps (sorry if it's bad form to dig this up) dmesg -l 0,1,2,3 returns pci 0002:01:00.0: calc_l12_pwron: Invalid T_PwrOn scale: 3 lspci returns (from a working OS) 0002:01:00.0 Non-Volatile memory controller: Sandisk Corp HD Blue SN570 NVMe SSD 1TB The problem is not present on FriendlyWrt (or OpenWRT 23.05) or the Ubuntu builds based on Focal, both of which initialise the NVMe successfully. |
|
can you try this version: |
|
Writing to SD... |
|
Bah, same problem :/ |
Can you post a listing of the /boot directory? |
|
Can confirm pcie_aspm=off works Will just get you that listing (does it make any difference now pcie_aspm=off is there or should I remove that and reboot first?) |
|
I wanted to be sure it is using the correct .dtb (the flag wont matter). |
|
Shows So as you posted above |
|
How are Ubuntu and the WRTs working? Different DTBs? |
|
Would it help if I flashed Ubuntu back on it and got the DTB from that? EDIT: Realised I can edit :) I have Ubuntu still on the MMC, so can just boot from that if it helps |
I have the r5s booting with both Samsung and Crucial NVMe using the upstream debian kernel, v 6.1.0-23 in this case: stock debian: OpenWRT is not available in a release, just overnight builds which are based off of the 6.6.43 kernel: nightly OpenWRT: My guess is that your Sandisk NVMe works better with a more recent kernel? To test this theory, you could try a later debian kernel: https://packages.debian.org/sid/kernel/linux-image-arm64 |
|
So just to clarify, boot with your latest image from SD and wget that sid kernel? (I booted from the MMC to Ubuntu just to see, but there is nothing in /boot) |
Surely FriendlyWrt and Ubuntu Focal aren't using such a recent kernel though? |
|
Yes, my assumption is that you are booting from MMC and trying to access your installed Sandisk NVMe. Then install this MMC and boot from it. I am unsure why you are getting this: |
|
I am booting your Debian image from an SD card, but have Ubuntu (FriendlyCore) on the internal eMMC I just installed that 6.9 kernel and again, the NVMe was initialised, but when I removed the pcie_aspm=off from the boot flags, it was no longer initialised, even with the 6.9 kernel. Yeah, Google doesn't have much regarding that T_PwrOn error. Again strange how those other two OSes can initialise it OK EDIT: just FYI, my NVMe is actually WD Blue SN570 500GB with firmware 234110WD, shown by cat /sys/class/nvme/nvme0/model and cat /sys/class/nvme/nvme0/firmware_rev, which I see is the same as the SSD you ordered in September :) |
|
I see my post: inindev commented on Sep 24, 2023 I can test the WD Blue again but I wont get a chance to do so for a few days. |
|
Fair enough. I'm curious whether you have the same dmesg output with your WD Blue. I ran a SMART check with nvme-cli which said the drive was OK. In the mean time, it works; albeit without power management on the PCIe bus. Thanks for this project too, by the way. Great to have vanilla Debian on this device! |
|
I found my WD Blue NVMe and am now seeing the same thing you are with the stock debian I also see that u-boot is initializing this device just fine: It looks timing related because a rescan is able to detect it: followed by a failure a short time later: |
|
I did find this procedure successfully initializes the WD Blue NVMe with the Though it is not a solution, it does point to an initialization issue during boot. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
After boot no NVME devices found. If I trigger pci rescan via
echo 1 > /sys/bus/pci/rescanNVMe device is found but non-functional:NVME device is WD Red SN700
The text was updated successfully, but these errors were encountered: