SoCRockchip RK3399
CPUCortex A53 4Core+A72 2.0GHz 2Core
GPUMali-T860 MP4 Quad-Core
Memory2GB Dual-Channel DDR3

スペック

  • CPU:Cortex-A53 Quad core 1.5GHz + Cortex-A72 Dual core 2.0GHz
  • Architecture:Armv8
  • cpuinfo
    firefly@firefly:~$ cat /proc/cpuinfo
    processor       : 0
    BogoMIPS        : 48.00
    Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
    CPU implementer : 0x41
    CPU architecture: 8
    CPU variant     : 0x0
    CPU part        : 0xd03
    CPU revision    : 4
    
    processor       : 1
    BogoMIPS        : 48.00
    Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
    CPU implementer : 0x41
    CPU architecture: 8
    CPU variant     : 0x0
    CPU part        : 0xd03
    CPU revision    : 4
    
    processor       : 2
    BogoMIPS        : 48.00
    Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
    CPU implementer : 0x41
    CPU architecture: 8
    CPU variant     : 0x0
    CPU part        : 0xd03
    CPU revision    : 4
    
    processor       : 3
    BogoMIPS        : 48.00
    Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
    CPU implementer : 0x41
    CPU architecture: 8
    CPU variant     : 0x0
    CPU part        : 0xd03
    CPU revision    : 4
    
    processor       : 4
    BogoMIPS        : 48.00
    Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
    CPU implementer : 0x41
    CPU architecture: 8
    CPU variant     : 0x0
    CPU part        : 0xd08
    CPU revision    : 2
    
    processor       : 5
    BogoMIPS        : 48.00
    Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
    CPU implementer : 0x41
    CPU architecture: 8
    CPU variant     : 0x0
    CPU part        : 0xd08
    CPU revision    : 2
    
    Serial          : 0000000000000000
  • auxv
    $ LD_SHOW_AUXV=1 ls
    AT_SYSINFO_EHDR:      0x7f7937d000
    AT_HWCAP:             ff
    AT_PAGESZ:            4096
    AT_CLKTCK:            100
    AT_PHDR:              0x5593973040
    AT_PHENT:             56
    AT_PHNUM:             9
    AT_BASE:              0x7f7934d000
    AT_FLAGS:             0x0
    AT_ENTRY:             0x5593978940
    AT_UID:               1000
    AT_EUID:              1000
    AT_GID:               1000
    AT_EGID:              1000
    AT_SECURE:            0
    AT_RANDOM:            0x7fe6ca4308
    AT_EXECFN:            /usr/bin/ls
    AT_PLATFORM:          aarch64
     
    0000000000000021 0000007fac7b8000
    0000000000000010 00000000000000ff
    0000000000000006 0000000000001000
    0000000000000011 0000000000000064
    0000000000000003 00000055929c8040
    0000000000000004 0000000000000038
    0000000000000005 0000000000000009
    0000000000000007 0000007fac788000
    0000000000000008 0000000000000000
    0000000000000009 00000055929ca0a4
    000000000000000b 00000000000003e8
    000000000000000c 00000000000003e8
    000000000000000d 00000000000003e8
    000000000000000e 00000000000003e8
    0000000000000017 0000000000000000
    0000000000000019 0000007fd967dce8
    000000000000001f 0000007fd967efeb
    000000000000000f 0000007fd967dcf8
    0000000000000000 0000000000000000
  • lscpu
    $ lscpu
    Architecture:        aarch64
    CPU op-mode(s):      32-bit, 64-bit
    Byte Order:          Little Endian
    CPU(s):              6
    On-line CPU(s) list: 0-5
    Thread(s) per core:  1
    Core(s) per socket:  3
    Socket(s):           2
    Vendor ID:           ARM
    Model:               4
    Model name:          Cortex-A53
    Stepping:            r0p4
    CPU max MHz:         1800.0000
    CPU min MHz:         408.0000
    BogoMIPS:            48.00
    Flags:               fp asimd evtstrm aes pmull sha1 sha2 crc32
  • cpufreq
    $ sudo cat /sys/bus/cpu/devices/cpu*/cpufreq/cpuinfo_max_freq
    1416000
    1416000
    1416000
    1416000
    1800000
    1800000
  • kernel
    $ uname -a
    Linux firefly 4.4.194+ #90 SMP Sat May 28 13:05:54 CST 2022 aarch64 aarch64 aarch64 GNU/Linux
  • OS
    $ lsb_release -a
    No LSB modules are available.
    Distributor ID: Ubuntu
    Description:    Ubuntu 20.04.4 LTS
    Release:        20.04
    Codename:       focal
  • gcc
    $ gcc --version
    gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
    Copyright (C) 2019 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    
  • clinfo
    $ clinfo
    arm_release_ver of this libmali is 'r18p0-01rel0', rk_so_ver is '4'.Number of platforms                               1
      Platform Name                                   ARM Platform
      Platform Vendor                                 ARM
      Platform Version                                OpenCL 1.2 v1.r18p0-01rel0.ddd394a39c9049aa64d45a44032b5335
      Platform Profile                                FULL_PROFILE
      Platform Extensions                             cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_3d_image_writes cl_khr_fp64 cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_fp16 cl_khr_icd cl_khr_egl_image cl_khr_image2d_from_buffer cl_arm_core_id cl_arm_printf cl_arm_thread_limit_hint cl_arm_non_uniform_work_group_size cl_arm_import_memory
      Platform Extensions function suffix             ARM
    
      Platform Name                                   ARM Platform
    Number of devices                                 1
      Device Name                                     Mali-T860
      Device Vendor                                   ARM
      Device Vendor ID                                0x8602000
      Device Version                                  OpenCL 1.2 v1.r18p0-01rel0.ddd394a39c9049aa64d45a44032b5335
      Driver Version                                  1.2
      Device OpenCL C Version                         OpenCL C 1.2 v1.r18p0-01rel0.ddd394a39c9049aa64d45a44032b5335
      Device Type                                     GPU
      Device Profile                                  FULL_PROFILE
      Device Available                                Yes
      Compiler Available                              Yes
      Linker Available                                Yes
      Max compute units                               4
      Max clock frequency                             5MHz
      Device Partition                                (core)
        Max number of sub-devices                     0
        Supported partition types                     None
        Supported affinity domains                    (n/a)
      Max work item dimensions                        3
      Max work item sizes                             256x256x256
      Max work group size                             256
      Preferred work group size multiple              4
      Preferred / native vector sizes                 
        char                                                16 / 16      
        short                                                8 / 8       
        int                                                  4 / 4       
        long                                                 2 / 2       
        half                                                 8 / 8        (cl_khr_fp16)
        float                                                4 / 4       
        double                                               2 / 2        (cl_khr_fp64)
      Half-precision Floating-point support           (cl_khr_fp16)
        Denormals                                     Yes
        Infinity and NANs                             Yes
        Round to nearest                              Yes
        Round to zero                                 Yes
        Round to infinity                             Yes
        IEEE754-2008 fused multiply-add               Yes
        Support is emulated in software               No
      Single-precision Floating-point support         (core)
        Denormals                                     Yes
        Infinity and NANs                             Yes
        Round to nearest                              Yes
        Round to zero                                 Yes
        Round to infinity                             Yes
        IEEE754-2008 fused multiply-add               Yes
        Support is emulated in software               No
        Correctly-rounded divide and sqrt operations  No
      Double-precision Floating-point support         (cl_khr_fp64)
        Denormals                                     Yes
        Infinity and NANs                             Yes
        Round to nearest                              Yes
        Round to zero                                 Yes
        Round to infinity                             Yes
        IEEE754-2008 fused multiply-add               Yes
        Support is emulated in software               No
      Address bits                                    64, Little-Endian
      Global memory size                              2050433024 (1.91GiB)
      Error Correction support                        No
      Max memory allocation                           512608256 (488.9MiB)
      Unified memory for Host and Device              Yes
      Minimum alignment for any data type             128 bytes
      Alignment of base address                       1024 bits (128 bytes)
      Global Memory cache type                        Read/Write
      Global Memory cache size                        262144 (256KiB)
      Global Memory cache line size                   64 bytes
      Image support                                   Yes
        Max number of samplers per kernel             16
        Max size for 1D images from buffer            65536 pixels
        Max 1D or 2D image array size                 2048 images
        Base address alignment for 2D image buffers   32 bytes
        Pitch alignment for 2D image buffers          16 pixels
        Max 2D image size                             65536x65536 pixels
        Max 3D image size                             65536x65536x65536 pixels
        Max number of read image args                 128
        Max number of write image args                8
      Local memory type                               Global
      Local memory size                               32768 (32KiB)
      Max number of constant args                     8
      Max constant buffer size                        65536 (64KiB)
      Max size of kernel argument                     1024
      Queue properties                                
        Out-of-order execution                        Yes
        Profiling                                     Yes
      Prefer user sync for interop                    No
      Profiling timer resolution                      1000ns
      Execution capabilities                          
        Run OpenCL kernels                            Yes
        Run native kernels                            No
      printf() buffer size                            1048576 (1024KiB)
      Built-in kernels                                (n/a)
      Device Extensions                               cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_3d_image_writes cl_khr_fp64 cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_fp16 cl_khr_icd cl_khr_egl_image cl_khr_image2d_from_buffer cl_arm_core_id cl_arm_printf cl_arm_thread_limit_hint cl_arm_non_uniform_work_group_size cl_arm_import_memory
    
    NULL platform behavior
      clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  ARM Platform
      clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   Success [ARM]
      clCreateContext(NULL, ...) [default]            Success [ARM]
      clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT)  Success (1)
        Platform Name                                 ARM Platform
        Device Name                                   Mali-T860
      clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No devices found in platform
      clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  Success (1)
        Platform Name                                 ARM Platform
        Device Name                                   Mali-T860
      clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No devices found in platform
      clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No devices found in platform
      clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  Success (1)
        Platform Name                                 ARM Platform
        Device Name                                   Mali-T860
    
    ICD loader properties
      ICD loader Name                                 OpenCL ICD Loader
      ICD loader Vendor                               OCL Icd free software
      ICD loader Version                              2.2.11
      ICD loader Profile                              OpenCL 2.1
  • cmake
    $ cmake --version
    cmake version 3.16.3
    
    CMake suite maintained and supported by Kitware (kitware.com/cmake).
  • ccache
    $ ccache --version
    ccache version 3.7.7
    
    Copyright (C) 2002-2007 Andrew Tridgell
    Copyright (C) 2009-2020 Joel Rosdahl
    
    This program is free software; you can redistribute it and/or modify it under
    the terms of the GNU General Public License as published by the Free Software
    Foundation; either version 3 of the License, or (at your option) any later
    version.
  • UART Debug console
  • Firefly RK3399

その他情報

セットアップ

  • まずはフラッシュストレージの消去から
  • Minimal イメージをインストールしたので、ユーザがrootしかいない
    groupadd -g 1000 firefly
    useradd -g firefly --uid 1000 firefly
    mkdir -p /home/firefly
    vim /etc/passwd
    # rewrite shell from /bin/sh to /bin/bash
    visudo
    # rewrite admin to wheel
    addgroup wheel
    usermod -aG wheel firefly
    nmcli dev wifi connect xxxxxxxx password xxxxxxxxx
  • 当然xxxxは適切なものに置き換える
  • ここからはSSH接続で
    sudo chown -R /home/firefly
    sudo mkdir /mnt/sdcard
    blkid
    sudo vim /etc/fstab
    sudo dpkg-reconfigure tzdata
    sudo dpkg -i rclone-*
    sudo apt update
    sudo apt install tmux git ccache cmake build-essential openjdk-17-jdk-headless libx11-xcb-dev python3-dev python3-pip cron opencl-headers ocl-icd-opencl-dev strace
    sudo apt upgrade
    pip3 install numpy

秘密鍵/公開鍵

  • SSH認証用に対称鍵を生成する
    $ ssh-keygen -t ecdsa
    $ ssh-keygen -f hoge
    $ cat hoge.pub >> .ssh/authorized_keys
  • .ssh/id_ecdsa.pub をGithubに、hoge の中身をJenkinsに登録する
  • githubで確認
    $ ssh -T git@github.com

OpenCL対応

  • OpenCL対応
    • 発売と同時に焼きこまれたイメージには、OpenCL対応のドライバが入ってない
    • で、公式っぽいアカウントから出てる身元不明のドライバを入れてみる(自己責任でね!良い子は真似しちゃダメだよ!)
      $ git clone git@github.com:rockchip-linux/libmali.git
      $ cd libmali
      $ cmake .
      $ make
      $ sudo make instal 
    • 全部インストールするのは怖かったので、libOpenCL.soだけ差し替えてみた(自己責任でね!良い子は真似しちゃダメだよ!)
      $ git clone git@github.com:rockchip-linux/libmali.git
      $ sudo rm /usr/lib/aarch64-linux-gnu/libOpenCL.so
      $ sudo cp -p libmali/lib/aarch64-linux-gnu/libmali-midgard-t86x-r13p0-wayland.so /usr/lib/aarch64-linux-gnu/
      $ cd /usr/lib/aarch64-linux-gnu/
      $ sudo ln -s libmali-midgard-t86x-r13p0.so libOpenCL.so
    • soだけ差し替えればOpenCLは動く
    • ただし、手元でビルドするためには、openclのヘッダやpcファイルが事実上必須
      $ sudo apt install opencl-headers ocl-icd-opencl-dev

Firefly の libmali を使っても、ERROR: The DDK is not compatible とかエラーメッセージが出た場合の対処

  • どうやら、インストールしただけでは/dev/mali0のパーミッションの問題でGPUにアクセスできなくなるらしい。
  • もともと使えなかった
    • →libmaliを差し替え
    • →使える
    • →再起動
    • →使えなくなる
  • という流れだった。
  • 使えたときは、OpenCVのテストプログラムの表示が
    Parallel framework: pthreads
    CPU features: neon fp16
    [ INFO:0] Initialize OpenCL runtime...
    OpenCL Platforms: 
        ARM Platform
            iGPU: Mali-T860 (OpenCL 1.2 v1.r13p0-00rel0-git(a4271c9).04dadb82d3612c978f88c00109101694)
    Current OpenCL device: 
        Type = iGPU
        Name = Mali-T860
        Version = OpenCL 1.2 v1.r13p0-00rel0-git(a4271c9).04dadb82d3612c978f88c00109101694
        Driver version = 1.2
        Address bits = 64
        Compute units = 4
        Max work group size = 256
        Local memory size = 32 KB
        Max memory allocation size = 494 MB 34 KB
        Double support = Yes
        Host unified memory = Yes
        Device extensions:
            cl_khr_global_int32_base_atomics
            cl_khr_global_int32_extended_atomics
            cl_khr_local_int32_base_atomics
            cl_khr_local_int32_extended_atomics
            cl_khr_byte_addressable_store
            cl_khr_3d_image_writes
            cl_khr_fp64
            cl_khr_int64_base_atomics
            cl_khr_int64_extended_atomics
            cl_khr_fp16
            cl_khr_gl_sharing
            cl_khr_icd
            cl_khr_egl_event
            cl_khr_egl_image
            cl_khr_image2d_from_buffer
            cl_arm_core_id
            cl_arm_printf
            cl_arm_thread_limit_hint
            cl_arm_non_uniform_work_group_size
            cl_arm_import_memory
        Has AMD Blas = No
        Has AMD Fft = No
        Preferred vector width char = 16
        Preferred vector width short = 8
        Preferred vector width int = 4
        Preferred vector width long = 2
        Preferred vector width float = 4
        Preferred vector width double = 2
    だったのに、
    Build type: release
    Parallel framework: pthreads
    CPU features: neon fp16
    ERROR: The DDK is not compatible with any of the Mali GPUs on the system.
    The DDK was built for 0x860 r2p0 status range [0..15], but none of the GPUs matched:
    [ INFO:0] Initialize OpenCL runtime...
    OpenCL is disabled
  • という具合に変わってしまった。
  • デレがツンになってしまった。
  • で、調べると、どうも/dev/mali0にアクセスしたのちに諦めているっぽい。
    $ strace ./bin/opencv_test_imgproc --gtest_filter=slkfjdslfkjdslfsd > stracelog.txt 2>&1
    $ tail -20 stracelog.txt
    openat(AT_FDCWD, "./.mali_config", O_RDONLY) = -1 ENOENT (No such file or directory)
    brk(0x1031000)                          = 0x1031000
    openat(AT_FDCWD, "/dev/mali0", O_RDWR|O_NONBLOCK|O_CLOEXEC) = -1 EACCES (Permission denied)
    write(2, "ERROR: The DDK is not compatible"..., 159ERROR: The DDK is not compatible with any of the Mali GPUs on the system.
    The DDK was built for 0x860 r2p0 status range [0..15], but none of the GPUs matched:
    ) = 159
    openat(AT_FDCWD, "/dev/mali0", O_RDWR|O_NONBLOCK|O_CLOEXEC) = -1 EACCES (Permission denied)
    munmap(0x7fa2385000, 200704)            = 0
    write(1, "[ INFO:0] Initialize OpenCL runt"..., 58[ INFO:0] Initialize OpenCL runtime...
    OpenCL is disabled
    ) = 58
    ioctl(1, TCGETS, 0x7fca4e7648)          = -1 ENOTTY (Inappropriate ioctl for device)
    write(1, "Note: Google Test filter = slkfj"..., 93Note: Google Test filter = slkfjdslfkjdslfsd
    [==========] Running 0 tests from 0 test cases.
    ) = 93
    write(1, "[==========] 0 tests from 0 test"..., 79[==========] 0 tests from 0 test cases ran. (2 ms total)
    [  PASSED  ] 0 tests.
    ) = 79
    exit_group(0)                           = ?
    +++ exited with 0 +++
  • で、調べたら、インストール用のスクリプトを公開している人がいた*4
  • 要は /etc/udev/rules.d/99-mali.rules を作り、その中に
    KERNEL=="mali0",SUBSYSTEM=="misc",MODE="0777"
  • と書いて再起動すれば良いらしい
    $ sudo vim /etc/udev/rules.d/99-mali.rules
    $ sudo reboot
  • 他のライブラリの配置も追々追従しよう

Arm


*1  Serial Debug — Firefly Wiki, 2019-07-08閲覧
*2  Firefly | Focus on establish an open source platform based on cutting-edge technology, 2017-03-21閲覧
*3  Firefly-RK3399:Six-Core 64-bit High-Performance Platform by Firefly team ― Kickstarter, 2016-12-05募集開始, 2017-01-10募集終了, 60,480$ 230backers, 2017-03-21閲覧
*4  Apokalypzx/gru-libgl: PKGBUILD for libgl userspace binaries for the rockchip RK3399 SoC containing the Mali Midgard T860 gpu., 2018-02-23最終コミット, 2018-04-17閲覧

トップ   編集 凍結 差分 バックアップ 添付 複製 名前変更 リロード   新規 一覧 単語検索 最終更新   ヘルプ   最終更新のRSS
Last-modified: 2022-07-14 (木) 08:58:45 (85d)