This article explains how to use YOLOv5 to detect objects in Android camera streams.

This article explains how to use YOLOv5 to detect objects in Android camera footage in two parts. This article explains how to set up the system and the code is explained in the following article.

YOLOv5 for Android uses ncnn, as described in this article How to run the ncnn version of YOLOv5 using Android was adapted from this repository.

Creating project
build.gralde(:app)
Adding files
AndroidManifest.xml
activity_main.xml
References

Creating project

Start Android Studio, select New Project > Empty Activity and go to Next.

The following description assumes that the project has been created in the /path/to/android_ncnn directory.

build.gralde(:app)

The entire build.gradle(:app) is shown below.

plugins {
    id 'com.android.application'
    id 'org.jetbrains.kotlin.android'
}

android {
    namespace 'com.example.android_ncnn'
    compileSdk 33

    defaultConfig {
        applicationId "com.example.android_ncnn"
        minSdk 24
        targetSdk 33
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

    externalNativeBuild {
        cmake {
            version "3.10.2"
            path file('src/main/jni/CMakeLists.txt')
        }
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    kotlinOptions {
        jvmTarget = '1.8'
    }
    buildFeatures {
        viewBinding true
    }
}

dependencies {

    implementation 'androidx.core:core-ktx:1.7.0'
    implementation 'androidx.appcompat:appcompat:1.6.0'
    implementation 'com.google.android.material:material:1.7.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.1.4'
    testImplementation 'junit:junit:4.13.2'
    androidTestImplementation 'androidx.test.ext:junit:1.1.5'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.5.1'

    def camerax_version = "1.1.0-beta01"
    implementation "androidx.camera:camera-core:${camerax_version}"
    implementation "androidx.camera:camera-camera2:${camerax_version}"
    implementation "androidx.camera:camera-lifecycle:${camerax_version}"
    implementation "androidx.camera:camera-video:${camerax_version}"

    implementation "androidx.camera:camera-view:${camerax_version}"
    implementation "androidx.camera:camera-extensions:${camerax_version}"
}
Code language: Gradle (gradle)

add lines after def camerax_version = "1.1.0-beta01"
add externalNativeBuild as above
also add viewBinding true as above
set minSdk to 24 or higher
compileSdk requires 33 or higher. Set the targetSdk to 33 in line with the compileSdk.
modify com.example.android_ncnn to the name you specified when you created your project.

Adding files

clone https://github.com/nihui/ncnn-android-yolov5 and copy files needed.

mkdir /path/to/android_ncnn/app/src/main/jni
mkdir /path/to/android_ncnn/app/src/main/assets
mkdir -p /path/to/android_ncnn/app/src/main/java/com/tencent/yolov5ncnn
git clone https://github.com/nihui/ncnn-android-yolov5
cp ncnn-android-yolov5/app/src/main/jni/{CMakeLists.txt,yolov5ncnn_jni.cpp} /path/to/android_ncnn/app/src/main/jni
cp -r ncnn-android-yolov5/app/src/main/assets /path/to/android_ncnn/app/src/main
cp ncnn-android-yolov5/app/src/main/java/com/tencent/yolov5ncnn/YoloV5Ncnn.java /path/to/android_ncnn/app/src/main/java/com/tencent/yolov5ncnn/
Code language: Bash (bash)

Adding ncnn library for Android

wget https://github.com/Tencent/ncnn/releases/download/20221128/ncnn-20221128-android-vulkan.zi
unzip ncnn-20221128-android-vulkan.zip -d /path/to/android_ncnn/app/src/main/jni
Code language: Bash (bash)

Modify CMakeLists.txt

-set(ncnn_DIR ${CMAKE_SOURCE_DIR}/ncnn-20201218-android-vulkan/${ANDROID_ABI}/lib/cmake/ncnn)
+set(ncnn_DIR ${CMAKE_SOURCE_DIR}/ncnn-20221128-android-vulkan/${ANDROID_ABI}/lib/cmake/ncnn)
Code language: Diff (diff)

Right click com/tencent/yolov5ncnn/YoloV5Ncnn.java and select Converto Java File to Kotlin File to convert to Kotlink.

Modify Detect function in YoloV5Ncnn.kt as follows: (Modify from Array<Obj?>? to Array<Obj>?)

external fun Detect(bitmap: Bitmap?, use_gpu: Boolean): Array<Obj>?
Code language: Kotlin (kotlin)

AndroidManifest.xml

Add the following to the top of the application tag in AndroidManifest.xml

<uses-feature android:name="android.hardware.camera.any" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"
   android:maxSdkVersion="28" />
Code language: HTML, XML (xml)

activity_main.xml

Open activity_main.xml

Right-click on the screen below and select Go to XML to edit the XML directly.

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <ImageView
        android:id="@+id/imageView"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:layout_weight="1" />

    <androidx.constraintlayout.widget.Guideline
        android:id="@+id/vertical_centerline"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:orientation="vertical"
        app:layout_constraintGuide_percent=".50" />

</androidx.constraintlayout.widget.ConstraintLayout>
Code language: HTML, XML (xml)