Android App with OpenCV and C++, Part 3

1. Add Camera

a. Add permission line  in file "AndroidManifest.xml": 

<uses-permission android:name="android.permission.CAMERA" />

b. Subclass SurfaceView: MyView:

 public class MyView extends SurfaceView{
     Bitmap bmp2Display;
     public MyView(Context context, AttributeSet attrs) {
           super(context, attrs);
     }
     Paint paint = new Paint();
     @Override
     protected void onDraw(Canvas canvas) {
         super.onDraw(canvas);
         canvas.drawText("time = "+System.currentTimeMillis(), 50, 100, paint);
     }
}

c. Add an instance of MyView in "activity_mail.xml":

<org.ski.helloworld.MyView
      android:id="@+id/myView"
      android:layout_width="wrap_content"
      android:layout_height="wrap_content"
      android:layout_alignParentLeft="true"
      android:layout_alignParentTop="true" />

d. Add "implements SurfaceHolder.Callback" in MainActivity, and added unimplemented functions as hinted

public class MainActivity extends Activity implements SurfaceHolder.Callback{
    //...
    // ...
}

e. Add MyView, Camera, its callback and image size:

MyView myView; 
int width = 960, height=720;
Camera mCamera;
Camera.PreviewCallback cbWithBuffer;

f. Define callback function:

 Camera.PreviewCallback cabWithBuffer = new Camera.PreviewCallback() {
    public void onPreviewFrame(byte[] _data, Camera _camera) {
                       // do something
        _camera.addCallbackBuffer(_data);
    }
 };

g. Add implementations for SurfaceHolder.Callback

@Override
public void surfaceCreated(SurfaceHolder holder) {
    mCamera = Camera.open();
    mCamera.addCallbackBuffer(new byte[width * height * 2]); // YUV
    mCamera.setPreviewCallbackWithBuffer(cabWithBuffer);  
    try {
        mCamera.setPreviewDisplay(holder);
    } catch (IOException exception) {
        mCamera.release();
        mCamera = null;
    }
}

public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
    Camera.Parameters parameters = mCamera.getParameters();
    parameters.setPreviewSize(width, height);
    mCamera.setParameters(parameters);
    mCamera.setPreviewCallbackWithBuffer(cbWithBuffer);
    mCamera.startPreview();
}

@Override
public void surfaceDestroyed(SurfaceHolder holder) {
    mCamera.stopPreview();
    mCamera.setPreviewCallbackWithBuffer(null);
    mCamera.release();
    mCamera = null;
}

2. Process video: 

a. Declare matrices:

Mat mYuv, mRgba;

b. Allocate space for mYuv, mRgb in createOpencvMatrices():

    mYuv = new Mat(height + height/2, width, CvType.CV_8UC1);
    mRgb = new Mat(height, width, CvType.CV_8UC3);

c. Process image in cbWithBuffer:

    mYuv.put(0, 0, _data); 
    Imgproc.cvtColor(mYuv, mRgb, Imgproc.COLOR_YUV2RGB_NV21, 3);
    processImage(mRgb.getNativeObjAddr());

d. Add function declaration

    public native int processImage(long matAddrRgb);

e. Add image processing implementation in HelloWorld.cpp:

JNIEXPORT void JNICALL Java_org_ski_helloworld_MainActivity_processImage(JNIEnv*, jobject, jlong addrRgb)
{
    Mat &rgb = *(Mat *) addrRgb;
    int w = rgb.cols, h = rgb.rows;
    Rect r(w/4,h/4,w/2,h/2);
    Mat roi = rgb(r).clone();
    Mat gray, edge;
    cvtColor(roi, gray, CV_RGB2GRAY);
    blur( gray, gray, Size(5,5));
    Canny( gray, edge, 50, 150, 3);
    edge.convertTo(gray, CV_8UC1);
    cvtColor(gray,roi, CV_GRAY2RGB);
    roi.copyTo(rgb(r));
}

The zipped project is HelloWorld_02.zip

Android App with OpenCV and C++, Part 2

1. Add opencv to the workspace

File->Import..->Android->Existing Code into Workspace.  Locate the path and check "Copy projects into workspace"

2. Add opencv dependency to the project:

Right click on the project->properties->Android, click "Add", and add the opencv lib:

3. Add opencv support code:

Add declarations:

Mat mat;
private BaseLoaderCallback  mLoaderCallback;

Overload onResume():

@Override
public void onResume(){
    super.onResume();
    OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_9, this, mLoaderCallback);
}

Add initilization in onCreate(), and any matrices are to be created after opencv lib are laoded successfully:

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    mLoaderCallback = new BaseLoaderCallback(this) {
    
        @Override
        public void onManagerConnected(int status) {
            switch (status) {
            case LoaderCallbackInterface.SUCCESS:
                Log.i("HelloWorld: ", "OpenCV loaded successfully");
                System.loadLibrary("HelloWorld");
                createOpencvMatrices();
                break;
            default:
               super.onManagerConnected(status);
            }
        }
    };
}
    
void createOpencvMatrices(){
    mat = new Mat();
    // other matrices to be created, ...
}

Add a test function:

 public native int test0(long matAddrDat); 

Edit file activity_main.xml in res->layout:

<TextView
    android:id="@+id/tvHello"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:text="Hello World" />

Declare TextView tvHello, and added this to onCreate:

tvHello = (TextView) findViewById(R.id.tvHello);

Add following after createOpencvMatrices();

createOpencvMatrices(); // matrix creation comes after loadLibrary
int out = test0(mat.getNativeObjAddr());  // test function
tvHello.setText("output from test0() = "+ out);

Edit HelloWorld.cpp:

......
JNIEXPORT int JNICALL Java_org_ski_helloworld_MainActivity_test0(JNIEnv*, jobject, jlong addrData)
{
    Mat& mat = *(Mat*)addrData;
    mat.create(1,3,CV_32F);
    Mat_<float> &tmp = (Mat_<float> &)mat;
    Mat_<float>::iterator it;
    int i=1;
    for (it =tmp.begin(); it!= tmp.end(); it++,i++)
        *it = i;

    int sum = 0;
    for (it = tmp.begin(); it!= tmp.end(); it++)
        sum += *it;
    return sum;
}
......

You should get "output from test0() = 6" after you launch the app. The zip file of the project: link.

Android App with OpenCV and C++, Part 1

1. Preparation

• Install Eclipse
• Install Android SDK
• Install Android NDK
• Download and save OpenCV4Android

2. Create a hello world project

a. Create a regular Android application project

File->New->Android Application Project:

Select Blank Activity, click "Next", and "Finish"

b. Add NDK support:

Right click on HelloWorld project->Android Tools->Add Native Support.

Now you have new folder jni with two files:

Create a file "Application.mk", with following content:

APP_ABI := armeabi-v7a
NDK_TOOLCHAIN_VERSION := 4.9
APP_CPPFLAGS += -std=c++11 -fexceptions  -frtti
APP_STL := gnustl_static

And put to folder "jni".

Edit "Android.mk" so it contains:

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
OPENCV_SDK_JNI = c:\\OpenCV-2.4.11-android-sdk\\sdk\\native\\jni
include $(OPENCV_SDK_JNI)/OpenCV.mk
LOCAL_MODULE    := HelloWorld
LOCAL_SRC_FILES := HelloWorld.cpp
LOCAL_C_INCLUDES += $(OPENCV_SDK_JNI)/include
include $(BUILD_SHARED_LIBRARY)

Where OPENCV_SDK_JNI should point to the correct location in your machine.

Edit HelloWorld.cpp. Save everything.

#include <jni.h>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
using namespace std;
using namespace cv;
extern "C" {
JNIEXPORT int JNICALL Java_org_ski_helloworld_MainActivity_test0(JNIEnv*, jobject, jlong addrData){
         Mat& mDat = *(Mat*)addrData;
        //  doing stuff ...
    }
}

Select "Build Project".  Now you should have correct syntax highlighting of c++ files.