Preview in CameraX Alpha 10 with floating widget background service has no image

2020-02-15 java android android-camerax

I am trying to use androids cameraX in a FLOATING WIDGET BACKGROUNDSERVICE but it doesn't display an image of the camera. This is specific to running this in a background service as the same functionality with identical code works in a regular android application. I've tried changing from a TextureView to a SurfaceView using previewView.setImplementationMode(PreviewView.ImplementationMode.SURFACE_VIEW); The only difference being I get a black image where the camera should be when using a SurfaceView. Comparing the logcats from the non-working background service to working application the most notable difference I see is the following:

D/Camera: CameraDevice.onOpened(): 1
D/Camera: Transitioning camera internal state: OPENING --> OPENED
D/UseCaseAttachState: All use case: [Preview] for camera: 1
D/CaptureSession: Opening capture session.
D/CaptureSession: Attempting to send capture request onConfigured
D/CaptureSession: Issuing request for session.
D/CaptureSession: CameraCaptureSession.onConfigured() mState=OPENED
D/CaptureSession: CameraCaptureSession.onReady() OPENED

---My Code---

public class FloatingViewService extends Service {
private ListenableFuture cameraProviderFuture;

private WindowManager mWindowManager;
private View mFloatingView;
public String StreamName = null;
TextureView textureView;
PreviewView previewView;
private int REQUEST_CODE_PERMISSIONS = 10; //arbitrary number, can be changed accordingly
private final String[] REQUIRED_PERMISSIONS = new String[]{"android.permission.CAMERA","android.permission.WRITE_EXTERNAL_STORAGE"}; //array w/ permissions from manifest

public FloatingViewService() {
}

@Override
public IBinder onBind(Intent intent) {
    return null;
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {

    String sname = intent.getStringExtra("streamname");
    StreamName = sname;
    //do something

    return START_STICKY;

}
@SuppressLint("RestrictedApi")
@Override
public void onCreate() {
    super.onCreate();

    //Inflate the floating view layout we created
    mFloatingView = LayoutInflater.from(this).inflate(R.layout.layout_floating_widget, null);

    //Add the view to the window.
    final WindowManager.LayoutParams params = new WindowManager.LayoutParams(
            WindowManager.LayoutParams.WRAP_CONTENT,
            WindowManager.LayoutParams.WRAP_CONTENT,
            WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY,
            WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE,
            PixelFormat.TRANSLUCENT);

    //Specify the view position
    params.gravity = Gravity.TOP | Gravity.LEFT;        //Initially view will be added to top-left corner
    params.x = 0;
    params.y = 100;

    //Add the view to the window
    mWindowManager = (WindowManager) getSystemService(WINDOW_SERVICE);
    mWindowManager.addView(mFloatingView, params);

    //The root element of the collapsed view layout
    final View collapsedView = mFloatingView.findViewById(R.id.collapse_view);
    //The root element of the expanded view layout
    final View expandedView = mFloatingView.findViewById(R.id.expanded_container);


    //Set the close button
    ImageView closeButtonCollapsed = (ImageView) mFloatingView.findViewById(R.id.close_btn);
    closeButtonCollapsed.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View view) {
            //close the service and remove the from from the window
            SwitchToGS("Ending Stream");
            EndStream();
            //stopSelf();
        }
    });


    //Set the close button
    ImageView closeButton = (ImageView) mFloatingView.findViewById(R.id.close_button);
    closeButton.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View view) {
            collapsedView.setVisibility(View.VISIBLE);
            expandedView.setVisibility(View.GONE);
            SendAlert("Resuming Stream");
        }
    });

    //Set the close button
    ImageView PlayButton = (ImageView) mFloatingView.findViewById(R.id.play_button);
    PlayButton.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View view) {
            collapsedView.setVisibility(View.VISIBLE);
            expandedView.setVisibility(View.GONE);
            ((ImageView) mFloatingView.findViewById(R.id.collapsed_iv)).setImageResource(R.drawable.ic_pause);
            if(StreamName == null || StreamName == "") {
                //MainActivity.intent(getApplicationContext()).start();
            }
            else {
                BroadCastScreenActivity_.intent(getApplicationContext()).extra("streamname", StreamName).start();

            }
        }
    });

    //Open the application on this button click
    ImageView openButton = (ImageView) mFloatingView.findViewById(R.id.open_button);
    openButton.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View view) {

            SwitchToGS("streampause");
        }
    });

    //Open the application on thi button click
    ImageView StopButton = (ImageView) mFloatingView.findViewById(R.id.stop_button);
    StopButton.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View view) {
            //EndStream();
            //SwitchToGS("streamend");
            startCamera();
        }
    });
    //Drag and move floating view using user's touch action.
    mFloatingView.findViewById(R.id.root_container).setOnTouchListener(new View.OnTouchListener() {
        private int initialX;
        private int initialY;
        private float initialTouchX;
        private float initialTouchY;

        @Override
        public boolean onTouch(View v, MotionEvent event) {
            switch (event.getAction()) {
                case MotionEvent.ACTION_DOWN:

                    //remember the initial position.
                    initialX = params.x;
                    initialY = params.y;

                    //get the touch location
                    initialTouchX = event.getRawX();
                    initialTouchY = event.getRawY();
                    return true;
                case MotionEvent.ACTION_UP:
                    int Xdiff = (int) (event.getRawX() - initialTouchX);
                    int Ydiff = (int) (event.getRawY() - initialTouchY);

                    //The check for Xdiff <10 && YDiff< 10 because sometime elements moves a little while clicking.
                    //So that is click event.
                    if (Xdiff < 10 && Ydiff < 10) {
                        if (isViewCollapsed()) {
                            //When user clicks on the image view of the collapsed layout,
                            //visibility of the collapsed layout will be changed to "View.GONE"
                            //and expanded view will become visible.
                            collapsedView.setVisibility(View.GONE);
                            expandedView.setVisibility(View.VISIBLE);
                            SendAlert("Pausing Stream");
                        }
                    }
                    return true;
                case MotionEvent.ACTION_MOVE:
                    //Calculate the X and Y coordinates of the view.
                    params.x = initialX + (int) (event.getRawX() - initialTouchX);
                    params.y = initialY + (int) (event.getRawY() - initialTouchY);

                    //Update the layout with new X & Y coordinate
                    mWindowManager.updateViewLayout(mFloatingView, params);
                    return true;
            }
            return false;
        }
    }); 
    //BroadCasterActivity_.intent(this).extra("streamname", StreamName).start();
    collapsedView.setVisibility(View.GONE);
    expandedView.setVisibility(View.VISIBLE);
    String[] permissions = {Manifest.permission.CAMERA, Manifest.permission.WRITE_EXTERNAL_STORAGE};
    Permissions.check(this/*context*/, permissions, null/*rationale*/, null/*options*/, new PermissionHandler() {
        @Override
        public void onGranted() {

        }
    });


}

private void startCamera()
{
    previewView = mFloatingView.findViewById(R.id.preview_view);
    if(previewView!=null) {
        cameraProviderFuture = ProcessCameraProvider.getInstance(this);
        cameraProviderFuture.addListener(() -> {
            try {
                ProcessCameraProvider cameraProvider = (ProcessCameraProvider) cameraProviderFuture.get();
                bindPreview(cameraProvider);
            } catch (ExecutionException | InterruptedException e) {
                SendAlert("CameraProvider Error");
                // No errors need to be handled for this Future
                // This should never be reached
            }
        }, ContextCompat.getMainExecutor(this));
    }
}

void bindPreview(@NonNull ProcessCameraProvider cameraProvider) {
    Preview preview = new Preview.Builder()
            .setTargetName("Preview").build();
            //.setTargetAspectRatio(AspectRatio.RATIO_4_3)
            //.setTargetResolution(new Size(640,640)).build();
    previewView.setImplementationMode(PreviewView.ImplementationMode.SURFACE_VIEW);
    preview.setSurfaceProvider(previewView.getPreviewSurfaceProvider());

    CameraSelector cameraSelector =
            new CameraSelector.Builder().requireLensFacing(CameraSelector.LENS_FACING_FRONT).build();
    //lifecycle for camera
    CustomLifecycle lifecycle = new CustomLifecycle();
    //lifecycle.doOnResume();
    //lifecycle.doOnStart();
    cameraProvider.bindToLifecycle(lifecycle, cameraSelector, preview);

}

private boolean allPermissionsGranted(){
    //check if req permissions have been granted
    for(String permission : REQUIRED_PERMISSIONS){
        if(ContextCompat.checkSelfPermission(this, permission) != PackageManager.PERMISSION_GRANTED){
            return false;
        }
    }
    return true;
}


private void ExpandWidget()
{
    mFloatingView.findViewById(R.id.collapse_view).setVisibility(View.VISIBLE);
    mFloatingView.findViewById(R.id.expanded_container).setVisibility(View.GONE);
}

private void CollapseWidget()
{
    mFloatingView.findViewById(R.id.collapse_view).setVisibility(View.GONE);
    mFloatingView.findViewById(R.id.expanded_container).setVisibility(View.VISIBLE);
}

private void EndStream() {
    Intent intent1 = new Intent();
    intent1.setAction("com.nhancv.kurentoandroid.broadcaster");
    intent1.putExtra("DATAPASSED", "endstream");
    sendBroadcast(intent1);
    StreamName = null;
    stopSelf();

}

private void SwitchToGS(String sCommand) {
    //Intent intent = new Intent();
    //intent.setComponent(new ComponentName("com.example", "com.example.gostream.appConfigs.App"));
    //startActivity(intent);
    //Intent intent = new Intent("com.example.gostream");
    //Intent intent = new Intent (Intent.ACTION_SEND, Uri.parse("http://androidium.org"));
    //startActivity(intent);

       Intent launchIntent = getPackageManager().getLaunchIntentForPackage("com.gostream.gostreamapp");
        if (launchIntent != null) {
            launchIntent.putExtra("DATAPASSED", sCommand);
            //launchIntent.addFlags(Intent.FLAG_ACTIVITY_SINGLE_TOP);
            launchIntent.addFlags(Intent.FLAG_ACTIVITY_REORDER_TO_FRONT);
            getApplication().startActivity(launchIntent);
            //startActivity(launchIntent);
        } else {
            Toast.makeText(
                    this,
                    "There is no package available in android",
                    Toast.LENGTH_LONG
            ).show();
        }

}
private void SendAlert(String sreportstring) {
    Context context = getApplicationContext();
    int duration = Toast.LENGTH_SHORT;
    Toast toast = Toast.makeText(context, sreportstring, duration);
    toast.show();
}
/**
 * Detect if the floating view is collapsed or expanded.
 *
 * @return true if the floating view is collapsed.
 */
private boolean isViewCollapsed() {
    return mFloatingView == null || mFloatingView.findViewById(R.id.collapse_view).getVisibility() == View.VISIBLE;
}

@Override
public void onDestroy() {
    super.onDestroy();
    if (mFloatingView != null) mWindowManager.removeView(mFloatingView);
}

}

---WORKING ANDROID APP LOGCAT---

2020-02-14 13:46:12.040 7959-7959/com.example.cameraxpreviewtest D/[email protected][MainActivity]: ViewPostIme pointer 0

2020-02-14 13:46:12.087 7959-7959/com.example.cameraxpreviewtest D/[email protected][MainActivity]: ViewPostIme pointer 1

2020-02-14 13:46:12.129 7959-8414/com.example.cameraxpreviewtest I/CameraManagerGlobal: Connecting to camera service

2020-02-14 13:46:12.134 7959-8414/com.example.cameraxpreviewtest D/VendorTagDescriptor: addVendorDescriptor: vendor tag id 3854507339 added

2020-02-14 13:46:12.162 7959-8414/com.example.cameraxpreviewtest D/CameraRepository: Added camera: 0

2020-02-14 13:46:12.177 7959-8414/com.example.cameraxpreviewtest I/Camera2CameraInfo: Device Level: INFO_SUPPORTED_HARDWARE_LEVEL_3

2020-02-14 13:46:12.181 7959-8415/com.example.cameraxpreviewtest D/UseCaseAttachState: Active and online use case: [] for camera: 0

2020-02-14 13:46:12.181 7959-8414/com.example.cameraxpreviewtest D/CameraRepository: Added camera: 1

2020-02-14 13:46:12.185 7959-8414/com.example.cameraxpreviewtest I/Camera2CameraInfo: Device Level: INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED

2020-02-14 13:46:12.186 7959-8414/com.example.cameraxpreviewtest D/CameraRepository: Added camera: 2

2020-02-14 13:46:12.189 7959-8415/com.example.cameraxpreviewtest D/UseCaseAttachState: Active and online use case: [] for camera: 1

2020-02-14 13:46:12.190 7959-8414/com.example.cameraxpreviewtest I/Camera2CameraInfo: Device Level: INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED

2020-02-14 13:46:12.194 7959-8415/com.example.cameraxpreviewtest D/UseCaseAttachState: Active and online use case: [] for camera: 2

2020-02-14 13:46:12.217 7959-7959/com.example.cameraxpreviewtest D/SurfaceView: onWindowVisibilityChanged(0) true 3142247 of [email protected][MainActivity]

2020-02-14 13:46:12.380 7959-8415/com.example.cameraxpreviewtest D/Camera: Use cases [Preview:Preview] now ONLINE for camera 1

2020-02-14 13:46:12.381 7959-8415/com.example.cameraxpreviewtest D/UseCaseAttachState: Active and online use case: [] for camera: 1

2020-02-14 13:46:12.381 7959-8415/com.example.cameraxpreviewtest D/Camera: Resetting Capture Session

2020-02-14 13:46:12.381 7959-8415/com.example.cameraxpreviewtest D/Camera: releasing session in state INITIALIZED

2020-02-14 13:46:12.382 7959-8415/com.example.cameraxpreviewtest D/Camera: Transitioning camera internal state: INITIALIZED --> OPENING

2020-02-14 13:46:12.382 7959-8415/com.example.cameraxpreviewtest D/Camera: Opening camera: 1

2020-02-14 13:46:12.384 7959-8415/com.example.cameraxpreviewtest D/UseCaseAttachState: All use case: [Preview] for camera: 1

2020-02-14 13:46:12.388 7959-8417/com.example.cameraxpreviewtest E/Camera_APM :: [APM]APM's Thread is started

2020-02-14 13:46:12.390 7959-8417/com.example.cameraxpreviewtest E/Camera_APM :: [APM] This app is forground app

2020-02-14 13:46:12.417 7959-7959/com.example.cameraxpreviewtest D/[email protected][MainActivity]: Relayout returned: old=[0,0][1440,2960] new=[0,0][1440,2960] result=0x1 surface={valid=true 514989817856} changed=false

2020-02-14 13:46:12.432 7959-7959/com.example.cameraxpreviewtest D/SurfaceView: show() Surface(name=SurfaceView - com.example.cameraxpreviewtest/[email protected]@0[7959])/@0x1e7086c androidx.camera.view.TransformableSurfaceView{3142247 V.E...... ......ID 0,0-1440,2448}

2020-02-14 13:46:12.441 7959-7959/com.example.cameraxpreviewtest D/SurfaceView: surfaceCreated 1 #8 androidx.camera.view.TransformableSurfaceView{3142247 V.E...... ......ID 0,0-1440,2448}

2020-02-14 13:46:12.441 7959-7959/com.example.cameraxpreviewtest D/SurfaceViewPreviewView: Surface created.

2020-02-14 13:46:12.442 7959-7959/com.example.cameraxpreviewtest D/SurfaceView: surfaceChanged (1440,2448) 1 #8 androidx.camera.view.TransformableSurfaceView{3142247 V.E...... ......ID 0,0-1440,2448}

2020-02-14 13:46:12.442 7959-7959/com.example.cameraxpreviewtest D/SurfaceViewPreviewView: Surface changed. Size: 1440x2448

2020-02-14 13:46:12.450 7959-7959/com.example.cameraxpreviewtest D/SurfaceViewPreviewView: Wait for new Surface creation.

2020-02-14 13:46:12.453 7959-7959/com.example.cameraxpreviewtest D/SurfaceView: setPackageUsesOwnResolution() (Java SurfaceView) for com.example.cameraxpreviewtest: 1280x960 (true)

2020-02-14 13:46:12.477 7959-7959/com.example.cameraxpreviewtest D/SurfaceView: show() Surface(name=SurfaceView - com.example.cameraxpreviewtest/[email protected]@0[7959])/@0x1e7086c androidx.camera.view.TransformableSurfaceView{3142247 V.E...... ......ID 0,0-1280,960}

2020-02-14 13:46:12.492 7959-7959/com.example.cameraxpreviewtest D/SurfaceView: surfaceChanged (1280,960) 1 #5 androidx.camera.view.TransformableSurfaceView{3142247 V.E...... ......ID 0,0-1280,960}

2020-02-14 13:46:12.493 7959-7959/com.example.cameraxpreviewtest D/SurfaceViewPreviewView: Surface changed. Size: 1280x960

2020-02-14 13:46:12.493 7959-7959/com.example.cameraxpreviewtest D/SurfaceViewPreviewView: Surface set on Preview.

2020-02-14 13:46:12.524 7959-8415/com.example.cameraxpreviewtest D/Camera: Use case Preview:Preview ACTIVE for camera 1

2020-02-14 13:46:12.527 7959-8415/com.example.cameraxpreviewtest D/UseCaseAttachState: Active and online use case: [Preview] for camera: 1

2020-02-14 13:46:12.530 7959-8415/com.example.cameraxpreviewtest D/Camera: CameraDevice.onOpened(): 1

2020-02-14 13:46:12.552 7959-8415/com.example.cameraxpreviewtest D/Camera: Transitioning camera internal state: OPENING --> OPENED

2020-02-14 13:46:12.554 7959-8415/com.example.cameraxpreviewtest D/UseCaseAttachState: All use case: [Preview] for camera: 1

2020-02-14 13:46:12.567 7959-8415/com.example.cameraxpreviewtest D/CaptureSession: Opening capture session.

2020-02-14 13:46:13.068 7959-8415/com.example.cameraxpreviewtest D/CaptureSession: Attempting to send capture request onConfigured

2020-02-14 13:46:13.068 7959-8415/com.example.cameraxpreviewtest D/CaptureSession: Issuing request for session.

2020-02-14 13:46:13.222 7959-8415/com.example.cameraxpreviewtest D/CaptureSession: CameraCaptureSession.onConfigured() mState=OPENED

2020-02-14 13:46:13.223 7959-8415/com.example.cameraxpreviewtest D/CaptureSession: CameraCaptureSession.onReady() OPENED

---NON WORKING FLOATING WIDGET BACKGROUND SERVICE LOGCAT---

2020-02-14 13:44:35.103 7038-7038/com.nhancv.kurentoandroid D/[email protected][]: ViewPostIme pointer 0

2020-02-14 13:44:35.154 7038-7038/com.nhancv.kurentoandroid D/[email protected][]: ViewPostIme pointer 1

2020-02-14 13:44:35.193 7038-7456/com.nhancv.kurentoandroid I/CameraManagerGlobal: Connecting to camera service

2020-02-14 13:44:35.204 7038-7456/com.nhancv.kurentoandroid D/VendorTagDescriptor: addVendorDescriptor: vendor tag id 3854507339 added

2020-02-14 13:44:35.233 7038-7456/com.nhancv.kurentoandroid D/CameraRepository: Added camera: 0

2020-02-14 13:44:35.249 7038-7456/com.nhancv.kurentoandroid I/Camera2CameraInfo: Device Level: INFO_SUPPORTED_HARDWARE_LEVEL_3

2020-02-14 13:44:35.253 7038-7457/com.nhancv.kurentoandroid D/UseCaseAttachState: Active and online use case: [] for camera: 0

2020-02-14 13:44:35.254 7038-7456/com.nhancv.kurentoandroid D/CameraRepository: Added camera: 1

2020-02-14 13:44:35.257 7038-7456/com.nhancv.kurentoandroid I/Camera2CameraInfo: Device Level: INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED

2020-02-14 13:44:35.258 7038-7456/com.nhancv.kurentoandroid D/CameraRepository: Added camera: 2

2020-02-14 13:44:35.262 7038-7456/com.nhancv.kurentoandroid I/Camera2CameraInfo: Device Level: INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED

2020-02-14 13:44:35.262 7038-7457/com.nhancv.kurentoandroid D/UseCaseAttachState: Active and online use case: [] for camera: 1

2020-02-14 13:44:35.267 7038-7457/com.nhancv.kurentoandroid D/UseCaseAttachState: Active and online use case: [] for camera: 2

2020-02-14 13:44:35.290 7038-7038/com.nhancv.kurentoandroid D/SurfaceView: onWindowVisibilityChanged(0) true 2fc6f83 of [email protected][]

2020-02-14 13:44:35.453 7038-7457/com.nhancv.kurentoandroid D/Camera: Use case Preview:Preview ACTIVE for camera 1

2020-02-14 13:44:35.455 7038-7457/com.nhancv.kurentoandroid D/UseCaseAttachState: Active and online use case: [] for camera: 1

2020-02-14 13:44:35.471 7038-7038/com.nhancv.kurentoandroid D/[email protected][]: Relayout returned: old=[0,196][1280,1460] new=[0,196][1280,1460] result=0x1 surface={valid=true 514848976896} changed=false

2020-02-14 13:44:35.480 7038-7038/com.nhancv.kurentoandroid D/SurfaceView: show() Surface(name=SurfaceView - @[email protected][7038])/@0xf9d14ea androidx.camera.view.TransformableSurfaceView{2fc6f83 V.E...... ......ID 0,0-1600,1200}

2020-02-14 13:44:35.491 7038-7038/com.nhancv.kurentoandroid D/SurfaceView: surfaceCreated 1 #8 androidx.camera.view.TransformableSurfaceView{2fc6f83 V.E...... ......ID 0,0-1600,1200}

2020-02-14 13:44:35.491 7038-7038/com.nhancv.kurentoandroid D/SurfaceViewPreviewView: Surface created.

2020-02-14 13:44:35.491 7038-7038/com.nhancv.kurentoandroid D/SurfaceView: surfaceChanged (1600,1200) 1 #8 androidx.camera.view.TransformableSurfaceView{2fc6f83 V.E...... ......ID 0,0-1600,1200}

2020-02-14 13:44:35.492 7038-7038/com.nhancv.kurentoandroid D/SurfaceViewPreviewView: Surface changed. Size: 1600x1200

2020-02-14 13:44:35.496 7038-7038/com.nhancv.kurentoandroid D/SurfaceViewPreviewView: Wait for new Surface creation.

2020-02-14 13:44:35.505 7038-7038/com.nhancv.kurentoandroid D/SurfaceView: setPackageUsesOwnResolution() (Java SurfaceView) for com.nhancv.kurentoandroid: 1280x960 (true)

2020-02-14 13:44:35.529 7038-7038/com.nhancv.kurentoandroid D/SurfaceView: show() Surface(name=SurfaceView - @[email protected][7038])/@0xf9d14ea androidx.camera.view.TransformableSurfaceView{2fc6f83 V.E...... ......I. 0,0-1600,1200}

2020-02-14 13:44:35.541 7038-7038/com.nhancv.kurentoandroid D/SurfaceView: surfaceChanged (1280,960) 1 #5 androidx.camera.view.TransformableSurfaceView{2fc6f83 V.E...... ......I. 0,0-1600,1200}

2020-02-14 13:44:35.541 7038-7038/com.nhancv.kurentoandroid D/SurfaceViewPreviewView: Surface changed. Size: 1280x960

2020-02-14 13:44:35.541 7038-7038/com.nhancv.kurentoandroid D/SurfaceViewPreviewView: Surface set on Preview.

Answers

Related