How to run Webcam Closer to Real-time Performance on Jupyter Notebook + Python 3 in 2021?

It is still a pain to work with webcam + Jupyter in 2021

Getting pretty good (close to) real-time performance with the 1080p input.

I’ve spent a few good hours getting my webcam to show properly in a Jupyter Lab today. My simple takeaway is you need this hack/workaround with the iPython display library to get enough performance to be interactive at all.

Here is a 2 minutes explanation of how it works:

And here is the code:

import numpy as np
import cv2
from IPython import display
from PIL import Image
import matplotlib.pyplot as py
%matplotlib inline
def showVideo(VideoIndex=0, scale=0.5):
    try:
        cap = cv2.VideoCapture(VideoIndex)
    except:
        print("Cannot Open Device")
    try:
        ret, frame = cap.read()
        
        while(ret==True):
            # Capture frame-by-frame
            ret, frame = cap.read()
            
            if not ret:
                # Release the Video Device if ret is false
                cap.release()
                # Message to be displayed after releasing the device
                print ("Released Video Resource")
                break
            #frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            half_frame = cv2.resize(frame, (0, 0), fx = scale, fy = scale)
#this fixed the pipeline issue as the imshow was converting to 
            #png!!!!
            #credit: 
            # https://medium.com/@kostal91/displaying-real-time-webcam-stream-in-ipython-at-relatively-high-framerate-8e67428ac522
            _,ret_array = cv2.imencode('.jpg', half_frame) 
            i = display.Image(data=ret_array)
display.display(i)
            display.clear_output(wait=True)
            
    except KeyboardInterrupt:
        # Release the Video Device
        cap.release()
        # Message to be displayed after releasing the device
        print("Released Video Resource from KeyboardInterrupt")
    pass
showVideo(1, 0.5)

The above solution was discovered by https://medium.com/@kostal91/displaying-real-time-webcam-stream-in-ipython-at-relatively-high-framerate-8e67428ac522. It’s 2021 now and it’s still a pain to work with real-time data with this! Next step: OpenGL + 3D on Jupyter Notebooks!

Note: I also tried the cv2.imshow function, and it does have great performance. However, having a pop-up window is not a great user experience, and I am not a big fan of that for sure.