Implementing USB Video Device Class (UVC) Device Driver on Microchip USB Stack, PIC32

I was recently working on a little project in which i need to stream video from mcu to PC, so i decided to look for a solution. My first obvious solution was to make a custom protocol over USB bulk or isochronous Transfer and also make some application in Qt or MATLAB to stream and display video.

But when i looked little bit more , I found out that there is already a USB video class assigned for this specific task. Driver for UVC device are already standard and present in many well used OS. Driver is present in Windows and Linux both.

As Video over USB is quite resource intensive task so it is not common to use a MCU,  I was using PIC32 MCU for project, and USB stack Provided by microchip was no exception, Microchip USB stack does not have support for UVC. So i decided to implement my self.

This project will be multi part project ,

in First part i will have static image 160x120 pixel resolution, stream over the USB and play in VLC.

Second Part i will interface a OV7670 camera to the MCU and stream live video on VLC.

A Sample image 160x120 Pixel converted to yuy2 , Streamed over USB played on VLC

I have First implemented UVC stack onto PIC32MX470F512H , which has USB FS, Limited to Max 12Mbit/S and RAM is also quite limited, So max Resolution and frame rate is limited.

Later i moved to PIC32MZ2048EFM064 which has USB HS, and now USB bandwidth is more than what we need for experiment.

Hardware

For implementation with PIC32MX470F512H i used my old PIC32 Breakout board , more details (schematic) about that board can be found here.

Here is  PIC32MX hardware , For simple static image you will just need board, another cape board is just simple breakout header so that this cheap ebay breakout board fits to my board.

Here is PIC32MZ Hardware , this board i recently made because it has faster processor and quite unique 480Mbit/S USB HS. More details about this board can be found here.

How to Convert image to YUV format

I have used ffmpeg to convert image to yuv format using command

ffmpeg.exe -i draw640.jpeg -s 640x480 -pix_fmt yuyv422 yuyv422.yuv

Viewing Raw YUV pixel image using a online viewer

To verify if output was correct format i used this website to view my output image for correct colors and everything else

Playback on PC

Device Will be detected as USB Camera As show in the image blow

VLC Playback : it simple as if playing from a webcam

Sample image displayed on Left, VLC on right streaming from Device over USB

Firmware 

i will go through the files which i have added into Microchip USB stack. all source present in my github here https://github.com/circuitvalley/USB-Video-Class-Stack-PIC32-UVC

USB Descriptor

USB Descriptor is really important to get usb device detected as UVC Device, Descriptor Defines what frame rate , what resolution camera supports, what is video format (nv12 or yuy2) , what endpoint is used to transfer data.

here is the complete listing from Descriptor for my device, USB Descriptor is really important, I have been Looking at USB UVC Documents and only get Partial info. Ultimately i looked at USB descriptor form Cypress USB 3.0 Camera Development board source, the after reading it a little bit i was able to complete my USB Descriptor

The Files which i have Added to Microchip USB Stacks are

usb_video.h

usb_video_local.h

usb_video.c

usb_device_video_read_write.c

You can find complete source here 

 https://github.com/circuitvalley/USB-Video-Class-Stack-PIC32-UVC

UVC Have certain class specific request as well , as PC send request for these messages , Firmware suppose to send back status. these request are implemented as application call  back.  APP_USBDeviceVideoEventHandler Function Handles these UVC specific request.

Next Part