Saturday, February 15, 2020

DIY IMX219 4 Lane MIPI CSI Breakout Board, Getting MIPI CSI 4 Lane From Raspberry PI Camera

I have published many post in which i have been interfacing various cameras sensors/modules to different processors. This post is also going to the next part the previous posts.

Currently i am trying to interface Raspberry pi camera V2.1 sony IMX219 Camera to Lattice FPGA  then ultimately to USB 3.0 with Cypress FX3 USB 3.0 controller.

As per schematic of Raspberry pi and Raspberry PI Camera module. It is two lane MIPI CSI Camera Board with 1 Clock and 2 MIPI CSI data lane. 









As my aim is to connect Raspberry pi V2.1 camera board to Lattice Machxo3LF FPGA. I searched for datasheet of the camera chip Sony IMX219 and found that IMX219 support 4 and 2 Lane MIPI CSI. Datasheet also mentions 755Mbps/Lane in 4 Lane mode and 912 Mbps/Lane in 2 Lane mode.
To get maximum performance out of the camera 4 Lane interface appears to be correct way.

But original Raspberry camera module is only 2 data lanes. But when i look at the schematic of the camera module it had few  pins empty and that is in two pairs surrounded by ground . It got me thinking that these empty are 3rd and 4th MIPI data lane.
Raspberry PI Camera module's connector
So i quickly made modifications to the camera board mounted a 24 Pin FPC connector and modified IMX219 Linux kernel driver just to write register 0x0114 to enable all 4 lanes of MIPI CSI.


As it turn out yes those 4 pins were actually other two MIPI CSI data lanes.


So i made setup shown in the image blow to interface this to a Lattice MachXO3LF FPGA.

Failed First Attempt to interface IMX219 to Lattice MachXO3
But it turned out the setup on cheap PCB , no impedance control no proper termination and large headers on FPGA and Breakout board had made 200 Mhz to 350 Mhz mipi signal unusable.
Signal quality was too poor with this setup i did many test  and only very low frequency mipi signal was functional. 
So i finally made my own IMX219 Camera module.

PCB






Blank IMX219 MIPI 4Lane Camera PCB is Available for purchase 9.99 EUR Free International Shipping. 
Buy Blank PCB
Schematic 





The Board is suppose to directly fit on Lattice MachXO3LF FPGA board. This way connection will be as short as possible, this setup will use Lattice boards's 100R termination register which is really important to have as near as possible to the Receiver FPGA itself.



This is a multiple part project complete description FPGA verilog source and every thing else about how to get this board up and running with Lattice machXO3 FPGA can be found in later post.

Next Part of the project

https://www.circuitvalley.com/2020/02/imx219-camera-mipi-csi-receiver-fpga-lattice-raspberry-pi-camera.html

Source Files

PCB and Schematic Source is available in the Github Repo

https://github.com/circuitvalley/mipi_csi_receiver_FPGA

11 comments:

  1. I'm interested in using this camera for a cheap Computer Vision solution. Do you have any idea how stable the current FPGA core is, and what features are currently missing or hacked together?

    ReplyDelete
    Replies
    1. Improvements need to be done at FPGA side to implement auto exposure, Brightness and white Balance correction.
      Other than this, its stable. Have been tested for hours and hours.

      Delete
    2. When you mention modifying the kernel driver for testing (the 0x114 register), which one are you referring to?

      Delete
    3. Yes register 0x114 control if camera output, 2 lane or 4 Lane MIPI data

      Delete
  2. You have done really a nice Job,I will happy to promote your blog link to People working on camera based design, this will be helpful.Do you have any plan to post design based on Lattice Crosslink-NX . Several people are interested on Crosslink-NX based design.

    ReplyDelete
    Replies
    1. There will be a project coming very soon with lattice Crosslink-NX.

      Delete
  3. I am interested in the other end of this problem where the camera MIPI CSI TX interface would be implemented in an FPGA to get highspeed sensor data into the Raspberry Pi or other processor with a MIPI CSI camera interface. Have you or do you know of someone who has attacked this problem?

    ReplyDelete
    Replies
    1. Using CSI port as input for devices other than camera definitely possible. Raspberry pi is pretty close in terms of how GPU handle data received from MIPI port, but with recent release of info on CSI from Raspberry pi people, it's somewhat easy now.
      I have not yet seen anybody using this capacity for RAW data, But one board which make use of Camera mipi CSI port as input with FPGA to feed HDMI video into Raspberry pi. Essentially giving Raspberry pi Capability to capture Video off HDMI.

      Using MIPI CSI as input port for other than camera or HDMI to CSI bridge can be done. But don't offer much of advantage over USB 3.0. MIPI CSI on Rapsberry pi Board with just 2 lane is not that fast.
      Only advantage MIPI CSI may offer will that bridge controller would be simple when compare to USB3.0

      Delete
    2. Thank you for your response. I have looked at all the high speed interfaces on these maker class boards and my concern with attempting to use something like USB 3.0 is there is the theoretically max speed of 5Gbps and then there is the real world performance. Especially in cases like the Raspberry Pi where I believe they enable USB via a PCI controller shared by several high speed interfaces. The fact that the MIPI CSI is dedicated and interfaces directly to the GPU is also a great advantage. The applications I am thinking of the data is imaging data just not camera or video imaging data.

      The board that you mentioned that goes HDMI to MIPI CSI do you have a link to it?

      BTW: Great job on the projects I have seen from you. I have reviewed them deeply, just skimmed but they look pretty thorough!!!

      Delete
  4. Hi I am interested in that project because mine is very similar. I have Raspberry Pi camera which is IMX219 also I am using Lattice fpga board which is LIFMD6000 and I am using Raspberry pi 3 board. I used Cmos to Dphy IP core , I embed inside the board an I configure the settings but I can not get any video output. I could not solve the problem. I am open to any idea about it. Can you help me with this issue?

    ReplyDelete
  5. Gaurav your Blog is very informative. Multiple customer those used XO2 and XO3 now moving toward crosslink and Crosslink-NX Make some tutorial series which can help all the communities whether they are Beginners or Professionals in step by step manner. scratch to finish.

    ReplyDelete