Currently i am trying to interface Raspberry pi camera V2.1 sony IMX219 Camera to Lattice FPGA then ultimately to USB 3.0 with Cypress FX3 USB 3.0 controller.
As per schematic of Raspberry pi and Raspberry PI Camera module. It is two lane MIPI CSI Camera Board with 1 Clock and 2 MIPI CSI data lane.
As my aim is to connect Raspberry pi V2.1 camera board to Lattice Machxo3LF FPGA. I searched for datasheet of the camera chip Sony IMX219 and found that IMX219 support 4 and 2 Lane MIPI CSI. Datasheet also mentions 755Mbps/Lane in 4 Lane mode and 912 Mbps/Lane in 2 Lane mode.
To get maximum performance out of the camera 4 Lane interface appears to be correct way.
But original Raspberry camera module is only 2 data lanes. But when i look at the schematic of the camera module it had few pins empty and that is in two pairs surrounded by ground . It got me thinking that these empty are 3rd and 4th MIPI data lane.
|Raspberry PI Camera module's connector|
As it turn out yes those 4 pins were actually other two MIPI CSI data lanes.
So i made setup shown in the image blow to interface this to a Lattice MachXO3LF FPGA.
|Failed First Attempt to interface IMX219 to Lattice MachXO3|
Signal quality was too poor with this setup i did many test and only very low frequency mipi signal was functional.
So i finally made my own IMX219 Camera module.
Blank IMX219 MIPI 4Lane Camera PCB is Available for purchase 9.99 EUR Free International Shipping.
Buy Blank PCBSchematic
The Board is suppose to directly fit on Lattice MachXO3LF FPGA board. This way connection will be as short as possible, this setup will use Lattice boards's 100R termination register which is really important to have as near as possible to the Receiver FPGA itself.
This is a multiple part project complete description FPGA verilog source and every thing else about how to get this board up and running with Lattice machXO3 FPGA can be found in later post.
Next Part of the project
PCB and Schematic Source is available in the Github Repo.
I'm interested in using this camera for a cheap Computer Vision solution. Do you have any idea how stable the current FPGA core is, and what features are currently missing or hacked together?ReplyDelete
Improvements need to be done at FPGA side to implement auto exposure, Brightness and white Balance correction.Delete
Other than this, its stable. Have been tested for hours and hours.
When you mention modifying the kernel driver for testing (the 0x114 register), which one are you referring to?Delete
Yes register 0x114 control if camera output, 2 lane or 4 Lane MIPI dataDelete
You have done really a nice Job,I will happy to promote your blog link to People working on camera based design, this will be helpful.Do you have any plan to post design based on Lattice Crosslink-NX . Several people are interested on Crosslink-NX based design.ReplyDelete
There will be a project coming very soon with lattice Crosslink-NX.Delete
I am interested in the other end of this problem where the camera MIPI CSI TX interface would be implemented in an FPGA to get highspeed sensor data into the Raspberry Pi or other processor with a MIPI CSI camera interface. Have you or do you know of someone who has attacked this problem?ReplyDelete
Using CSI port as input for devices other than camera definitely possible. Raspberry pi is pretty close in terms of how GPU handle data received from MIPI port, but with recent release of info on CSI from Raspberry pi people, it's somewhat easy now.Delete
I have not yet seen anybody using this capacity for RAW data, But one board which make use of Camera mipi CSI port as input with FPGA to feed HDMI video into Raspberry pi. Essentially giving Raspberry pi Capability to capture Video off HDMI.
Using MIPI CSI as input port for other than camera or HDMI to CSI bridge can be done. But don't offer much of advantage over USB 3.0. MIPI CSI on Rapsberry pi Board with just 2 lane is not that fast.
Only advantage MIPI CSI may offer will that bridge controller would be simple when compare to USB3.0
Thank you for your response. I have looked at all the high speed interfaces on these maker class boards and my concern with attempting to use something like USB 3.0 is there is the theoretically max speed of 5Gbps and then there is the real world performance. Especially in cases like the Raspberry Pi where I believe they enable USB via a PCI controller shared by several high speed interfaces. The fact that the MIPI CSI is dedicated and interfaces directly to the GPU is also a great advantage. The applications I am thinking of the data is imaging data just not camera or video imaging data.Delete
The board that you mentioned that goes HDMI to MIPI CSI do you have a link to it?
BTW: Great job on the projects I have seen from you. I have reviewed them deeply, just skimmed but they look pretty thorough!!!
Hi I am interested in that project because mine is very similar. I have Raspberry Pi camera which is IMX219 also I am using Lattice fpga board which is LIFMD6000 and I am using Raspberry pi 3 board. I used Cmos to Dphy IP core , I embed inside the board an I configure the settings but I can not get any video output. I could not solve the problem. I am open to any idea about it. Can you help me with this issue?ReplyDelete
Gaurav your Blog is very informative. Multiple customer those used XO2 and XO3 now moving toward crosslink and Crosslink-NX Make some tutorial series which can help all the communities whether they are Beginners or Professionals in step by step manner. scratch to finish.ReplyDelete
Hi, I am using OV5640 with ESP32 for a low power project, I would love to upgrade the camera to 8MP(IMX219) only for still pictures. Is this something you are interested in? I would to hire you help us with the driver. Thanks!ReplyDelete
Interfacing OV5640 to ESP32 is comparatively easy because it has a parallel output. IMX219 is totally different class of solution need some logic and some memory. You can drop an email you feel like.Delete
Hello brother, am a mobile technician. My name is Karnvir Singh, Nd am from Punjab. My YouTube channel is Punjabi GSM Team. Am trying to find some new ways to repair or analyse the mobile devices, can I talk to you for some topics, if yes then can you plz share you whatsapp. Or if u dnt want to public ur whatsapp, can u plz ping on my whatsapp at 98888 98083. Thanks and regards Karnvir Singh. And your knowledge is awesome.ReplyDelete
I have few questions related to Soc and protocols and tools for analysing them. Mostly few confusions about debugging.Delete
If you have a question please let me know here.Delete
Hi, very nice project! I´m would like to build it up by myself to learn about the Cam to FPGA interface.ReplyDelete
Therefore I would like to order the bridge PCBs, but I couldn´t figure out what´s the frequency of the Oscillator X1 on the CAM to FPGA bridge PCB. Could you please give more information about that part?
It's 24 Mhz, all parts on my camera board are same as original Raspberry camera. Actually they were desoldered from the RPI camera board.Delete
Perfect! Thank you :)Delete
Hello Gaurav Singh,ReplyDelete
We are having a Nvidia Xavier board which has 2-lane and 4-lane mipi camera interface. we have 2 cameras of 2-lane mipi, we dont know how to interface 2-lane mipi camera with 4-lane nvidia interface. Is that the above solution will be suitable for this?
Getting camera working on linux board is totally different game. Number of lanes is totally non issue. You just need appropriate driver.Delete
This comment has been removed by the author.ReplyDelete
Would it be possible to interface a smartphone's camera CSI2 video output to the Raspberry Pi (or the Nvidia Nano) CSI2 interface?
On the LCMX03LF-6900C-S-EVN can I assume you populated the relevant resistors (100R) on the differential lines, which by default are unpopulated.
Yes LCMX03LF-6900C-S-EVN I mounted 100R resistors, board has foot prints for LVDS termination resistors. There resistor need to be on near to FPGA so need to be on the FPGA board itself.Delete
Crosslink-NX and Most FPGA which have hard DPHY have these resistors built in to the chip. But Generic FPGAs on which you are most likely using LVDS receiver, need these termination resistors.