Camera

A smartphone normally has multiple cameras (also named sensors) which are made by companies such as Sony, OmniVision or Samsung.

These are normally controlled via an I²C bus and deliver image data via MIPI CSI-2 (Camera Serial Interface). Some sensors are connected via a parallel interface rather than serial (CSI-2) which seems to be called digital video port (DVP). This fact is currently mostly ignored in this article and covers CSI-2 only for now. Many of the same concepts should apply to parallel also.

In many cases the sensor package also comes with additional components:
 * VCM (voice coil motor) actuator which can physically move the lens for focusing, image stabilization or optical zoom.
 * EEPROM which presumably contains some calibration information and some other details

A sensor normally has the following connections to the SoC:
 * A clock input, commonly named MCLK or XVCLK
 * Several power inputs, commonly named e.g. AVDD (analog supply), DVDD (digital supply), DOVDD/IOVCC (I/O supply)
 * One or more GPIOs for reset and/or shutdown purposes. These are not used in some setups when e.g. the reset pin is connected to DOVDD, so when the power supplies are on, the sensor is on.
 * Two pins for the I2C connection (SDA & SCL). Also named SCCB (serial camera control bus) or CCI (camera control instance or camera control interface) or just 2-wire serial communication
 * Between 3 and 9 pins for MIPI CSI-2 connection (1 pin for the clock lane, 2 pins for each of the 1-4 data lanes depending on the sensor and setup)

On SoC side the following components exist:
 * I2C bus (one or more, sometimes shared between multiple camera sensors, sometimes one camera sensor per bus)
 * Receiver of the physical MIPI CSI-2 signals (multiple in most cases)
 * Various other blocks for (efficient) image processing

Some sensors support lane remapping or lane swap for the MIPI CSI-2 connection, when the hardware design requires that. Also the sensor must be configured for the right number of lanes from the driver side (declared in  property) so that it sends the data correctly.

Sensor example
An example for a sensor is below where we provide the beforementioned clocks, supplies, gpios. The pinctrl makes sure the MCLK0 pin on the SoC is set to the correct function.

The sensor node can also provide  (front or rear camera) and   (0°, 90°, 180°, 270°) which define metadata for user space.

You can also link  and   to the VCM and flash LED respectively. This information can (or at least could) be used by user space to e.g. trigger the flash when you take a picture with the rear camera.

For more information about these properties, see.

Qualcomm
Further information: Qualcomm Camera Subsystem

Qualcomm SoCs contain a block called CAMSS (camera subsystem). For more information about some of the components that are part of it see Qualcomm Camera Subsystem driver - The Linux Kernel documentation.

A Qualcomm SoC commonly contains somewhere between two and four 4-lane CSIPHY receivers, so normally each sensor is connected to a dedicated CSIPHY and do not share them. For bringup you can focus on this CSIPHY and probably ignore the other components of CAMSS.

The I²C bus used for the camera sensors is not a regular QUP/BLSP bus but is the CCI (camera control interface) which has a separate driver in Linux.

Example
The following example shows how the camss side can be configured in dts. Each port here is a CSIPHY receiver, so the part that receives CSI-2 data. In this example CSIPHY0 and CSIPHY2 is used. CSIPHY1 is unused.

On the schematics there's 5 equal lanes per CSIPHY, where normally (e.g. for 4 data lanes) the CSI data lanes are connected to lane 0, 2, 3 and 3, while clock lane is connected to lane 1. This is represented using "clock-lanes" and "data-lanes" properties. Note: Both properties refer to the lane number and is not a count of how many lanes are used for a particular purpose. A sensor with 1 data lane would have  and   on the SoC side.

For more information about these properties, see.

Downstream devicetree
In most cases the downstream devicetree doesn't describe the actual hardware setup well but you can still extract some useful information from it.


 * csiphy-sd-index: which CSIPHY the sensor is connected to
 * cci-master: which CCI (I2C) bus the sensor sits on
 * clocks: the clock used for the sensor
 * gpios: various gpios (reset, shutdown, etc). Double-check with the names provided in the dts what function the gpio actually has. For example  would be a fixed-regulator that provides power to one of the supply rails. The gpios named   you can probably ignore here.

Bringup
Before bringing up the sensor you should make sure the I2C bus (CCI) works correctly. For this you can enable the bus in your dts and run  (needs kernel module  ) command on it. This should pretty much immediately give you the result back, probably with no devices listed since everything's turned off by default. If it hangs and you get some timeout errors in dmesg you need to make sure that the pullup for the I2C bus is enabled. This is in many cases one of the regulators used for a supply for the cameras. If you're not sure, you can try enabling various regulators you think might provide this by putting  on the regulator node.

Then for actualling getting the sensor up, you should first establish (manual) I2C communication with it. For this you enable the required regulators with  and also force the clock for the sensor to on using   +. Also don't forget to set the pinctrl correctly otherwise the clock signal won't actually reach the sensor.

An example of forcing on mclk2:

Then when you run  on the bus and see if a sensor appears. It's possible also multiple I2C addresses pop up, only one of them is actually the sensor, the others might be eeprom or VCM or others.

If you have access to a datasheet for the sensor, be careful as with any I2C device the address might be given as 8-bit address while Linux always uses 7-bit addresses. For example if the datasheet says the address is 0x20 it might actually be 0x10 instead (shift one bit to the right).

When you know which I2C address the sensor uses and it appears, you can try reading to the chip ID register to make sure communication really works and you're talking to the correct chip.

Since for some sensors 16-bit registers are used then reading is a bit more tricky manually but still easily possible. For example the IMX582 has register  which holds the value. In this example the sensor is connected to I2C bus 3 (check ) and has the I2C address.

TODO: Expand on CSI-2 stuff and what to do after i2c communication is established. ( + Some drivers in Linux support configuration of the number of CSI-2 lanes but some do not.)

Testing
TODO: introduce libcamera, manual exposure and gain with v4l-ctl, "media-ctl -p", qcam, cam, bayer format, test patterns in ISP and sensor, v4l-ctl for flash/torch, example for qcam usage that works on FP2

Apps
There are multiple camera apps available, in various states of development and functionality.


 * megapixels - good support for PinePhone, limited support otherwise
 * millipixels - fork of megapixels of Librem 5, uses libcamera
 * Camera (pinhole) - uses libaperture & gstreamer
 * harbour-pinhole - uses libcamera