As hardware is complicated, Hardware Abstraction Layers, which are used for higher-in-the-stack applications to use, are needed.
Recently the "libcamera" library was started to basically create a HAL for camera APIs. http://libcamera.org/
A "libsensor" would also be great, handling IIO sensors and sensors on the DSP via qmi/libqmi. For libsensor there are many projects implementing something like that (excluding the qmi part): https://github.com/hadess/iio-sensor-proxy , https://git.merproject.org/mer-core/sensorfw/tree/master , https://github.com/analogdevicesinc/libiio , https://doc.qt.io/qt-5/qtsensors-index.html and probably many more