Feature Proposal - Capability to accept Raspberry Pi images in alternative image formats

[cryptoclidus]

While trying to figure out the best way to get data from the raspberry pi, one concern is that the jpg format of images is negatively affecting the data. I'd like to explore this topic, but with feedback because if I go ahead with it, I want to make sure it integrates nicely with existing applications (e.g. infragram, spectral workbench capture utility)

Problem Description

The Pi Cam Web Interface (which currently comes with pi-builder) image format is limited to jpeg only (although picamera library can output many formats including png and rgb arrays). Since jpg is a lossy format, is it acceptable? Do we need to have a way to access non-lossy data?

If png or rgb array data is desired, what is the best solution? Will a new program be required for the raspberry pi and if so, how will it integrate smoothly with existing applications?

Possible Approach

[] Directly compare results from jpg and png to see if the difference makes it worth it
[] If lossless data is desired, consider rgb arrays for speed optimization, and brainstorm how that might integrate with other applications

Preliminary Notes

• png encoding is substantially slower than jpg as the pi camera hardware is optimized for jpg encoding.
• "raw" rgb numpy array is available as an output in the picamera library, which is also used in opencv library
• streaming rgb arrays from the pi camera is reportedly faster than using png or jpg encoding (source: https://www.pyimagesearch.com/2015/03/30/accessing-the-raspberry-pi-camera-with-opencv-and-python/)
• the rgb format represents data after GPU processing, and before jpg or png encoding (source: https://picamera.readthedocs.io/en/release-1.13/recipes2.html#unencoded-image-capture-rgb-format)
• raw "bayer data" is also available (before GPU processing; 'straight from the sensor'), but requires more post processing to make the data usable. Should this be considered as well?

Current capability

Using the picamera library, I was able to capture images and send them over a socket connection to my pc (using python on both ends). This capability can be used for preliminary testing.

@jywarren @sidntrivedi012

[welcome]

Thanks for opening your first issue here! Please follow the issue template to help us help you 👍🎉😄
If you have screenshots to share demonstrating the issue, that's really helpful! 📸 You can make a gif too!