Better clean up of blobs

Start by tracing the blob HTTP GET path and the request data that can indicate whether a blob should persist. Review the separate programming-client and UI cases described in the issue, including short and long-running tasks. Done means a GET removes the blob by default while an explicit persistence request preserves it, without breaking clients that need the data.

Blobs currently live for a fixed time on the server. This means that any blob that creates a sizeable chunk of data that is run often could waste a fair amount of space.

Proposal

Delete the blob when it is read with GET, unless the data attached to the specific GET call specifies that the blob should remain.

Effects

The HTTP API tends to be used for two use cases - The UI, programming clients. It is worth considering these separately.

From a Programming client

I cannot think of many cases where a programming client would want to get the data multiple times (or multiple clients get the data). I don't deny there may be a specific use case, but the ability to request that the blob persists allows someone writing a client to do this if there edge case allows.

UI

I think it is worth considering short and long running tasks separately.

• Short tasks - (For example: Capturing an image)
  • If I have a client open and capture an image I don't expect a different client to be able to access it
  • If I select capture and then close the client before it finishes, I don't expect I will ever get that image (It will in this case sit on the server for 5 mins)
  • If I have 2 clients open, or open a second one mid-capture I don't expect the second client should receive the image as well
• Long-running task - (For example: Sample scanning) - Returning a blob for such a long running task is a bad idea in the first place as there is a high chance that no client is still open to receive the blob on completion. We will also always want the UI to reconnect to long-running actions, which means that the chance of multiple clients being sent a blob is high, which always seems like it is confusing.

All of these cases imply that the blob being deleted on first GET is fine.

For the short task we probably (In OpenFlexure Microscope) will need to make sure that short tasks don't auto-connect to ongoing actions as a second client could receive the image of captured by another client. This is already weird behaviour, but would be doubly bad if this issue is implemented as the client that captured may not actually receive the requested image.