Real-time Log Enrichment: perfect for my example [0], you're firing off endless tasks. RT logs have a timestamp.
Finding the First Match in a File List: Files tend to be static. I'd use a queue to first build an index and then a queue to process the index.
Time Series Data Processing: Break the data into chunks, you mention 600MB, which isn't that big at all given that Cloud Run memory maxes out at 32GB.
[0] https://news.ycombinator.com/item?id=45094387
Real-time Log Enrichment: perfect for my example [0], you're firing off endless tasks. RT logs have a timestamp.
Finding the First Match in a File List: Files tend to be static. I'd use a queue to first build an index and then a queue to process the index.
Time Series Data Processing: Break the data into chunks, you mention 600MB, which isn't that big at all given that Cloud Run memory maxes out at 32GB.
[0] https://news.ycombinator.com/item?id=45094387