What should these settings be if I want it to update blob storage in real-time/hot or as close to it as possible? It is sporadic. Sometimes it is fast and sometimes it can take many hours before it refreshes. I have the Data Lake as an input to an Azure Stream Analytics job so I need it to be fast consistently.
Not sure if it might be the flush size min of 1000. I’m only testing with a few messages. But that doesn’t explain why some go through fast and some take a long time to refresh the hour folder. I see a new entry created with zero bytes but the data don’t get added to the json files fast enough to trigger the ASA job.
