Tweets From V
What’s this algorithm called? I store o…
What’s this algorithm called? I store objects in an S3 bucket with sequential numeric filenames: - bucket/ - 0000000001 - 0000000002 … - 0000004242 Each new object gets an incremented sequence number. I need a way to find the last inserted object, i.e., the object with the highest number. Since S3 can’t fetch the last inserted item directly or use reverse-sorted listing, I need to find the latest object without scanning the entire bucket of thousands of items. My idea: I search for objects at exponential intervals (1000th, 10k, 50k, 100k) in parallel. When I find a gap (e.g., 100k missing but 50k exists), I binary search that range (60k, 75k, 90k) until I narrow it to a manageable gap (5-10k objects). Then I use S3’s list API to fetch objects from that point. While I feel happy about coming up with this, I am curious to know what this kind of search is called. It looks close to binary search, but with an unknown upper bound.