The other day I was talking with Kyle Baley on twitter and I gave an example of how it is possible to store “highly critical” data only in memory. I mentioned to him a similar problem, here it is.
The basics of the problem are that:
we need to support 10000 concurrent queries, we can reject any past that immediately
we are given a latent but extremely high bandwith that guarantees packet ordering
we are only allowed to store 10 of the 100000 objects of the dataset in memory per instance
Feel free to email me with your potential solutions to this problem (or put them in comments).
What did I not specify that we need for the solution to work? Is there anything that would make it easier?
What is the latency of a query in your solution?
How do we handle fail-over situations?
What is the total general purpose memory needed on our machine?
How would we scale to supporting 10,000,000 concurrent queries?
What happens if we need to support 1,000,000 objects?
What happens when we can change data not just query it?
Can we implement a unique constraint on say the id of our data? If so how?
Have fun guys