Design to generate a Fixed-Rate flow
Hello,
I have a Java client that has to issue requests at fixed rate to a remote server.
The original programmer did it genuilely (and monothreaded):
publicclass Client{
public Client(int rate){
this.delay = ...;//conversion and rounding algorithm omitted;
}
publicvoid sendRequests(){
while (...){
serverProxy.sendRequest(...);
Thread.sleep(delay);
}
}
}
Obviously this doesn't take into account the time it takes to perform the requests themselves, and the jitter of this duration (not fixed as it depends on remote server load,...).
The client was updated to:
publicvoid sendRequests(){
while(...){
long nextTime = System.currentTimeMillis() + delay;
serverProxy.sendRequest(...);
long sleepTime = nextTime - System.currentTimeMillis();
if (sleepTime>0)
Thread.sleep(sleepTime);
}
}
However this doesn't guarantee that the client will be able to respect the rate, even though there is spare CPU: if a request takes too long, the thread won't sleep but the next request is sent late.
Obvisouly we can augment the "scalability" by flinging more threads at work: we could have several instances of Client executed by a bunch of threads, each thread asking a shared object whether it is time to send a new request (code omitted, active wait or awkward wait/notify scheme).
But again at some point, a fixed number of threads will eventually reacha limit (e.g if each request takes 200ms and the number of threads is 100, the client program won't scale above 500 req/s; and if the requests start taking longer...).
Eventually I'd like to make an adaptive scheduler do the job of scaling the threadpool:
I'd write a Scheduler class that triggers aClient.send() method at the fixed rate. The Client methods would no more handle timing, and just issue individual requests.
The Scheduler would use an extensible thread pool, automatically firing a new thread if all existing threads are already busy, or reusing idle threads otherwise.
The limiting factor now, would be the CPU or memory of the client JVM (hopefully the server app will hit its wall before), but not the artificially serial algorithm.
This solution seems awkwardly familiar. I guess it is quite a common algorithmic problem, and possibly such scheduling libraries already exist. Do you know any such library?
Otherwise, do you foresee a problem in me implementing this scheduling?
Thanks in advance.
J.
P.S.: my "client app" is indeed a proprietary throughput test tool, and I'm already looking for standard or off-the-shelf tools instead (see http://forum.java.sun.com/thread.jspa?threadID=737887). But if I have to follow the proprietary route, I need to have this fixed-rate issue solved...
