Resource Manager

Resource Manager is required only if a computational cluster is used in the system. Its logic is also straightforward and directly depends on the policies of the cluster used. We tested our framework on the ​Millipede HPC cluster. This cluster has a policy that any cluster job execution should not exceed the ten days limit to assure availability of cluster resources to all users. This means, that Resident Worker cannot run longer that ten days either. In our current implementation to keep a cluster as a part of our computational cloud, Resident Worker starts a new Resident Worker node in some time before it will be removed by the cluster administrator, e.g. two days before the end of a ten-days period. The script to starting a new Resident Worker is submitted to the cluster scheduler and processed in some time depending on a cluster load. Hence, we assure that at least one Resident Worker is running on the cluster.

So, Resource Manager is a simple scheduler implemented using the java.util.Timer class

public class ResourceManager
{
     ...
    
    private Timer timer = new Timer();

    private ExecutorService exec = null;
    private RemoteGridGainStarter remoteGridGainStarter = new RemoteGridGainStarter();

    public void start()
    {
        timer.scheduleAtFixedRate(new TimerTask()
        {
            public void run()
            {
                Future<String> future = exec.submit(new RemoteGridGainStarter());

                try
                {
                    String result = future.get();
                    System.out.println(result);
                }
                catch (InterruptedException e)
                {
                    e.printStackTrace();
                }
                catch (ExecutionException e)
                {
                    e.printStackTrace();
                }
            }
        }, delay * 24 * 60 * 60 * 1000, period * 24 * 60 * 60 * 1000);
    }

As you can see in this, Resource Manager repetitively submits RemoteGridGainStarter for execution. The code Remote Grid Gain Starter depends on the cluster policy and in this example, it looks as

public class RemoteGridGainStarter implements Callable<String>, Serializable
{
    private static final String SUB = "qsub -q nodeslong /data/byelas/scripts/gridgain/resident_worker.sh";

    public String call() throws Exception
    {
        System.out.println(">>> start another gridgain");

        SysCommandExecutor cmdExecutor = new SysCommandExecutor();
        cmdExecutor.runCommand(SUB);
        String cmdOutput = cmdExecutor.getCommandOutput();

        return cmdOutput;
    }
}

Back to ​Examples