Running on a cluster of computers for NetLogo models

The asynchronous search techniques can be remarked by the existence of a very large number of elements that can be introduced, without affecting the completeness of the algorithm. For example, processing the message in packets or individual, storage or not of the nogood messages, message filtering, nogood learning and storing them for each value. Thus, a correct evaluation assumes a large number of runs, with different data sets. We present an open-source solution for implementation and evaluation of the asynchronous search techniques in NetLogo, for a great number of agents, model that can be run on a cluster of computers. In this page we will present a methodology to run the proposed NetLogo models in a cluster computing environment or on a single machine [1][2].

We utilize the Java API of NetLogo as well as LoadLeveler. LoadLeveler is a job scheduler written by IBM, to control scheduling of batch jobs. This solution is not restricted to operate only in this configuration, it can be used on any cluster with Java support and it operates with other job schedulers as well (such as Condor). Such a solution will allow running a large number of agents (nodes, variables,
robots, queens, etc.). The first tests allowed running of as much as 500 agents, in the conditions of a high density constraint graph. The first experiments were done on the InfraGrid cluster from the UVT HPC Centre [3], on 100 computing systems (an hybrid x86 and NVIDIA Tesla based). InfraGRID is an Linux only cluster based on a mixture of RedHat Enterprise Linux 6 and CentOS 6. For Workload Management, JOB execution is managed at the lowest level by IBM LoadLeveler.

The methodology proposed in the previous pages that uses the GUI interface will run on a single computer. In this paragraph we will present a new solution, without GUI, that can run on a single computer or on a cluster. The proposed approach uses the NetLogo model presented previously, runnable without the GUI, with many modifications. In order to run the model in that manner is used a tool named BehaviorSpace, existent in NetLogo. BehaviorSpace is a software tool integrated with NetLogo that allows you to perform experiments with models in the “headless” mode, that is, from the command line.without any graphical user interface (GUI).

BehaviorSpace runs a model many times, systematically varying the model’s settings and recording the results of each model’s run. Using this tool we develop an experiment that can be runned on a single computer (with a small number of agents) or, in the headless mode on a cluster (with a large number of agents). The steps necessary for the implementation of a multiagent system are as follows:

  1. (S1). Create a NetLogo model according to the previous model for the asynchronous search techniques and for the types of problems used at the evaluation. For running it on the cluster and without GUI some adaptations have to be made. First, the NetLogo model must have a procedure called setup to instantiate the model and to prepare the output files. For details, see [1][2].
  2. (S2). Create an experiment using BehaviorSpace and parse the NetLogo file into an input XML file (so that it can be runned in the headless mode, that is without GUI). To finalize the run and adding up the results it is recommended the use of a Netlogo reporter and a routine that writes the results. The run stops if this reporter becomes true.
  3. (S3) Create a Linux shell script (in sh or in bash) that describes the job for LoadLeveler. Once the Netlogo model is completed with the experiment created with the BehaviorSpace tool, it is time to prepare the system for multiple runs. To do this, first create a script that allows running with no GUI, an example script is presented in next figure:.

#!/bin/sh

# @ error = logs/$(job_name).$(Host).$(Cluster).$(Process).err
# @ output = logs/$(job_name).$(Host).$(Cluster).$(Process).out
# @ job_type = serial
# @ class = medium
# @ environment = COPY_ALL
# @ job_name = ABT
# @ queue = 10

. /etc/profile

export

if [ -z "${LOADL_STEP_ID}" ]; then
    llsubmit $0
    exit 0 #
else

    mkdir -p infragrid/output/
    mkdir -p inputs/
    echo "Step Number: $LOADL_STEP_NUMBER"
    #gen_experiment > inputs/experiment_$LOADL_STEP_NUMBER
    #./../netlogo-headless.sh --model ABT-template/Templates-ABTk-brp-max-card-running-cluster.nlogo --experiment experiment
   ./../netlogo-headless.sh --model ABT-template/Templates-ABTk-brp-max-card-running-cluster.nlogo --experiment experiment

fi

netlogo-headless.sh

 

 

[1]. Ionel Muscalagiu, H.E. Popa, Jose Vidal. Large Scale Multi-Agent-Based Simulation using NetLogo for implementation and evaluation of the distributed constraints. In proceedings of IJCAI – DCR 2013 (23rd International Joint Conference on Artificial Intelligence - Workshop on Distributed Constraint Reasoning ), Beijing, August 2013. Available: http://www.cs.nmsu.edu/~wyeoh/DCR2013/index.html.

[2]. Ionel Muscalagiu, H.E. Popa, Jose Vidal. Clustered Computing with NetLogo for the evaluation of asynchronous search techniques. In proceedings of 12th IEEE International Conference on Intelligent Software Methodologies, Tools and Techniques (SOMET 2013), pag. 115-120, September,  Budapest 2013.

[3]. InfraGRID Cluster. Available:http://hpc.uvt.ro/infrastructure/infragrid/.

Templates in DisCSP-Netlogo for the Asyncronous Search Technique

Title Description Last Modified

Templates for running on a cluster of computers:
Templates-ABTk-brp-max-card-running-cluste
by Ionel Muscalagiu, Jose M. Vidal and Horia Popa Emil

This is the implementation of the Asynchronous Backtracking algorithm for the random binary problems in NetLogo (5.1).

This is an example for running on a cluster of computers.

 Octomber 20th, 2014
Template-script.sh
by Ionel Muscalagiu

This is the examples of Linux shell script that describles the job for LoadLeveler

 Octomber 20th, 2014
ABT-template-cluster.rar
by Ionel Muscalagiu

All the necessary files

 

 Octomber 20th, 2014
Last modified Monday, 20 octomber 2014, 11:50AM. Send comments to Ionel Muscalagiu