User account

A user account on the grass machine (or locally) can be provided to anyone that wants to execute the tutorial.

For more information, please contact j.borgdorff@uva.nl.

Part 1: installing and testing

In this assignment you will download MUSCLE 2.0 and compile it on the grass machine in Poznan. To run the tutorial on your own machine at a later stage (or instead of the following instructions), follow the instructions on the MUSCLE installation page.

Go to http://apps.man.poznan.pl/trac/muscle, containing the installation guide, documentation and MUSCLE packages. We will not download it from the webpage, but directly on the grass machine.
Log in to the grass machine using the provided log-in credentials with a terminal (Linux and Mac OS X) or with PuTTY (Windows, be sure to enable X11 in the X11 tab on the left).
```
ssh [mschoolXX]@grass1.man.poznan.pl -X
```
Once logged in there, download and unzip MUSCLE-2.0.1-sources.zip.
```
wget http://www.mapper-project.eu/documents/10155/44992/MUSCLE-2.0.1-sources.zip?version=1.0
unzip MUSCLE-2.0.1-sources.zip
```
you can then build and install it to ~/muscle with
```
cd MUSCLE-2.0.1/build
./build.sh ~/muscle install
```
All prerequisites described on the installation page are already installed on the grass machine.

To add MUSCLE to the path and set a few environment variables, run the following (don’t forget the dot at the start!):
```
. ~/muscle/etc/muscle.profile
```
Now you can run muscle2. We will show its functionality with an example model that is packaged with MUSCLE, which is specified in src/cxa/SimpleSubmodelExample.cxa.rb. To show the submodels that the example contains, run:
```
cd ..
muscle2 --cxa src/cxa/SimpleSubmodelExample.cxa.rb
```
This example has a writer submodel w and a reader submodel r. The writer sends data to the reader, and the reader prints that data to screen.

You can run the simple example by specifying that the current MUSCLE command will run the Simulation Manager with the --main flag or its shorthand -m, and specifying all submodels that should run in this command:
```
muscle2 -mc src/cxa/SimpleSubmodelExample.cxa.rb w r
```
Usually, if you want to run all the submodels instances with the current MUSCLE command, specify the --allinstances or its shorthand -a:
```
muscle2 -amc src/cxa/SimpleSubmodelExample.cxa.rb
```
So what do we observe? MUSCLE detects that it should start the Simulation Manager and a Simulation. It first starts the manager, which shows its TCP/IP address. Then the Simulation starts and it executes the submodel instances w and r. These start their respective conduits, and start ‘computing’.

On local machines, these commands are sufficient to run any MUSCLE model. You can try the other examples in the src/cxa directory, just run
```
muscle2 -amc src/cxa/[...].cxa.rb
```
Scientific applications that are started remotely generally do not have a direct feedback to a user interface, but we will run one that does. This is why we logged in with the -X flag of SSH in step 2.
```
muscle2 -amc src/cxa/LaplaceExample.cxa.rb
```
It will open two windows that compute temperature dissipation, that are coupled on the boundary. Red is a +1 temparature, blue a -1 temperature, and green a zero temperature. The initial condition, far right and far left correspond to zero temperature, the top to a sine function and the bottom to a cosine. Temperature is dissipated during each time step by taking the average of the four neighboring cells. The adjacent boundary is transmitted each iteration.
Take a look at all the options by running
```
muscle2
```

Part 2: distributed execution

In this part we will do an inter-process and an inter-machine run of MUSCLE.

Open a second terminal, and also log in to grass1. When logged in do
```
. ~/muscle/etc/muscle.profile
cd MUSCLE-2.0.1
```
Now we can run muscle on the same site. In the first terminal window, start the main MUSCLE with the writing submodel w. This is done by typing:
```
muscle2 -mc src/cxa/SimpleSubmodelExample.cxa.rb w
```
So what do you observe?

In the second terminal, start the reading submodel r, make sure you replace 90XX with the port number mentioned for the Simulation Manager in the other terminal window..
```
muscle2 -c src/cxa/SimpleSubmodelExample.cxa.rb r --manager localhost:90XX
```
What happens in both terminals?
To isolate the output from the Simulation Manager, we can run it separately.

In the first terminal window, start the Simulation Manager:
```
muscle2 -mc src/cxa/SimpleSubmodelExample.cxa.rb
```
In the second terminal, start the submodels. Again, replace the port number with the one that is printed by the Simulation Manager.
```
muscle2 -ac src/cxa/SimpleSubmodelExample.cxa.rb -M localhost:90XX
```
What happens in both terminals?
In the second terminal, now log in to grass2
```
ssh mschoolXX@grass2
. ~/muscle/etc/muscle.profile
cd MUSCLE-2.0.1
```
We will perform the same experiment, but now on different hosts. So on grass1, for instance run only plumber
```
muscle2 -mc src/cxa/SimpleSubmodelExample.cxa.rb w
```
On grass2, we now also need to specify which host and port the other MUSCLE instance is running on
```
muscle -c src/cxa/SimpleSubmodelExample.cxa.rb -M grass1:90XX r
```
What is the output in grass1? And in grass2?

Related: inter-cluster executions can be done with the --intercluster tag, with which the MUSCLE transport overlay (MTO) is used. We will not be doing this directly, because we have only one cluster arranged. In the MTO documentation this is explained more thoroughly.

Part 3: The power of coupling

First, download an extension of the LaplaceExample that was shown in item 1.4.

wget http://www.mapper-project.eu/documents/10155/44992/laplace.zip?version=1.0
unzip laplace.zip
cd laplace
ant

The source of the submodels can be found in src/laplace. This example code shows how the same code can be reused in different scenarios:

Coupling as in example 1.4 - run muscle2 -amc laplace.cxa.rb
Doing some simple transformation with filters after sending the data - muscle2 -amc laplaceFilter.cxa.rb
Setting one boundary explicitly using terminals - muscle2 -amc laplaceTerminal.cxa.rb
Using a periodic boundary condition - muscle2 -amc laplacePeriodic.cxa.rb
Doing domain decomposition using mappers - muscle2 -amc laplaceMapper.cxa.rb

Their coupling is shown in the following diagrams:

Scenarios

Try the different scenarios and see what the differences are between the CxA files.
Modify the configuration files to get different geometries or transformations. For example (pick one):
- add a submodel instance so that you get a coupling between West - Middle - East;
- change the filters and see how it affects the computations (see Filter documentation); or
- modify the file src/laplace/filter/InvertFilter to do something additional (see Java filter documentation)
- modify the file src/laplace/terminal/DoubleArraySource.java to do something additional, such as computing a function instead of sending a fixed number (see Java terminal documentation)

Part 4: a diagram of your tightly coupled model

To apply this programming paradigm to your own multiscale model, you can apply the methodology as presented on day 1.

Identify the processes in your problem, and their scales. Draw them on the same diagram, taking time as a x-axis, and space as the y-axis. Each process should have a granularity (step size), which is the lower bound for the rectangle, and a problem size (total size), which is the upper bound for the rectangle.
Draw arrows between interacting processes. Now your scale separation map is more or less complete.
When you think about implementation of that scale separation map, could you use any of the coupling types shown in part 3? Would it be tightly or loosely coupled?
Work out coupling diagram of your own application.

Part 5: modifying the simple example (optional)

To start on using MUSCLE for your own applications, you can start by modifying the examples. Depending on the programming language of your application, you can start with (Java documentation) the files in src/java/examples/simplesubmodel or with (C++/C documentation) src/cpp/examples/simplecpp or (Fortran documentation) src/cpp/examples/simplefortran. The last two only contain the Sender (the submodel w) of the simple example.

Modify src/java/examples/simplesubmodel/ConsoleWriter.java so that it sends a message back to Sender.java; modify Sender.java to receive this message. In this example, call both the sending and the receiving port “messages”, for example. Test your implementation by running
```
cd build
./build.sh ~/muscle
cd ..
muscle2 -amc src/cxa/SimpleSubmodelExample.cxa.rb
```
Spoiler:
- In ConsoleWriter.java
  - create the method protected void intermediateObservation()
  - send some data over it in execute() by adding
```
double[] msg = {1.0, 0.0, 1.0};
out("messages").send(msg);
```
- In Sender.java
  - Create the method protected void solvingStep()
  - Receive the data
```
double[] msg = (double[])in("messages").receive();
```
  - Print the data with
```
for (double d : msg) {
     log("Message from ConsoleWriter: " + d);
}
```
- In the configuration file src/cxa/SimpleSubmodelExample.cxa.rb
  - Append the lines
```
cs.attach('r' => 'w') {
    tie('messages', 'messages')
}
```
  - Since the names of the ports are equal (messages), you can also use the abbreviated form:
```
cs.attach('r' => 'w') {
    tie('messages')
}
```
- Build it and execute

Modify src/cpp/examples/simplecpp/Sender.cpp to receive a message from ConsoleWriter

cd build
./build.sh $HOME/muscle
muscle2 -amc ../src/cxa/NativeExample2.cxa.rb

Spoiler:

In Sender.cpp

Receive some data after sending by coding in the main() function

size_t sz;
double* msg = (double *)
    muscle::env::receive("messages", (void *)0, sz, MUSCLE_DOUBLE);

Print the data with

for (size_t j = 0; j < sz; j++) {
    logger::info("Message from ConsoleWriter: %f", msg[j]);
}

Free the data with

muscle::env::free_data(msg, MUSCLE_DOUBLE);

In the configuration file src/cxa/NativeExample2.cxa.rb

Change the line

cxa.add_kernel('r', 'examples.simplejava.ConsoleWriter')

cxa.add_kernel('r', 'examples.simplesubmodel.ConsoleWriter')

Append the lines

cs.attach('r' => 'w') {
    tie('messages')
}

If you would prefer to use the free-form API of MUSCLE, take a look at the SimpleExample, this doesn’t implement the different formal operators of MML, but just an execute() method.