Version 5 of CIP - calculations in parallel

Updated 2023-01-10 16:55:41 by saito

Arjen Markus (7 january 2023) The break during the holiday season allowed me to pick up an old project again: a way to define a bunch of calculations and have them run partly simultaneously and partly sequentially, so that you use the computing resources efficiently. The main steering program was complete and even documented, but I thought it would be nice to have some more supporting programs:

  • A program to prepare the input files based on a simple table
  • A program to report on the variations
  • Various modes for the first program that the reporting program should act upon

Well, I was able to finish a lot of what I had in mind with the exception of the additional documentation. So, this page is mostly a placeholder :) at this moment. But I wanted to publish it now.

saito (6 january 2023) Sounds interesting. I can see how it relates to some of the previous comments on the Tasks page. It might be helpful to show a specific use case where this is used to solve a problem for you. Not necessarily the code you have developed but its usage, as in: calculation-a does this, calculation-b does this, this is how they are related and/or how they vary, and here is how they should all fit together, using your terminology above.


arjen - 2023-01-08 10:33:38

A quick answer to your question: one application of this would be the Morris method for examining the sensitivity of parameters in a numerical model. The idea is to run a set of simulations, each with a slightly different value for a parameter (but the variation accumulates). The set of programs I have makes this very easy, including the subsequent overview of the result: you can simply specify what parameters to change and the "cip-prepare" program generates the input. There are of course a bunch of conventions, but they should be straightforward to meet.


saito - 2023-01-10

Thank you for the comment - I see that you have also published the code. I can see the similarities and the differences between this and the Tasks, and what challenges you aimed to solve with your code. It is interesting to see how different people approach a basically identical problem and the resulting differences in their solution. In any case, this topic is an area of interest to me. I have done something very similar to this in my data flow code.


arjen - 2023-01-08 15:35:40

I published the code with documentation and an example in its current incomplete state on Github - https://github.com/arjenmarkus/cip-git .