Advanced Search

Question: Grid problem when using module. Does not see local module variable

Question:Grid problem when using module. Does not see local module variable

nm 10783 Maple 2024
module grid local
February 07 2025
2

I am calling a proc inside a module, where this module had local module variable initialized to false;

This works fine when calling the proc normally.

When calling it from Grid:-Run() it says it is not seeing the module local variable at all.

Why does this happen and is there a workaround this? Here a simple worksheet showing this.

I hope there is a way to make the proc inside the module see the local module variables when using it from Grid node. Otherwise, this whole thing will not work for me, as I have few variables initialized at modules level in number of places.

> 

restart;
> 

A := module()
  local DEBUG_MSG::truefalse:=false;
  export work_proc:=proc(n::integer)
     print("My flag is ",DEBUG_MSG);
  end proc;
end module;
> 

 

_m1690230418240

> 

#THis works as expected
A:-work_proc(0)

"My flag is ", false

> 

#this does not work. THe call works OK, but inside A:-workproc() it does not see module local variables.
> 

Grid:-Set(A:-work_proc):
Grid:-Run(0,A:-work_proc,[ 0 ]);
Grid:-Wait();

"My flag is ", DEBUG_MSG

 

 

Download grid_with_module_local_variable_feb_7_2025.mw


Update 2/7/2025

I tried to change the access to the module local variables from the module local procs, by adding explicit A:- to each variable name where A here is the module name.

This works for normal calls, but not when using Grid to call the proc.

This workaround fail, it gives error that module does not export `%1`

May be I have to redo all my code not to use local module variables. But in some places I have to do this, in order to detect if something has happened before or not. I use module local variables to store state the presist after call is completed.

It looks like Grid is not meant to be used for calling proc() that uses/lives inside modules. But this makes Grid not very useful then for large application.

> 

restart;
> 

A := module()
  local DEBUG_MSG::truefalse:=false;
  export work_proc:=proc(n::integer)
     print("My flag is ",A:-DEBUG_MSG);
  end proc;
end module;
> 

 

_m1690230418240

> 

#THis works as expected
A:-work_proc(0)

"My flag is ", false

> 

#this does not work. THe call works OK, but inside A:-workproc() it does not see module local variables.
> 

Grid:-Set(A:-work_proc):
Grid:-Run(0,A:-work_proc,[ 0 ]);
Grid:-Wait();

Error, (in work_proc) module does not export `%1`

 

 

Download grid_with_module_local_variable_feb_7_2025_V2.mw

I also tried to make the module variable as export instead of local, and that also did not work. I am starting to run out of ideas what else to try...

> 

restart;
> 

A := module()
  export DEBUG_MSG::truefalse:=false;
  export work_proc:=proc(n::integer)
     print("My flag is ",A:-DEBUG_MSG);
  end proc;
end module;
> 

 

_m1690230418208

> 

#THis works as expected
A:-work_proc(0)

"My flag is ", false

> 

#this does not work. THe call works OK, but inside A:-workproc() it does not see module local variables.
> 

Grid:-Set(A:-work_proc):
Grid:-Run(0,A:-work_proc,[ 0 ]);
Grid:-Wait();

Error, (in work_proc) `%1` does not evaluate to a module

 

 

Download grid_with_module_local_variable_feb_7_2025_V3.mw

Update

I think now that using module level local variables will not work with parallel processing anyway. It is like using global variable in parallel processing. Does not work without synchorization of access to this shared variable. 

So this means I have to change my code and not use any module local variables, and pass any information between functions via arguments only.

Using module local variable is more convenient, but I now realize this design is not good for parallel processing.  This means some code changes I have to do. 

On the positive side, I find using Grid can really speed things up. On some tests, up to 10 times faster. The larger the number of problems to process, the more speed up is gained.

So I think it is worth to rewrite my code and remove any use of local level module variables.

Update Feb 11, 2024

Found the solution!  With grid, one must pass in all variables that to be used on a node. So the above now becomes this

> 

restart;
> 

A := module()
  export DEBUG_MSG::truefalse:=false;
  export work_proc:=proc(n::integer)
     print("My flag is ",DEBUG_MSG);
  end proc;
end module;
 

_m1739526597408

> 

#THis works as expected
A:-work_proc(0)

"My flag is ", false

> 

#this does not work. Because we did not pass all variables to be used at node
Grid:-Set(0,A:-work_proc);
Grid:-Run(0,A:-work_proc,[ 0 ]);
Grid:-Wait();

"My flag is ", DEBUG_MSG

> 

#this now worksheetdir
Grid:-Set(0,A:-work_proc,'A:-DEBUG_MSG');
Grid:-Run(0,A:-work_proc,[ 0 ]);
Grid:-Wait();

"My flag is ", false

 

 

Download grid_with_module_V4_feb_11_2025.mw

Actually the above could be done easier this way. By passing in the name of the whole module A, now nodes are able to see all module A local vaiables. Like this

> 

restart;
> 

A := module()
  export DEBUG_MSG::truefalse:=false;
  export work_proc:=proc(n::integer)
     print("My flag is ",A:-DEBUG_MSG);
  end proc;
end module;
 

_m1739526597408

> 

#THis works as expected
A:-work_proc(0)

"My flag is ", false

> 

#this does not work. Because we did not pass the name of the A
Grid:-Set(0,A:-work_proc);
Grid:-Run(0,A:-work_proc,[ 0 ]);
Grid:-Wait();

Error, (in work_proc) `%1` does not evaluate to a module

> 

#this now works, becuase we pass the module A
Grid:-Set(0,'A:-work_proc','A');
Grid:-Run(0,A:-work_proc,[ 0 ]);
Grid:-Wait();

"My flag is ", false

 

 

Download grid_with_module_V6_feb_11_2025.mw

And to be able to access global variables in another module from inside the worker module, we must pass the other module name. Here is an example

> 

restart;
> 

B := module()
  export some_value::integer:=10;
end module;

A := module()
  export work_proc:=proc(n::integer)
     print("the value is ",B:-some_value);
  end proc;
end module;
 

_m1739526597280

_m1739594742272

> 

#THis works as expected
A:-work_proc(0)

"the value is ", 10

> 

#this does not work. Because we did not pass the name of the other module B in this case
Grid:-Set(0,A:-work_proc);
Grid:-Run(0,A:-work_proc,[ 0 ]);
Grid:-Wait();

Error, (in work_proc) `%1` does not evaluate to a module

> 

#this now works, becuase we pass the module name B to the node, so it sees it
Grid:-Set(0,'A:-work_proc','B');
Grid:-Run(0,A:-work_proc,[ 0 ]);
Grid:-Wait();

"the value is ", 10

 

 

Download grid_with_module_V5_feb_11_2025.mw

The main point, if one is using modules with Grid, names of all modules to be accessed from a node need to be passed in in the Set call. Otherwise the node will not see them.

Update For the above method to work, all module variables to be accessed at node, have to export and not local.

Update 2/11/2025  4 AM

I think I'll give up on Grid. It is completely useless. I found out that I have to make all my modules proc's export now, even though they are only used in that one module, just in order to call them from same module when running in Grid node.

Otherwise, I have to list each proc name in each module and sub module in the Set command. This is ridiculous design. I have 5,000 proc's. I am not going to list them all each time.

Here is an example where a boo() is local proc to a module. But it is not called, because the proc is local.

> 

restart;
> 

A := module()

  export foo:=proc()
       print("entering A:-foo(), before calling local proc boo()");
       boo(); #this call will not work in Grid setting
  end proc;

  local boo:=proc()
     print("inside local A:-foo()");
  end proc;
end module:
> 

A:-foo();

"entering A:-foo(), before calling local proc boo()"

"inside local A:-foo()"

> 

Grid:-Set(0,A:-foo,'A');
Grid:-Run(0,A:-foo,[0]);
Grid:-Wait();

"entering A:-foo(), before calling local proc boo()"

> 

 


 

Download calling_proc_in_module_V1.mw

To make it work, I have to make all my local proc export, and not only that, I have to call it using A:-boo() as well. So the above becomes


 

> 

restart;
> 

A := module()

  export foo:=proc()
       print("entering A:-foo(), before calling local proc boo()");
       A:-boo(); #this call will not work in Grid setting
  end proc;

  export boo:=proc()
     print("inside local A:-foo()");
  end proc;
end module:
> 

A:-foo();

"entering A:-foo(), before calling local proc boo()"

"inside local A:-foo()"

> 

Grid:-Set(0,A:-foo,'A');
Grid:-Run(0,A:-foo,[0]);
Grid:-Wait();

"entering A:-foo(), before calling local proc boo()"

"inside local A:-foo()"

> 

 

 

Download calling_proc_in_module_V2.mw

I am not going to edit 100,000 lines of Maple code and change all the procs to export and make sure each call have the fully qualified module name attched to it even though it is local to the module.

I will not use Grid. It simply does not work with modules. Maplesoft should fix Grid so code works as with minimal changes.
Please Wait...