Python 3 incompatibility in rios.parallel.cloud_playdoh

[gillins]

Original report by Daniel Clewley (Bitbucket: danclewley, GitHub: danclewley).

---

There are some print statements within the cloud_playdoh module and cloud_pickle.py uses a long (line 262) which causes problems under Python 3.

As module is still in progress and not part of a release have set priority as minor.

[gillins]

Original comment by Neil Flood (Bitbucket: neilflood, GitHub: neilflood).

---

Thanks Dan. As you say, this is an experimental module, and I am not even sure of there is a use case. It seems like the sort of thing which might be useful for very CPU-intensive applications, but I am unsure. What do you think ? Do you have an interest in using this for a particular application, or were you just exploring?

I assume (???) that this problem does not appear until one actually tries to use the parallel stuff?

thanks again
Neil

[gillins]

Original comment by Daniel Clewley (Bitbucket: danclewley, GitHub: danclewley).

---

It causes a problem as it is imported when you try to import applier.

I can think of some CPU intensive operations where being able to run in parallel would speed things up (e.g., fitting a model to a time series of data). I haven't actually got round to looking at any of the parallel stuff in RIOS yet though.

[gillins]

Original comment by Neil Flood (Bitbucket: neilflood, GitHub: neilflood).

---

Ah, OK, then I ought to put in something to disable it. I will make that import conditional on the setting of an environment variable, so that by default it will be disabled, and can be enabled by setting the variable. That should work around it for now.

Yes, that sort of timeseries work was the thing I had in mind when I added that module. I programmed one application with it, and it dropped the run time from 24 hours to about 3, but the method in question was not then used operationally anyway, so I have not had any other reason to test it further.

I will do something with that environment variable this morning.
Thanks again
Neil

[gillins]

Original comment by Neil Flood (Bitbucket: neilflood, GitHub: neilflood).

---

Have now replaced the explicit "L" with the suggested workaround for this issue in Python 2/3 compatability. Let me know if this fixes the problem.

[gillins]

Original comment by Daniel Clewley (Bitbucket: danclewley, GitHub: danclewley).

---

Sorry for the delay in getting back to you on this. No this still doesn't fix the problem as there are still some Python 3 incompatible parts in cloud_playdoh. I tried converting with 2to3 but there were still some problems (such as using the 'new' module).

[gillins]

Original comment by Neil Flood (Bitbucket: neilflood, GitHub: neilflood).

---

Thanks Dan. OK, I will chip away at it. Apologies for the inconvenience. Neil

[gillins]

Original comment by Daniel Clewley (Bitbucket: danclewley, GitHub: danclewley).

---

No problem. The release version has everything I need.

[gillins]

Original comment by Daniel Clewley (Bitbucket: danclewley, GitHub: danclewley).

---

Commit 98b876e fixed this problem so RIOS will now work with Python 3. Once the parallel work is finalised some testing with Python 3 will be required (suggest opening as separate issue though).

[gillins]

Original comment by Neil Flood (Bitbucket: neilflood, GitHub: neilflood).

---

Hi Dan,

thanks for checking that out. I was kind of hoping you would. :-)

cheers Neil