Hello,

I need to control an external program (MarvelousDesigner 10) from within Poser12.

A bit of explanation: Like Poser, MarvelousDesigner migrated to Python 3 (.7.9) and they dropped the support for pythonQt, which i used for the interface of MDBridge for Poser. I therefore need to develop an altenative. The provided Python version is not complete, no tkinter and no pip.

Best solution is to go where I feared to treat when I made the MDBridge: Remote Process Calls. This works fine. I can set up a SimpleXMLRPCServer in MD10 and corresponding client in Poser and let functions in MD do simple things and return results. I can also, from Poser, tell the server to stop listening. Problem is that MD locks up completely when the server is waiting for input. I would like to let the script also listen to the keyboard and stop the server from listening when I press a key.

This is my test server in MD10:

import datetime
from xmlrpc.server import SimpleXMLRPCServer
import xmlrpc.client

quit = 0

def today():
    today = datetime.datetime.today()
    print("Hij vraagt hoe laat het is")
    return xmlrpc.client.DateTime(today)

def whoareyou():
    print("Hij vraagt hoe ik heet")
    return "I am MarvelousDesigner 10"
    
def zeggesA():
    print("ik moet A zeggen")
    return "A"

def telop(a,b):
    print("hij vraagt me iets op te tellen")
    return a+b

def kill():
    global quit
    quit = 1
    print("Hij zegt dat hij me niet meer nodig heeft")
    return 1

server = SimpleXMLRPCServer(("localhost", 8000))
print("Listening on port 8000...")
server.register_function(today, "today")
server.register_function(whoareyou, "wiebenjij")
server.register_function(kill, "kill")
server.register_function(zeggesA, "zeggesA")
server.register_function(telop,"telop")
# server.serve_forever()
while not quit:
    try: 
        server.handle_request()
    except KeyboardInterrupt:
        quit = 1

this is the client:

import xmlrpc.client
import datetime


proxy = xmlrpc.client.ServerProxy("http://localhost:8000/")

today = proxy.today()
# convert the ISO8601 string to a datetime object
converted = datetime.datetime.strptime(today.value, "%Y%m%dT%H:%M:%S")
print("Today: %s" % converted.strftime("%d.%m.%Y, %H:%M"))

wbj = proxy.wiebenjij()
print(wbj)

# proxy.kill()

the client calls 'today' and 'wiebenjij' to show the link was made. The other functions, including 'kill' can now be called from the Python shell.

The construct at the end of the server:

while not quit:
    try: 
        server.handle_request()
    except KeyboardInterrupt:
        quit = 1

I saw in some examples. It appears however that the keystrokes do not reach the script. What's the best solution here?

Fascinating. So it seems that a PoserPython script could, theoretically, have Poser call Photoshop - and have Photoshop load/composite the render-set that Poser had just made? Am I correct on that?

HartyBart posted at 11:27PM Thu, 10 December 2020 - #4407031

Fascinating. So it seems that a PoserPython script could, theoretically, have Poser call Photoshop - and have Photoshop load/composite the render-set that Poser had just made? Am I correct on that?

If you can manage to have a server run inside of Photoshop: Yes.

@FVerbaas

You run the server inside of MD. But this server does blocking. If you want to change that: Check "async" and "non-blocking" in Python 3.

Here is something that may help: https://realpython.com/async-io-python/

There is a good chance you can't manage that while running an embedded script (one inside MD). You may try to let Poser do the the server part.

Maybe a better solution is to go with pipes (at least if both apps run on the same machine). Pipes are fully supported with Python 3.

Here is a start: https://www.python-course.eu/pipes.php

Windows and named pipes:

https://stackoverflow.com/questions/48542644/python-and-windows-named-pipes

Here is a simple test if async runs with MD:

import asyncio

async def main():
    for word in "Test if this runs in MD, word for word, with 1 sec in between".split():
        print(word, end=" ")
        await asyncio.sleep(1)

asyncio.run(main())

Text is written word for word with 1 second delay between words. If not, async won't work and so no Python standard server.

If the test works, read this:

https://docs.python.org/3.7/library/asyncio-task.html#coroutine

and this:

https://docs.python.org/3.7/library/asyncio-stream.html

MD10 can import asyncio, but the text appears in one go, including the decorative texts I put around the call to see whether anything is happening.

import asyncio

print(' before defining main and unning asyncio \n')

async def main():
    for word in "Test if this runs in MD, word for word, with 1 sec in between".split():
        print(word, end=" ")
        await asyncio.sleep(1)


asyncio.run(main())
print(' \n after running asyncio')

Bad luck.

What about Poser as server and MD as Client?

P11 can handle threads at least. I assume P12 can it too.

Was thinking about simulating threads by setting a one second timeout for the server. This should, I would expect, cause

    try: 
        server.handle_request()

to exit with an error status and move thread execution on to checking the keyboard input buffer. The requests given from Poser are not a very high frequency. Functions called include writing or loading a geometry to or from file, and some even start a 30 frames simulation. A delay of a second or so is totally acceptable.

What I find very odd is that the server has a queue for 5 requests, but as ar as I can see no way of checking if there are any requests in the queue. If there was, life would be simple: A timer that checks every second or so if the number of pending requests is greater than 0, and if so send out the request handler. That would, I think, cause a lot less overhead than sending out the complete handler, let it sit its timeout and raise the exception. I have not come across a good documentation, though. Not even sur about the unit of the timeout setting, miliseconds or seconds.

You suggest to reverse roles and use Poser as the server and MD as the client. This would be like in MD starting the exchange by saying to Poser: "tell me what to do!", and Poser returning a code from which MD can determine what to do, right? Would be worth considering.

Tried setting the timeout option but and it gave no success.

Anyway many thanks for thinking along. Much appreciated.

MD has nailed the Python interpreter to one thread only. I assume the reason is that a running Python-Thread is able to "inspect" what MD is doing (re-engeneering). Seems that is the reason why they canceld UI-support too (UI needs an event-loop).

Poser up to version 9 did that too. The only chance was to use Poser-Pythons "callback" (called every time something happens in the UI, and generates "empty" events too – kind of a so called heartbeat).

Timeout can't work in such a scenario, because a timeout-loop would require event notifications (at least a timer able to interrupt a process). Python does no polling in the standard libs. I had to rewrite some parts of Pythons libs too to get Poser 9/10 running with standard async TCP libs and Posers "callbacks".

Thanks for the reply, re: my quick notion of 'Python to Photoshop via a server'. After some research it appears that no complex scripted server linkage is needed. The user simply has Poser's PoserPython script call Photoshop, and the script then feeds Photoshop a javascript script. This can then run almost any complex process in Photoshop, including having Photoshop act on the renders that Poser just saved. Working example for PoserPython....

FVerbaas posted at 9:01PM Fri, 11 December 2020 - #4407133

You suggest to reverse roles and use Poser as the server and MD as the client. This would be like in MD starting the exchange by saying to Poser: "tell me what to do!", and Poser returning a code from which MD can determine what to do, right? Would be worth considering.

Or use the filesystem. Depending on the content of the received file, a response file is then generated. A fast and dirty way to get things running and come up with a better solution later.

HartyBart posted at 9:19PM Fri, 11 December 2020 - #4407178

Thanks for the reply, re: my quick notion of 'Python to Photoshop via a server'. After some research it appears that no complex scripted server linkage is needed. The user simply has Poser's PoserPython script call Photoshop, and the script then feeds Photoshop a javascript script. This can then run almost any complex process in Photoshop, including having Photoshop act on the renders that Poser just saved. Working example for PoserPython....

Nice solution Adobe did with that possibility. Ok, not a Server/Client solution. More like One Way Messaging. But, what if you had Photoshop already running?

adp001 posted at 11:28PM Fri, 11 December 2020 - #4407177

MD has nailed the Python interpreter to one thread only. I assume the reason is that a running Python-Thread is able to "inspect" what MD is doing (re-engeneering). Seems that is the reason why they canceld UI-support too (UI needs an event-loop).

I doubt they are so sharp. Most of the implementation is quite rag-rag. Looks more like they had trouble getting it to work and decided to drop this. The used PythonQt, a lesser known version.

Timeout can't work in such a scenario, because a timeout-loop would require event notifications (at least a timer able to interrupt a process).

Good point. Will check if I can run a timer at all.

adp001 posted at 11:48PM Fri, 11 December 2020 - #4407179

Or use the filesystem. Depending on the content of the received file, a response file is then generated. A fast and dirty way to get things running and come up with a better solution later.

I used something like that in the bridge for MD9, with MD looking every 3 seconds if a file giving avatar vertices is updated, and if so morph the avatar to match the figure pose in Poser. I will first see if I can slave MD to Poser to do the same. The only thing I am missing now is the ability to switch MD locally from 'slave' mode to independent mode, and in MD a more simple way than starting a script to do the reverse. I can unslave MD from Poser but not from MD itself. MD now goes into 'program does not respond' mode when I click on its window'.

FVerbaas posted at 12:46AM Sat, 12 December 2020 - #4407187

MD now goes into 'program does not respond' mode when I click on its window'.

Stick in a poll-loop. Post a code snippet. Maybe I can come up with something helpful.

Found something on stackoverflow.com that might be helpful. Can*t test it myself, because I'm not on Windows.

Keyboard interrupt after 5 seconds:

import sys, time, msvcrt

def readInput( caption, default, timeout = 5):
    start_time = time.time()
    sys.stdout.write('%s(%s):'%(caption, default));
    input = ''
    while True:
        if msvcrt.kbhit():
            chr = msvcrt.getche()
            if ord(chr) == 13: # enter_key
                break
            elif ord(chr) >= 32: #space_char
                input += chr
        if len(input) == 0 and (time.time() - start_time) > timeout:
            break

    print ''  # needed to move to next line
    if len(input) > 0:
        return input
    else:
        return default

Threads work:

# threading_simpleargs.py

import threading


def worker(num):
    """thread worker function"""
    print('Worker: %s' % num)


threads = []
for i in range(5):
    t = threading.Thread(target=worker, args=(i,))
    threads.append(t)
    t.start()

This example, from Module Of The Week gives, as it should:

Worker: 0
Worker: 1
Worker: 2
Worker: 3
Worker: 4

somehow the text shown in the shell appears delayed.

To test and demonstrate this I tweaked the example you gave a little to read:

import os
import sys, time, msvcrt

def readInput( caption, timeout = 20):
    start_time = time.time()
    print(time.ctime(start_time))
    print(caption)
    input = ''
    while True:
        if msvcrt.kbhit():
            chr = msvcrt.getche()
            if ord(chr) == 13: # enter_key
                break
            elif ord(chr) >= 32: #space_char
                input += chr
        ti = time.time()
        if len(input) == 0 and (ti - start_time) > timeout:
            break

    print('')  # needed to move to next line
    if len(input) > 0:
        return input
    else:
        return ('timedout at '+ time.ctime(ti))

inp = readInput('Press a key:')

print('inp was: '+ inp)

Final result is as expected:

but the buildup is not correct. Obviously the start time and the request to press a key would show up, stay there for 20 seconds or until Enter was pressed, and then the result would be shown. This is not the case: Nothing is shown for 20 seconds and then the whole text block as shown appears in one screen update.

Sounds like I need to change some screen delay settings.

The thread example also works with an implementation of "dummy-threads".

Try this:

import 
    threading, 
    time

starttime = time.time() + 60

def test():
    print(time.asctime())
    if time.time() < starttime:
        threading.Timer(5, test).start()

test()

Run this. You should be able to use the console while in a 5 second interval something is written to the console (date/time). If nothing shows up, threads are there but not working. If you see something, you are able to use "threaded async".

In view of the delay in the console, the async could well have worked but the results held in cache.

I made a call to MD-support about this delay. It realy sets one on the wrong foot during development and limits possibilities. Good to hear that threads have their own Timer. I was just sorting that out. Under Qt I had a QtTimer type that kicked a given method exery xx seconds. Will look into the above since it seems to be an alternative.

I quickly wrote you a framework for communication via sockets and threads. However untested. But the principle is well visible. You just have to adapt it to your needs.

import threading, socket

from collections import deque  # thread save list

HOST, PORT = "127.0.0.1", 8888

queue_IN = deque()  # incomming data
queue_OUT = deque() # outgoing data
tasklist = list()  # holds running tasks

_tcpsocket = None
stop_flag = threading.Event()


def _setSocket(sock_family=socket.AF_UNSPEC, sock_type=socket.SOCK_STREAM):
    # setup TCP and bind to HOST, PORT
    global _tcpsocket, HOST

    if sock_family == 6:
        sock_family = socket.AF_INET6
    elif sock_family == 4:
        sock_family = socket.AF_INET

    if HOST is None:
        HOST = socket.getaddrinfo(None, 0, sock_family)[-1][-1][0]

    for res in socket.getaddrinfo(HOST, PORT, sock_family,
                                  sock_type, 0, socket.AI_PASSIVE):
        af, socktype, proto, canonname, sa = res
        try:
            _tcpsocket = socket.socket(af, socktype, proto)
        except socket.error as e:
            _tcpsocket = None
            continue
        else:
            _tcpsocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

        try:
            _tcpsocket.bind(sa)
            _tcpsocket.listen(1)
        except socket.error:
            _tcpsocket.close()
            _tcpsocket = None
            continue
        break

    return _tcpsocket


def client(connection):
    assert isinstance(connection, socket.socket)
    connection.settimeout(.1)

    while not stop_flag.is_set():
        try:
            d = connection.recv(8 * 1024)
        except TimeoutError:
            break

        if not d or d.endswith(b"nn"):
            break

        queue_IN.append(d)

    # Data from client received. Do something with it in another thread.
    t = threading.Thread(None, target=worker, args=[], kwargs=dict())
    t.start()
    tasklist.append(t)


def worker():
    for data in queue_IN:
        if stop_flag.is_set():
            return


while not stop_flag.is_set():
    """
    Dispatcher part.
    Wait for a connection and start tasks.
    """
    assert isinstance(_tcpsocket, socket.socket)
    try:
        connection, addr = _tcpsocket.accept()
    except socket.timeout:
        # Timeout is fine here.
        threading.Event().wait(.2)
        continue
    except socket.error as err:
        # This is bad. We better stop the server.
        stop_flag.set()  # signal to other threads.
        for task in tasklist:  # type: threading.Thread
            if task.is_alive():
                task.join()
        _tcpsocket.shutdown(socket.SHUT_RDWR)
        break

    # if we come her a connection is made
    t = threading.Thread(None, target=client, args=[connection, ])
    t.start()
    tasklist.append(t)

OK. Many thanks. I will study this and find how to apply.

An important part is not mentioned above: Socket initialisation.

call 'setSocket' once. The function will initialize the global variable 'tcpsocket'. After that set the timeout _tcpsocket.settimeout(.1)

Had already wondered where that had gone. Thanks again. Will study this further but beginning to see where problems were with previous attempts.

A final point re: calling subprocesses. DAZ Studio can also call Photoshop and feed it a script, though only via calling a BAT file on Windows which has in it....

Photoshop isn't the only useful thing you can use via a subprocess call. GIMP (https://www.gimp.org/) has an easy usable Python-Interface for years. ImageMagick is a tool made fo be called via subprocess (original download: https://github.com/ImageMagick/ImageMagick). OpenCV is maybe the most advanced tool if it comes to image processing. And the list goes on and on, Most of them Open Source and free to use.

With P12 it is even possible to use mutch of this tools "embedded" in Poser instead of using subprocess calls. Because PIP is supported in P12.

Do you know "Krita"? (https://krita.org)

from krita import *

d = Krita.instance().createDocument(512, 512, "Python test document", "RGBA", "U8", "", 120.0)
Krita.instance().activeWindow().addView(d)

Yes, thanks, I'm aware of Krita and am up-to-date with it, but the main problem with it is it doesn't have recordable-replayable Actions like Photoshop. Krita might be used to load the render stack to a saved PSD. As might the free and faster Paint.net (?). But it could not then juggle the order of the imported layers, rename them to meaningful names, knock out white from line-art, run a Photoshop filter on a color layer, and so on. Most people will want a Photoshop PSD and Actions in a format they know how to tweak and edit, plus easy access to Photoshop plugins for filtering some of the layers. Paint.net does have a plugin to record Actions, but it's not ideal. Same with Photoline's limited Actions capabilities. I'm not sure about the other budget software. Last time I looked Photoshop Elements didn't do Actions.

I'd never heard of OpenCV, so thanks for that. Seems to be a Library for the computer vision people?

As for Poser 12, hopefully the final will be out any day now, and then we'll see all the new features added since the early access version. The PIP sounds very interesting. It can't just be a static reference image able to be stuck in the viewport, surely, from the sound of it? I imagine a new full-Viewport PIP Window which "lays" on top of Poser's native viewport, and whatever this special Window "sees" sitting beneath it is "piped out" to Photoshop and then "back in again" in real-time? Thus giving near real-time visual filters for the Viewport in Poser, powered by Photoshop? Kind of like how some software already does a round-trip of an image to another software, but in near-realtime. But I guess that could only work either with the OpenGL or with a very fast graphics card / workstation running a near real-time 3Delight viewport.

Photoshop Actions: Sounds to me like ImageMagick calls on different images or part of images. Or the same with OpenCV. Sure: "Call the process again and again", does not sound as professional as "Photoshop Action" :)

By the way – I personally can't see the advantage of having full fledged Image-Processing inside of Poser. Can you give a practical example?

About OpenCV: There is actually nothing you can't do with OpenCV when it comes to images or videos. From simple things to complicated processes, even AI (recognizing things, faces, people for example). Problem is: it has a very steep learning curve.

But I see the difference – you see image not as a bunch of numbers in a file like I do :)

HartyBart posted at 9:00AM Fri, 18 December 2020 - #4407870

As for Poser 12, hopefully the final will be out any day now, and then we'll see all the new features added since the early access version. The PIP sounds very interesting. It can't just be a static reference image able to be stuck in the viewport, surely, from the sound of it? I imagine a new full-Viewport PIP Window which "lays" on top of Poser's native viewport, and whatever this special Window "sees" sitting beneath it is "piped out" to Photoshop and then "back in again" in real-time? Thus giving near real-time visual filters for the Viewport in Poser, powered by Photoshop? Kind of like how some software already does a round-trip of an image to another software, but in near-realtime. But I guess that could only work either with the OpenGL or with a very fast graphics card / workstation running a near real-time 3Delight viewport.

Hmmmm – seems we are talking about different things. "PIP" in Python does not mean "Picture In Picture". It's a tool to install libraries for Python.

https://pypi.org/project/pip/

Having PIP in Poser allows to include all those ready to go libraries out there. To manipulate geometries, even images with Python for example :)

Maybe you should open a new thread with "Poser to Photoshop" in the title (or something like that). So that other people with the same interested in this can participate.

adp001 posted at 8:59AM Sat, 19 December 2020 - #4407180

HartyBart posted at 9:19PM Fri, 11 December 2020 - #4407178

Thanks for the reply, re: my quick notion of 'Python to Photoshop via a server'. After some research it appears that no complex scripted server linkage is needed. The user simply has Poser's PoserPython script call Photoshop, and the script then feeds Photoshop a javascript script. This can then run almost any complex process in Photoshop, including having Photoshop act on the renders that Poser just saved. Working example for PoserPython....

Nice solution Adobe did with that possibility. Ok, not a Server/Client solution. More like One Way Messaging. But, what if you had Photoshop already running?

it uses the running version of photoshop. ( at least on win 10 )

Thanks for the tip, Structure.

adp001: "I personally can't see the advantage of having full fledged Image-Processing inside of Poser. Can you give a practical example?"

My imagined real-time PIP Photoshop example was a bit fantastical. But, to take something more practical - let's say you wanted to combine and blend three renders, a Sketch, a Comic Book and a Firefly render. Perhaps your aim is combine these to emulate the style of the sort of drawings an architectural studio produces. Or you want to produce medical diagrams which have to have a certain house-style demanded by the publisher (who doesn't want to pay for hand-drawn). You could not currently do that sort of combination in Poser 11.

But what if Poser 12 could combine those renders into a PSD automatically, like you can currently with the Firefly G-buffer renders (combo1.psd)? Then you'd stay within Poser, and blend the layer opacities and also run a filter on one or more of the layers, then save to a new PSD (combo2.psd). That would remove the need for software like Photoshop. At present you would have to send out the render set out to Photoshop, and 'cross-your fingers' that even the basic layer-stacking import script doesn't blow up Photoshop.

"Having PIP in Poser allows to include all those ready to go libraries out there".

That sounds useful. So it expands the range of what can be done in Python. Well, hopefully the full Poser 12 will be released tomorrow - if they are still on schedule - and we'll see what it can do.

Only thing you need for this is a Python graphics lib. PIL is available since ever for Poser. Anything you decribed can be done already if you can deal with Python.

With P12 even Photoshops PSD format is easy: https://pypi.org/project/psd-tools/

(With a bit know-how this works also with P11: https://github.com/psd-tools/psd-tools/tree/master/src/psd_tools)

I still would prefer creating the single images with the right tool for the job (Poser, Blender, Krita), assemble them with Gimp or Inkscape and make a PDF to publish it (Open Office?). No need for a bridge to an expensive Software and/or a lot of trial and error.

Thanks for the tip on Kota Yamaguchi's PSD Tools for Poser 11. The working example given on the homepage is to open a PSD and extract a layer as a PNG. While there are several other such export scripts to be found, it apparently can't be done the other way. StackOverflow has (from 2017).... "there still is currently no way to write PSD files in pure Python in a manner that would satisfy at a productive level." The best answer there is to suggest a subprocess call to GIMP via command-line. Has anything changed in the three years, on having Python itself do a basic "stack six PNG renders into a new layered PSD, then save it with filename xyz.PSD"?

Or, is that function enabled by the newer Poser 12-only PSD Tools you mention? But if the basic starting stack still can't be made in this newer version, then the other (obviously powerful) features of the PSD Tools modules are of no use - they will have no saved PSD file to work on and re-sort / re-name / blend layers.

GIMP was for a moment looking like the best free alternative to calling Photoshop as a subprocess, but a robust stacking script would need to be found for it. Judging by a short search, again everyone seems to want to export from PSD - but no-one wants to import via a script.

Even then the problem would be that GIMP has no recordable Actions by which users could then brew their own artistic automation, by working on the saved "base" PSD file. So far as I know GIMP has no record-what-I-do / replay-what-I-did Actions feature like Photoshop has. Instead GIMP appears to expect users to write code in Script-fu, to create an automated script. This is not viable for the average creative user.

One interesting automated possibility I found mention of, however, was ImageMagick's "compose". This can be used when images are all of exactly the same size as Poser scene renders are...

convert render1.png render2.png render3.png -gravity center -composite combo.psd

The problem then is that ImageMagick cannot then run through and re-name and re-sort the layers in the resulting PSD. Nor can it set opacity blending on these layers. Also, Photoshop is said to throw errors on opening such a file ('blending modes not set').

Thus it all comes back to Photoshop. I would love to combine Poser renders via free software, to be combined to a "base" PSD for users to then play with creatively. But it doesn't seem viable.

Well, I tried to go free. Here's a new working GIMP script that simply imports a folder of PNG renders and puts each one onto a new layer. Why such a basic and useful import script does not ship with GIMP, and is seemingly unknown to the forums, is beyond me. It took about five hours to find this, and even then it needed to be tweaked and hacked.

As for having a PoserPython script then call GIMP + this Python script as a subprocess.... that is a total nightmare to set up and nowhere near as simple as 'c:/gimp.exe calls c:/batch.py'. It appears to involve having GIMP 'know' where the called script is, and be able to start its other modules in subprocess mode. I just cannot get that even remotely working. So much so that I just have to go back to calling Photoshop + script, which is vastly easier to call.

I made a final try at a possible free workflow. The U.S. government offers a free Java-driven image tool ImageJ, intended for scientists and doctors needing to build vast image stacks. Importing a render stack is very easy. Being in such wide use, ImageJ also has many free plugins. There's one free PSD stack exporter plugin. But even with this, and all the other plugins, ImageJ still cannot write out its imported stack to a layered PSD. It can only save the stack out as individual PSD layer files, or as a PSD with one flattened layer.