Fix refcounting and lifetime management around async python callbacks

* We need to keep a PythonContext (and its globals Dict) around while
  we still have some pending callbacks happening. So now the external
  code creates a PythonContext and then releases it when it's done, but
  the context will hang around until the global redirector object is
  destructed, which is responsible for deleting the context.
* The global redirector is deleted when a refcounting cycle is detected
  and the dict is unreachable, which only happens after the context is
  released.
* Any time a callback is passed to something and converted to a
  std::function we add a reference on the global redirector to keep it
  alive. When the callback has finished executing we remove the ref.
* This way, any pending callbacks that have been called but not finished
  or converted (queued) and not called yet asynchronously will keep the
  context object alive to be able to output, handle exceptions, etc.
* Additionally we need to detect when we're being called asynchronously
  and handle exceptions separately instead of trying to propagate up the
  call chain, because there might not be any more python code up the
  chain (e.g. the render manager calling a python callback).
This commit is contained in:
baldurk
2017-04-18 14:57:42 +01:00
parent c49670cfad
commit 6969b5b677
5 changed files with 238 additions and 74 deletions
+28
View File
@@ -367,6 +367,34 @@ PyObject *PassObjectToPython(const char *type, void *obj)
return SWIG_InternalNewPointerObj(obj, t, 0);
}
// this is defined elsewhere for managing the opaque global_handle object
PyThreadState *GetExecutingThreadState(PyObject *global_handle);
void HandleException(PyObject *global_handle);
// this function handles failures in callback functions. If we're synchronously calling the callback from within an execute scope, then we can assign to failflag and let the error propagate upwards. If we're not, then the callback is being executed on another thread with no knowledge of python, so we need to use the global handle to try and emit the exception through the context. None of this is multi-threaded because we're inside the GIL at all times
void HandleCallbackFailure(PyObject *global_handle, bool &fail_flag)
{
// if there's no global handle assume we are not running in the usual environment, so there are no external-to-python threads
if(!global_handle)
{
fail_flag = true;
return;
}
PyThreadState *current = PyGILState_GetThisThreadState();
PyThreadState *executing = GetExecutingThreadState(global_handle);
// we are executing synchronously, set the flag and return
if(current == executing)
{
fail_flag = true;
return;
}
// in this case we are executing asynchronously, and must handle the exception manually as there's nothing above us that knows about python exceptions
HandleException(global_handle);
}
%}
%header %{