Script Cancellation


#1

I was wondering if anybody had any thoughts on a good mechanism to cancel a running chai script. For example we may have a long running script on a thread, that we would like to stop running before attempting to join the thread. Maybe because the program has been shutdown or maybe because it is taking too long to execute.

I’ve been looking at inserting some sort of cancellation flag into AST_Node::eval, that would force an immediate return if the engine had been cancelled.

Does this seem like a reasonable approach? or is there already a mechanism for achieving this that I have missed?


#2

The one issue there’s simply no way to work around: if the user has called into a C++ function that refuses to terminate.

Otherwise, I’d say the idea is sound. I’m slightly concerned about the overhead of checking a flag for each invocation of AST_Node::eval, but it should be easy to test. Make it as lightweight as possible. No virtual function call, etc. I’ll be happy to evaluate it for mainstream inclusion if you implement it.

-Jason


#3

Sorry for the very extended delay to this reply (and what has turned into quite a long post)

I have added a hack into our local copy with a std::atomic<boolean> cancellation flag. I ended up adding the flag to the Dispatch_Engine and checking its state during each call to AST_Node::eval. The more I think about this solution the more I wonder if there is a better way.

Over the past year I have experimented with adding different features to ChaiScript like a tracer, profiler, and this cancellation stuff. But it is difficult to implement it in a way that would not adversely impact the performance for most users, and therefore be suitable to contribute back to the core project.

The fact that the AST_Node::eval is the main entry point, and it is recursively called back into from sub-classes in the AST means that any of this sort of code needs to be inserted there.

If the ‘evaluator’ could be abstracted from the AST it would allow the user to swap in different evaluators for different purposes, and thus only pay the performance penalty for the feature that they require.

The current executor could look something like this:

// The current evaluator
struct Evaluator {
  Boxed_Value operator()(AST_Node& n, const chaiscript::detail::Dispatch_State &t_e) {
      try {
         //evaluator passed to eval_internal. It might be better to add it to dispatch_state?
        return n.eval_internal(t_e, this);
      } catch (exception::eval_error &ee) {
        ee.call_stack.push_back(n.shared_from_this());
        throw;
      }
   }
 }

Other evaluators that log the amount of time in each eval_internal call could be written, or even one that provides a debug api like the low-level lua debug api could be written (I’d be happy to have a go at it).

I am not really sure on the best way to architect this - I’ve shown the evaluator as a functor above but there is probably a better way to do it.

Do you have any thoughts on this sort of change to the architecture?


#4

These are topics I’ve often pondered myself, but haven’t spent any time investigating.

I thought it would be possible to simply check a “should I be doing anything ‘pause’ like right now?” flag, like you suggested. But if it causes too much slowdown for the normal case then it’s clearly not a good answer.

I agree that some kind of compile-time swap-able option makes sense. Ideally, something that can be aggregated with compile time options to allow tracing/breakpoints/cancelation as individual things.

Maybe some sort of Callable object, as you show an object with operator(), or make it generic enough to allow a lambda or std::function if so desired.

I do like the idea. Sorry for the slow response, I’ve been letting it sink in.

Are you able to move to the latest C++14 updates I’m working on on develop branch, or are you stuck with older compilers and need to stay in C++11 land?

-Jason


#5

What would you think of something like this:

ChaiScript<> chai; // default with no interruptions
ChaiScript<chaiscript::Tracing> chai2; // automatic code tracing (maybe output to callgrind format?
ChaiScript<somethinguserdefined> chai3; // whatever you want it to do

#6

No worries about the “slow” response - Its got nothing on my six month reply time above :smile:

For my specific use case the cancellation flag solution works fine, especially since we are not overly concerned with speed (at least not to the level that an extra flag check adds). It just got me thinking about how it could be solved in a more generic way…

Visual Studio 2013 - which seems to have some C++14 features. I wouldn’t worry too much about my requirements, as the project I am working on that uses Chai is nearing the end of development and will probably stick with version 5.7.1.

I was proposing these changes as features that would help the Chai user base as a whole. I think it is an amazingly useful project that would really benefit from optional features like breakpoints and tracing, etc.


#7

I think that would make for quite a nice interface. Depending on how it was implemented the default, no interruptions scenario could probably be optimised by the compiler resulting in little to no run-time penalty.

Would the template parameters be policy types that the ChaiScript class inherits from? or a type more like an allocator that is instantiated and passed around within the ChaiScript object graph.

What about composability?

ChaiScript<chaiscript::Tracing, chaiscript::Cancellable> chai; // or maybe
ChaiScript<chaiscript::MultiPolicy<chaiscript::Tracing, chaiscript::Cancellable>> chai2 // or even
ChaiScript<CustomCancellableTracingPolicy> chai3;

#8

FYI, with some twitter help:

and

http://goo.gl/4W3hjS

-Jason


#9

cool. let me know if there is anything I can do to help.


#10

The latest develop branch has the ability to add code tracers at runtime. They can do whatever they want to do on specific AST_Node objects during execution. This should provide a good place to hook in some script cancellation type features.

Example from the unit tests here:

The actual capability is really quite untested. Let me know if you have any luck with it.

-Jason


#11

I disagree with going with policies, it’s not general enough and will be a pain each time someone will want you to add policies.

Before considering canceling of scripts, just allowing the user code to do evaluations in an incremental way would be better and more useful. Then the user can decide how to execute the code, in which thread, if they want or not to allow pausing, resuming, canceling, budgeting execution.

Basically, it would look like evaluation functions would return something like an iterator. On each call or iteration, it would execute one unit of work (a expression?).
Then the user can just loop until it ends, or do something in-between, etc.

That way, all usage can be described using algorithms instead of having policies which are basically hacky plugins (in this context).

I think this change could be hard but it would sitll be possible with the current way we launch evaluation.


#12

The method I implemented, which would allow the user to plug in whatever they want to, took a considerable amount of engineering effort to figure out how to implement it without impacting the regular performance of the code.

Doing evaluating in an incremental way, as you suggest, would be quite impossible with how ChaiScript does its stack management today.

But with this implementation it would be quite possible to implement a callback that notifies the developer each time a unit has been executed and let the developer take some action (cancellation, etc).

-Jason


#13

Users can of course work without it, but it’s definitely the way to go to provide them control over execution, which is one thing I often miss in scripting languages (because then I have to setup hacks to unnatural hacks).

Does that imply that the callback can specify if the execution should continue or not?

Anyway, I’m just pointing that having a way to let the user do the progress of execution is better for the user, simply to allow them to schedule script execution and setup script vm state load/save easily and where they think it make sense for them. Even what you are providing, from what you describe -I could be wrong- seem to only solve half of the problem by keeping part of the control of execution.

I don’t know if this kind of feature would mean lower performance in the case where something like Chaiscript was designed this way from the ground up. I suspect not but you can’t rely on intuition in these matters of course.
I do understand that it would radically hard to change the current design or the new one to a full-control-over-execution kind of design, but I still need to point it out because it’s often (with state serialization) the thing I am not happy with in embedded scripting languages. Pardon my critics.

I’ll give feedback on the new design as soon as I can use it.