Implicit conversion of types and nullptr


#1

Hi,

In my application, I use ChaiScript as my language for configuration files. I provide several predefined variables to the user, so he can overwrite their values to configure my program. Some of these variables have complex types and are created by the user with helper functions. Given they have polymorphic behavior, they are registered as shared pointers. Some of the variables have to be set by the user, therefore the shared pointers are initialized with nullptr to be later substituted. Here it’s an example of what a user would write in my configuration files:

domain = discrete_domain([ "1", "2", "3", "4", "5", "6" ])

The variable domain has type Domain and discrete_domain is a helper function to create a DiscreteDomain, one of the subclasses of Domain.

My problem is: in ChaiScript, variables are converted behind the scenes to make concrete objects, shared_ptr, references, raw pointers, etc. all be used indistinguishably. By default, operator= receives the assigned and assignee as references. As domain is initialized as a nullptr, when the shared_ptr is converted I get undefined behavior.

Is there a good way to avoid this problem? When my shared_ptrs are inside some container, everything works fine as ChaiScript doesn’t convert the template parameter. I thought about creating a wrapper Box<T> just to avoid the conversion, but I don’t think it’s an elegant solution. After all, this is a wider question: it’s related to the way ChaiScript deals with nullptr when implicitly converting types.


#2

As a follow up, I tried to create a MWE to reproduce the problem. However, everything worked fine:

// Standard headers
#include <string>
#include <memory>
#include <iostream>

// External headers
#include <chaiscript/chaiscript.hpp>
#include <chaiscript/dispatchkit/bootstrap_stl.hpp>

struct Domain {
};

struct DiscreteDomain : public Domain {
};

int main() {
  using namespace chaiscript;
  ChaiScript chai;

  chai.add(fun([] () {
    return std::static_pointer_cast<Domain>(std::make_shared<DiscreteDomain>());
  }), "discrete_domain");

  auto module = std::make_shared<Module>();
  module->add(user_type<Domain>(), "Domain");
  bootstrap::standard_library::assignable_type<Domain>("Domain", module);
  chai.add(module);

  std::shared_ptr<Domain> domain;
  chai.add(var(&domain), "domain");

  chai.eval(R"(
    domain = discrete_domain()
  )");

  return 0;
}

On the other hand, my larger (and much more complex) system gets the following error:

Error: "Unable to find appropriate'=' operator." With parameters: (Domain, Domain) during evaluation at (script/models/lccrf/cassino.tops 10, 1)

  26 overloads available:
      (Type, const Type)
      (Map, const Map)
      (Vector, const Vector)
      (Alphabet, const Alphabet)
      (Alphabets, const Vector)
      (Alphabets, const Alphabets)
      (Probabilities, const Probabilities)
      (Domain, Domain)
      (Domains, const Domains)
      (Duration, Duration)
      (Model, Model)
      (Models, const Models)
      (State, State)
      (States, const Map)
      (States, const States)
      (DependencyTree, DependencyTree)
      (DependencyTrees, const DependencyTrees)
      (FeatureFunctions, const FeatureFunctions)
      (FeatureFunctionLibraries, const FeatureFunctionLibraries)
      (Size, const Size)
      (bool, const bool)
      (Assignable_Function, const Function)
      (Number, const Number)
      (Object, Function)
      (Object, const Function)
      (Object, Object)

The line I’m testing is exactly the same in both programs:

domain = discrete_domain()

#3

Well, I’m still working in the problem. I found out (by pure luck) that the “Unable to find…” error happens when I register a std::shared_ptr<T> instead of T itself in ChaiScript engine. I actually was registering the smart pointer version in my more complex application. Here it’s a MWE reproducing this behavior:

// Standard headers
#include <string>
#include <memory>
#include <iostream>

// External headers
#include <chaiscript/chaiscript.hpp>
#include <chaiscript/dispatchkit/bootstrap_stl.hpp>
using namespace chaiscript;

struct Domain {};
using DomainPtr = std::shared_ptr<Domain>;

struct DiscreteDomain : public Domain {};
using DiscreteDomainPtr = std::shared_ptr<DiscreteDomain>;

int main() {
  ChaiScript chai;

  chai.add(fun([] () {
    return std::static_pointer_cast<Domain>(std::make_shared<DiscreteDomain>());
  }), "discrete_domain");

  auto module = std::make_shared<Module>();

  // Here it's the problematic use
  module->add(user_type<DomainPtr>(), "Domain");
  bootstrap::standard_library::assignable_type<DomainPtr>("Domain", module);

  chai.add(module);

  DomainPtr domain;
  chai.add(var(&domain), "domain");

  chai.eval(R"(
    domain = discrete_domain()
  )");

  return 0;
}

The output is:

terminate called after throwing an instance of 'chaiscript::exception::eval_error'
  what():  Error: "Unable to find appropriate'=' operator." With parameters: (Domain, Domain)
Aborted (core dumped)

Perhaps user_type could to some metaprogramming manipulation over its template parameter in order to avoid this. However, I don’t know if it worth in the general case. I made that myself in my complex application because it suited for the way I’m registering types on it.

After solving this, I keep on my tests in order to check if my function discrete_domain would be able to set the variable correctly. Unfortunately, it doesn’t. Here it’s another MWE:

// Standard headers
#include <string>
#include <memory>
#include <iostream>

// External headers
#include <chaiscript/chaiscript.hpp>
#include <chaiscript/dispatchkit/bootstrap_stl.hpp>
using namespace chaiscript;

struct Domain {};
using DomainPtr = std::shared_ptr<Domain>;

struct DiscreteDomain : public Domain {};
using DiscreteDomainPtr = std::shared_ptr<DiscreteDomain>;

int main() {
  ChaiScript chai;

  chai.add(fun([] () {
    return std::static_pointer_cast<Domain>(std::make_shared<DiscreteDomain>());
  }), "discrete_domain");

  auto module = std::make_shared<Module>();
  module->add(user_type<Domain>(), "Domain");
  bootstrap::standard_library::assignable_type<Domain>("Domain", module);
  chai.add(module);

  DomainPtr domain;
  chai.add(var(&domain), "domain");

  chai.eval(R"(
    domain = discrete_domain()
  )");

  if (domain == nullptr)
    std::cerr << "'domain' is null!" << std::endl;
  else
    std::cerr << "'domain' is not null!" << std::endl;

  return 0;
}

The output is:

'domain' is null!

As I said in my previous comments, everything works right when I have a std::vector of std::map of pointers. I even tried to register a function similar to nullify_shared_ptr (from the cheatsheet) to print if the pointer was null inside the script, but the function didn’t work (the engine was unable to convert the variable to the pointer version).

Am I registering my shared_ptrs in the right way? Or is there a bug in the engine?


#4

On this line:

chai.add(var(&domain), "domain");

You are registering a std::shared_ptr<DiscreteComain> * a pointer to a shared_ptr, that is. Honestly, I don’t know if I have any tests for this, or if it’s handled properly at all internally. You probably want to do one of two different things:

// let chaiscript participate in the ownership by giving it a copy of the shared_ptr
chai.add(var(domain), "domain");

Or

// Give chaiscript a pointer to the object that the share_ptr points to
chai.add(var(&*domain), "domain");

It seems like you want the first one, because you seem to want ChaiScript to be aware of nullability and such.


#5

Thans you, @lefticus!

This was precisely the problem. There was just one more issue: I also had to initialize the shared_ptr with a non-null content. Otherwise the engine would throw an exception when trying to pass it for the assignment function. This is the MWE tha really works:

// Standard headers
#include <string>
#include <memory>
#include <iostream>

// External headers
#include <chaiscript/chaiscript.hpp>
#include <chaiscript/dispatchkit/bootstrap_stl.hpp>
using namespace chaiscript;

struct Domain {};
using DomainPtr = std::shared_ptr<Domain>;

struct DiscreteDomain : public Domain {};
using DiscreteDomainPtr = std::shared_ptr<DiscreteDomain>;

int main() {
  ChaiScript chai;

  chai.add(fun([] () {
    return std::static_pointer_cast<Domain>(std::make_shared<DiscreteDomain>());
  }), "discrete_domain");

  auto module = std::make_shared<Module>();
  module->add(user_type<Domain>(), "Domain");
  bootstrap::standard_library::assignable_type<Domain>("Domain", module);
  chai.add(module);

  auto domain = std::make_shared<Domain>();
  chai.add(var(domain), "domain");

  chai.eval(R"(
    domain = discrete_domain()
  )");

  if (domain == nullptr)
    std::cerr << "'domain' is null!" << std::endl;
  else
    std::cerr << "'domain' is not null!" << std::endl;

  return 0;
}

I was setting lots of shared_ptrs in my application by passing their address. Luckily this was not causing bigger crashes. Now I believe I’ll be able to fix everything. Thanks again!


#6

If you want to assign the shared_ptr instead of the value that it points to, you’ll want to implement your own assignment operator for it. The assignable_type is going to do value assignment. You would want to do something like assignable_type<DompainPtr>() if that’s what you need.

-Jason


#7

Yes, I need my external shared_ptr to change, not its content. This way I’ll get the polymorphic behavior I need. Here it’s an example of how the pointer assignment is working:

// Standard headers
#include <memory>
#include <string>
#include <cstdlib>
#include <iostream>

// External headers
#include <chaiscript/chaiscript.hpp>
#include <chaiscript/dispatchkit/bootstrap_stl.hpp>
using namespace chaiscript;

struct Domain {
  virtual std::string name() const { return "Domain"; }
};
using DomainPtr = std::shared_ptr<Domain>;

struct DiscreteDomain : public Domain {
  std::string name() const override { return "DiscreteDomain"; }
};
using DiscreteDomainPtr = std::shared_ptr<DiscreteDomain>;

int main() {
  ChaiScript chai;

  chai.add(fun([] () {
    return std::static_pointer_cast<Domain>(std::make_shared<DiscreteDomain>());
  }), "discrete_domain");

  auto module = std::make_shared<Module>();
  module->add(user_type<Domain>(), "Domain");
  bootstrap::standard_library::assignable_type<Domain>("Domain", module);
  chai.add(module);

  chai.add(fun([] (DomainPtr &l, const DomainPtr &r) {
    std::cerr << "l: " << l->name() << std::endl;
    std::cerr << "r: " << r->name() << std::endl;
    l = r;
    std::cerr << "l: " << l->name() << std::endl;
    return l;
  }), "=");

  auto domain = std::make_shared<Domain>();
  std::cerr << "in: " << domain->name() << std::endl;

  chai.add(var(domain), "domain");

  chai.eval(R"(
    domain = discrete_domain()
  )");

  if (domain == nullptr) {
    std::cerr << "'domain' is null!" << std::endl;
    return EXIT_FAILURE;
  }

  std::cerr << "out: " << domain->name() << std::endl;

  return EXIT_SUCCESS;
}

The output is:

in: Domain
l: Domain
r: DiscreteDomain
l: DiscreteDomain
out: Domain

The assignment works perfectly inside ChaiScript engine, but I really need is the external variable domain to change. I tried to register it by reference (using std::ref), but I got the same result. Is there a way to make ChaiScript to use my external variable? Perhaps this would be the right behavior for std::ref(model_ptr) and std::addressof(model_ptr).


#8

You’re in a weird place wanting to be able to work with it both as a value and a shared_ptr. I think there’s a good chance that you/I are overthinking this…

So you need inside of ChaiScript, to be able to reassign a polymorphic shared_ptr?

chai.add([](shared_ptr<Domain> &lhs, const shared_ptr<Domain> &rhs){
  lhs = rhs;
}, "=");

chai.add(var(&domain), "domain");

I think that combination of assignment operator and registered shared_ptr object will give you what you want, but if it doesn’t I will have to test some stuff.


#9

The most important for me is to reassign the shared_ptr with a new value. I tried this combination:

// Standard headers
#include <memory>
#include <string>
#include <cstdlib>
#include <iostream>

// External headers
#include <chaiscript/chaiscript.hpp>
#include <chaiscript/dispatchkit/bootstrap_stl.hpp>
using namespace chaiscript;

struct Domain {
  virtual std::string name() const { return "Domain"; }
};
using DomainPtr = std::shared_ptr<Domain>;

struct DiscreteDomain : public Domain {
  std::string name() const override { return "DiscreteDomain"; }
};
using DiscreteDomainPtr = std::shared_ptr<DiscreteDomain>;

int main() {
  ChaiScript chai;

  chai.add(fun([] () {
    return std::static_pointer_cast<Domain>(std::make_shared<DiscreteDomain>());
  }), "discrete_domain");

  auto module = std::make_shared<Module>();
  module->add(user_type<Domain>(), "Domain");
  bootstrap::standard_library::assignable_type<Domain>("Domain", module);
  chai.add(module);

  chai.add(fun([] (DomainPtr &lhs, const DomainPtr &rhs) {
    std::cerr << "lhs: " << lhs->name() << std::endl;
    std::cerr << "rhs: " << rhs->name() << std::endl;
    lhs = rhs;
    std::cerr << "lhs: " << lhs->name() << std::endl;
  }), "discrete_domain");

  auto domain = std::make_shared<Domain>();
  chai.add(var(&domain), "domain");

  std::cerr << "in: " << domain->name() << std::endl;

  chai.eval(R"(
    domain = discrete_domain()
  )");

  if (domain == nullptr) {
    std::cerr << "'domain' is null!" << std::endl;
    return EXIT_FAILURE;
  }

  std::cerr << "out: " << domain->name() << std::endl;

  return EXIT_SUCCESS;
}

The output is:

in: Domain
out: Domain

As you can see, it didn’t print the lhs and rhs related messages that I put inside the overload of the = operator. So it means the engine used the assignment operator for Domain in order to make the assignment. However, this didn’t affect the external variable. Without the assignment operator for Domain, the engine throws an exception:

in: Domain
terminate called after throwing an instance of 'chaiscript::exception::eval_error'
  what():  Error: "Unable to find appropriate'=' operator." With parameters: (Domain, Domain)
Aborted (core dumped)

The same behavior happens if I change var(&domain) by var(std::ref(domain)).


#10

This works:

// Standard headers
#include <memory>
#include <string>
#include <cstdlib>
#include <iostream>

// External headers
#include <chaiscript/chaiscript.hpp>
#include <chaiscript/dispatchkit/bootstrap_stl.hpp>
using namespace chaiscript;

struct Domain {
  virtual std::string name() const { return "Domain"; }
};
using DomainPtr = std::shared_ptr<Domain>;

struct DiscreteDomain : public Domain {
  std::string name() const override { return "DiscreteDomain"; }
};
using DiscreteDomainPtr = std::shared_ptr<DiscreteDomain>;

int main() {
  ChaiScript chai;

  chai.add(fun([] () {
    return std::static_pointer_cast<Domain>(std::make_shared<DiscreteDomain>());
  }), "discrete_domain");

  auto module = std::make_shared<Module>();
  module->add(user_type<Domain>(), "Domain");
  chai.add(module);


  chai.add(fun([] (DomainPtr &lhs, DomainPtr rhs) {
    std::cerr << "lhs: " << lhs->name() << std::endl;
    std::cerr << "rhs: " << rhs->name() << std::endl;
    lhs = rhs;
    std::cerr << "lhs: " << lhs->name() << std::endl;
  }), "=");

  {
    auto domain = std::make_shared<Domain>();
    chai.add(var(domain), "domain");
  }
  // ok, we've just added the domain into chaiscript engine
  // we will now let it own the main copy and get a reference back to the
  // one that chaiscript owns
  auto &domain_ptr = chai.eval<DomainPtr &>("domain");

  std::cerr << "in: " << domain_ptr->name() << std::endl;

  chai.eval(R"(
    domain = discrete_domain()
  )");

  if (domain_ptr == nullptr) {
    std::cerr << "'domain' is null!" << std::endl;
    return EXIT_FAILURE;
  }

  std::cerr << "out: " << domain_ptr->name() << std::endl;

  return EXIT_SUCCESS;
}

But it is definitely not ideal. The ultimate problem is that I’m not handling pointers-to-shared_ptr, which is really what you want.

I might try to add it… but I think it’s going to be hard to get right and might add undesired overhead in some places. OTOH, if you’re willing to let the chaiscript engine own the shared_ptr and ask for it back out by & then it works.

0Jason


#11

This workaround is nice, but it unhappily does not address my full problem. In my system, the variable domain is part of a data structure that represents (for the system) my configuration files. Thus I really needed the fields of this data structure to be filled. For now, I’ll have to find an alternative solution. I tried to use a shared_ptr<shared_ptr<...>> and thought about creating a template wrapper struct over the shared_ptr (so the code would work as it does with a vector<shared_ptr<...>>), but this makes the code cumbersome. I’ll review my logic and try to avoid this polymorphism.


#12

I’ll try to spend some more time on the pointer-to-shared_ptr solution. Not sure exactly what it would mean. Might be easy - might be hard.

What version of ChaiScript are you using?


#13

Hi Jason,

Thanks for your dedication! ChaiScript is a better and better project due to your work!

I’ll follow the issue and I can test something if you need help. I hope this feature is easier than harder and that it doesn’t harm the performance of the engine.