Extract internals

The extract_internals program was written as an attempt to fix a common problem when using object persistence: if two programs are going to send objects to each other, how do they know what objects to expect to receive? The extract_internals program analyses a program and outputs the list of object types that this program can send. The list is written in a format that can easily be used by the program that receives the objects.

The problem

Three problems, actually

1. When compiling a program, SmartEiffel only generates code to support classes of which instances are created in that program. Receiving objects through the STORABLE interface does not count as creating for this matter, so we need to cheat SmartEiffel into believing that all those types that can be received through the STORABLE interface can also be created locally (that is, mark those types as live).
2. We also need the compiler to know that we want to do introspection on types that we want to receive through the STORABLE interface, since STORABLE relies on introspection. Mainly, this means marking TYPED_INTERNALS[the type that we want to receive] as live.
3. Actually, SmartEiffel even strips unused attributes off classes (at least in -boost mode). If two different programs use the same class, there is no guarantee that the same attributes will be stripped for both programs. That is unfortunate if those progams exchange objects.

Installing the tool

The first version of the extracting tool is bundled with SmartEiffel 2.2 final, so start by downloading and installing that if you don't already have it. http://smarteiffel.loria.fr/general/download.html

Now, go to some directory in your path (for instance, SmartEiffel/bin) and compile the extractor tool:

se c -boost -no_gc -no_split -clean -o extract_internals extract_internals

You may also want to add extract_internals to the [tools] section of your SmartEiffel configuration file to be able to use extract_internals through the se front-end. The configuration file is often named .serc, se.cfg or system.se.

One-way sending

Let's see how to use the extract_internals tool. We have two programs, WRITER and READER. We want the WRITER to send DATA objects to the READER through an xml file called data.xml. Here's the source code of the WRITER (programs courtesy of Damian Dobroczyński):

class WRITER

creation
   make

feature
   make is
      local
         r: XML_REPOSITORY[DATA]
         x: DATA
      do
         create r.make
         create x.make
         r.put (x, "qoocku")
         r.commit_to_file ("data.xml")
      end

end

And now for the READER. Unfortunately, this code does not work because the compiler does not know yet that objects of type DATA can be created by READER (through the repository).

class READER

creation
   make

feature
   make is
      local
         r: XML_REPOSITORY[DATA]
         x: DATA
      do
         create r.from_file ("data.xml")
         x := r.at ("qoocku")
         check
            x /= Void
         end
         print (x.a)
      end

end

For the sake of completeness, here is the code of DATA (it is not important for this discussion)

class DATA

inherit
   STORABLE

creation
   make

feature
   a: STRING

   make is
      do
         a := "Qoocku"
      end

end

Now the fun stuff begins: we use extract_internals to get the list of objects that can be created by the WRITER:

extract_internals writer

This will generate two files:

• writer_internals_helper.e is the list of live classes that WRITER can export. This list is written as an Eiffel class.
• writer_internals_helper.se is the list of live attributes of those classes. This list is written as a Cecil file.

To use the information we obtained, we have to add a call to WRITER_INTERNALS_HELPER.storable_quit as the last instruction of READER. Then we compile READER using the cecil file writer_internals_helper.se. Here is the modified READER:

class READER

creation
   make

feature
   make is
      local
         r: XML_REPOSITORY[DATA]
         x: DATA
      do
         create r.from_file ("data.xml")
         x := r.at ("qoocku")
         check
            x /= Void
         end
         print (x.a)
         (create {WRITER_INTERNALS_HELPER}).storable_quit
      end

end

Now we can compile the WRITER and the READER. Remember to use the cecil file when compiling the READER.

se c -o writer writer
se c -o reader reader -cecil writer_internals_helper.se

Now we can enjoy the result of our work and run the writer, followed by the reader...

./writer
./reader

...and enjoy the output, namely "Qoocku" (By the way does that mean anything?)

N-way sharing

If you have several programs sharing data through the STORABLE interface, you could arguably call extract_internals for each of them, and add calls to all the xxx_INTERNALS_HELPER.storable_quit at the end of all of then. Quite possibly, you would have to run extract_internals several times until the helpers stabilise. But there's an easier way.

Let's assume we have three communicating programs, and their root classes/root procedures are PROGRAM_1.make, PROGRAM_2.make and PROGRAM_3.make. Now, we create a virtual common root:

class COMMON_ROOT

creation
   make

feature
   make is
      do
         create PROGRAM_1.make
         create PROGRAM_2.make
         create PROGRAM_3.make
      end

end

Now, we run extract_internals on common_root.e, and we add the following line at the end of PROGRAM_1.make, PROGRAM_2.make and PROGRAM_3.make:

(create {COMMON_ROOT_INTERNALS_HELPER}).storable_quit

We can then compile PROGRAM_1, PROGRAM_2 and PROGRAM_3 as usual, just adding the -cecil common_root_internals_helper.se parameter.

FAQ

You suggest to only initialise introspection at the end of the program. Isn't that a little late?

No, because the important action of storable_quit is performed at compile-time, not at run-time.

Some of the creation procedures used by xxx_INTROSPECTION_HELPER have nasty side-effects

That's no problem, because those creation instructions never actually get executed (see the "die" instruction above). We only want the compiler to believe that those instructions could could possibly get executed one day.

Some of the creation instructions in xxx_INTROSPECTION_HELPER will crash at run time because they have invalid parameters

No, because those instruction won't get executed.