BeerBaron Forum - Lemonball

January 2009 - This topic is a small guide to obtain a Python language interpreter running on a VxWorks VxSIM Simulator.
The process of embedding Python in VxWorks is quite simple but needs a significant number of file manipulations.
Due to copyright issues, I can not attach the source code of the project, but I can post hints to re-create a similar one by yourself.

The main concept followed was to cross-compile Python source code under Tornado 2.2, and add a simple wrapper to execute Python code from .py files on host machine.


Tool Set:

   
• Pyhton 2.6.1
• Tornado 2.2
• CygWin 1.5.24

Instructions:

Configuring the enviroment

Extract Python source code in a proper directory.


From the CygWin shell, add Tornado host compiler path to the Cygwin PATH, i.e.
$ export PATH=/cygdrive/c/tornado2_2/host/x86-win32/bin/:$PATH
and then, move to source directory and launch the ./configure script.

This will cause the script to prepare a pyconfig.h file that will be refferred to by the compiler to correctly compile a host compliant Python interpreter.

To obtain cross-compiling for the simulated target, the ./configure script must be launched with the following parameters:

$ CC=ccsimpc ./configure -host=x86-win32

The CC variable is set to ccsimpc which is the Tornado compiler for VxSIM Simulator, while host parameter is the target architecture.
In this case the Simulator will be running under a Win32 PC.


This is beautiful but not enough.


The ./configure script seems to be not optimized for cross-compilation with VxWorks, and with these parameters returns a few errors. It needs to be patched to correctly run.

All references to "cross compiling" that are followed by pieces of C code have been removed, because the script could not execute them and check the results.

NOTE: If you search for "cross compiling" keywords in the script, you will find many references. DO NOT delete all of them, but only those references followed by C code and some script commands used to evaluate the results.
In my experience, only three occurrencies of cross compiling test had to be removed.

NOTE: Replace the cygwin1.dll file in Tornado2.2\host\x86-win32\bin\ with the one present in your CygWin installation!!!

Compiling

After the script execution, some files are created: pyconfig.h, Makefile and Modules/config.c.

Give a glance to the first file (pyconfig.h), because it has to be edited in some details:
be sure to uncomment the definition of HAVE_DIRENT_H, and then check for the other uncommented defines to be sure that the script correctly identified the feature of the target OS.

Now it's time to create a VxSim project to compile the source files and link them in a Simulator.

Project type is "Bootable VxWorks image (custom configured)", and the selected BSP is "simpc" on "gnu" toolchain.

In the VxWorks tab from the Workspace window, it is important to include all the modules labelled "C++ components" and "operating system components".

Now look at the Makefile in the Python home directory: all the .c files related to .o (objects from compilation) listed in it must be added to the project.
They should be a little more than one hundred.

Edit the project build options to add all the include-paths needed by Python to compile:

   
• Python-2.6.1/Include
• Python-2.6.1/Objects
• Python-2.6.1/Python
• Python-2.6.1/Parser
• Python-2.6.1/Modules
• Python-2.6.1
• Python-2.6.1/Objects/stringlib (NOTE: this path must be included after "Tornado2.2/target/h" path)

Since Python source code is not optimized for VxWorks, a few modules have to be edited to better discriminate OS-dependant details.

The following files have to be removed from the project:

   
• "Modules/posixmodule.c"
• "Modules/main.c"
• "Parser/tokenizer_pgen.c"
• "Python/dup2.c"

Edit the "Modules\getbuildinfo.c" by substituting the Py_GetBuildInfo(void) function by the following code:
const char *
Py_GetBuildInfo(void)
{
    static char buildinfo[50];
    const char *revision = Py_SubversionRevision();
    const char *sep = *revision ? ":" : "";
    const char *branch = Py_SubversionShortBranch();
    PyOS_snprintf(buildinfo, sizeof(buildinfo),
        "%s%s%s, %.20s, %.9s", branch, sep, revision,
            __DATE__, __TIME__);
    return buildinfo;
}
And remove all the #define stuff about DATE and TIME in the head of the file.

Edit "Objects\stringlib\ctype.h" file by adding in the head these lines:
#ifndef Py_BYTES_CTYPE_H_MAX
#define Py_BYTES_CTYPE_H_MAX

and in the tail this line:
#endif

Edit also "Parser\tokenizer.c" adding in the head this line:
#define PGEN

In the "Module/config.c" file delete all the lines containg references to "initposix" and "initpwd" (should be 4 lines)

Now build the project.


Executing Python code on target

At this point a running Simulator has been created, and Python code can be executed in three ways:

1. By invoking Python Shell. Simply, call:

Py_Main 1, ""

2. By executing a single line expression. Enter this sequence:

Py_Initialize
PyRun_SimpleString "insert python code here"

3. By adding a C wrapper to execute .py files on host machine. To obtain this feature, add a function like this one in a separate .cpp file and build again the Simulator:

#include "Python.h"
#include "fioLib.h"

extern void Py_MyRun(const char* filename)
{
  FILE * fd = fopen(filename, "r");

  if(!Py_IsInitialized())
    Py_Initialize();

  PyRun_SimpleFile(fd, filename);

  Py_Finalize();

  fclose(fd);
}

And then call the function from VxWorks Shell:
Py_MyRun "host:C:/my_python_file.py"

Enjoy python!

GRANDE!!!

edited to obtain correct __DATE__ and __TIME__

First of all add Modules/socketmodule.c file to your VxWorks project.

This file looks like a wrapper of the basic C socket functions.
In order to compile (a part of) Python socket module you have to edit this file and another couple of files.

socketmodule.c editing
To use OS functions which (given a host name or a host address) return host C structure, you have to comment out all HAVE_GETHOSTBYNAME_R* directives and replace Unix C socket standard functions with VxWorks functions:

• gethostbyname becomes resolvGetHostByName
• gethostbyaddr becomes resolvGetHostByAddr

Then, comment out all functions based on an association between standard port number and service name (functions based on Unix /etc/services file, I suppose...):

• socket_getservbyname
• socket_getservbyport
• socket_getprotobyname

and you have to exclude these 3 functions from PyMethodDef socket_methods list.


Delete all calls to fcntl function (declaration of fcntl is missing in VxWorks fcntl.h file).
For example substitute line 638:

delay_flag = fcntl(s->sock fd, F_GETFL, 0);
with
delay_flag = 0;
and remove line 643:
fcntl(s->sock fd, F_SETFL, delay_flag);
Be careful: this is a very quick way to obtain a workaround on fcntl function, but maybe it is not the best problem solution!!


Add import and define directives useful to VxWorks, adding these lines (for example after line 260):
#include<socket.h>
#  define NO_DUP
int h_errno; /* not used */
#  define INET_ADDRSTRLEN 16

#define gethostbyname resolvGetHostByName
#define gethostbyaddr resolvGetHostByAddr


socketmodule.h editing
In order to import socket module from Python-shell typing "import socket", you have to edit also a #define contained in sockemodule.h.

#define PySocket_MODULE_NAME "socket"


getaddrinfo.c and getnameinfo.c editing
sockemodule.c imports also 2 C-file (getaddrinfo.c and getnameinfo.c), that use unmapped functions, so you have to edit also these 2 files.

In the file getaddrinfo.c, in the function getaddrinfo, comment out the part related to service name, i.e. comment out branch of the if chain related to service name (from line 335 to 376)
...
if (servname) {
...
In the file getnameinfo.c, in the function getnameinfo, comment you the branch of the if chain related to getservbyport function (from line 144 to 149)
...
sp = getservbyport(port, (flags ...
...


In the Modules/config.c file (automatically generated by configure script) add these lines by hand:

#include <socketmodule.h>

extern void init_socket(void);

and in the _inittab _PyImport_Inittab[] list add the following entry:

{"socket", init_socket},



Now compile the project.
Socket module will be available in VxWorks!

To obtain time module in your Python port to VxWorks, simply add the file Modules/timemodule.c to your project and build it!
Now you can invoke the sleep() function, for example, directly from the Python Shell

Nice post.

How long did the Python port take you?
What are the copyright issues?  VxWorks?  Python?

-Sam

Quote from: sbq on May 20, 2011, 08:45:32 PM
Nice post.

How long did the Python port take you?
What are the copyright issues?  VxWorks?  Python?

-Sam

Hi Sam,

the whole porting operation did not take too long: it has been done in the spare time in about three or four days for the main language porting.
The most of the time was spent in searching infos on the web or to understand the behaviour of VxWorks source files (when possible).

Copyright issues are related to the project I was working on (custom application based on VxWorks) so I could not post the complete integrated environment.

A cleaner way to do this porting should be the creation of a Python library: up to now the interpreter is merged into the main application code.

If you need more infos, feel free to ask.
  Max