I've been so busy that I didn't even had the time to put down this notes recently.
Long story short:
Parallel programming is writing programs where one or more parts are designed to potentially or actually run at the same time on different computational nodes (processors, computers, co-processors).
Multithreading programming (or simply threading) consists in using a set of tools to control physical computational parallel resources, together with their physical synchronization and sharing devices. Some multithreading structures abstract the final physical layer, but MT never gives completely off the ability to control the minute instruction flow on every processor you have at your disposal. Mutexes and MT-wise semaphores (i.e. MS-Windows events or POSIX condition variables) may be implemented differently on different processor architectures, but their point is granting that the underlying processors and/or the operating system tightly bound to it will do a set of operation granting some specific behavior of the hardware.
So, is MT parallel programming? -- No, but you can DO parallel programming with MT tools/programs/libraries. When using MT, either you want to control the underlying hardware parallel abilities for other reasons (and you don't want to mess with the raw ASM needed to do that), or you're writing your own parallel programming environment, for how simple it is, to make your program to do tasks in parallel.
After some years in the field and in the field of programming languages, I finally came to the conclusion that high level languages, and especially scripting languages, doesn't need MT. Worse, exposing MT-level operations and concepts to high level scripts (threads, mutexes, and so on) can harm more than help.
What you want when using a low level language is to be low level as low as you can, but not lower. In C and C++, you want threads, CPU affinity masks, mutexes, condition variables. What you want when you use an high level language is to be high level as much as you can, but not more. Like, doing all the messy stuff of a whole C program in a couple of well-designed instructions or commands. You want to tell the system the LOGIC of your THOUGHTS, and then you want the system to do the stuff for you.
In short, the next Falcon version will have threads and parallel structures removed, and will embed threading in the language itself, not differently from Erlang or Ocalm. Coroutining may be reviewed as well, and be considered a special case of threading. The idea is to be able to define parallel code, and find the compiler (for what can be told at compile time) and the VM (for the runtime conditions) to run it "as parallel as possible". A VM may be assigned the limit of 0 thread, thus making all the parallel code to run as coroutines.
More on the upcoming idea in the next post.
Monday, June 28, 2010
Tuesday, May 11, 2010
Last night we (Maik, Paul, Luca and me) had an interesting talk on "variable classes". Many scripting languages allow to change the structure of a class, and doing so, changing the runtime structure of the instances they have generated. Although this is generally regarded as "horror" in terms of object oriented programming, and relying on this is considered a bad practice, Maik and Paul (exp. Paul) convinced me that the thing has "a meaning" when profiling the an application at design/setup time.
Also, if specifically constrained and shaped down with the help of well-defined language extensions, as the type contracts we're designing, altering a class may not be as bad as it sounds.
Also, if specifically constrained and shaped down with the help of well-defined language extensions, as the type contracts we're designing, altering a class may not be as bad as it sounds.
[ Read More... ]
Thursday, April 08, 2010
Ok, the problem with the "opaque object* and class handler" model I just posted about is inheritance. But it's a problem even now, as pure CoreObject derived entities cannot be inherited at all, while more complex FalconObject based entities can inherit only from ONE non-pure falcon base class.
This should fix the problem and allow interface-based classing to work.
The system is based on 3 classes: the dear old CoreClass implementing pure language-level classes, a base abstract class for manipulation of foreign data (UserClass) and a class glueing the two, called HyperClass. A new CoreClass can be derived from any count of base CoreClass parents; it knows that its opaque "object" is actually an array of items parallel to its property table (and there's a good news about property tables too). The CoreClass is meant to be final (or nearly so).
UserClass doesn't provide any default way to access its properties and makes no assumption on the nature of the opaque data used as object to be manipulated.
The HyperClass is created when linking a CoreClass with a UserClass-derived entity. As the new model doesn't require anymore a VM to be there to link a final class, nor the class structure declaration to be unmodifiable, the CoreClass itself can perform its own link (up to the constructor call). If all its parents are CoreClasses, it will return itself modified so that it has the union of properties provided by the parents (using the current logic). If one or more parents are not CoreClasses (if they are user or hyper classes), it creates an HyperClass that will take its place.
The Hyperclass knows its opaque structure to be an HyperObject, that is, an array of opaque pointers where the element 0 MAY be occupied by an item array storing data coming from the CoreClasses, while the others are the opaque pointers known and manipulated by each non-CoreClass parent.
The HyperClass handles its properties differently from CoreClasses: it holds a list of HyperProperties each indicating the owning class (after link resolution), the position of the owning class in the HyperObject array (0 if the parent is a CoreClass) and, in case the property is coming from the CoreClass hierarcy, the position to access the property.
In this way, if willing to access a property provided by a UserClass (of which Falcon knows nothing about the internal object manipulation logic), it will just hand down to the right class the request to get that property on an opaque data of the type that the UserClass is expecting to manipulate.
In case of explicit subclass access (i.e. HyperClass.BaseClass.prop), the process is even simpler. If the BaseClass is a CoreClass, we simply ask for the default value of that property (as defined by our standard); if it's a UserClass, we simply hand down the request for that property to the UserClass, feeding it with its own data (Note to me: a flag should indicate if we're taking the default property via the baseclass access).
The HyperClass fits also the interface object picture.
Interfaces are inherited exactly like objects. When forming an HyperClass, if a subclass provides an interface object (i.e. the sequence interface for hooking into for/in, or the array subscript interface), the winning interface gets wrapped into a HyperInterface for the same task, which records that interface and the position in the HyperObject that must be passed to that interface if invoked.
This makes possible to turn all the basic structures into UserClasses, with the ability to be even derived. The special callback hooks used for operator overload can force language-based CoreClasses to push in the hierarcy a standard InterfaceObject, whose action is that of calling the required callback. This allows, for example to override the language-defined action for an Array (declared as a UserClass) providing a __getIndex callback on a language level Array subclass (implemented as a CoreClass at module level, and becoming a HyperClass during the link step).
... cool, huh?
This should fix the problem and allow interface-based classing to work.
The system is based on 3 classes: the dear old CoreClass implementing pure language-level classes, a base abstract class for manipulation of foreign data (UserClass) and a class glueing the two, called HyperClass. A new CoreClass can be derived from any count of base CoreClass parents; it knows that its opaque "object" is actually an array of items parallel to its property table (and there's a good news about property tables too). The CoreClass is meant to be final (or nearly so).
UserClass doesn't provide any default way to access its properties and makes no assumption on the nature of the opaque data used as object to be manipulated.
The HyperClass is created when linking a CoreClass with a UserClass-derived entity. As the new model doesn't require anymore a VM to be there to link a final class, nor the class structure declaration to be unmodifiable, the CoreClass itself can perform its own link (up to the constructor call). If all its parents are CoreClasses, it will return itself modified so that it has the union of properties provided by the parents (using the current logic). If one or more parents are not CoreClasses (if they are user or hyper classes), it creates an HyperClass that will take its place.
The Hyperclass knows its opaque structure to be an HyperObject, that is, an array of opaque pointers where the element 0 MAY be occupied by an item array storing data coming from the CoreClasses, while the others are the opaque pointers known and manipulated by each non-CoreClass parent.
The HyperClass handles its properties differently from CoreClasses: it holds a list of HyperProperties each indicating the owning class (after link resolution), the position of the owning class in the HyperObject array (0 if the parent is a CoreClass) and, in case the property is coming from the CoreClass hierarcy, the position to access the property.
In this way, if willing to access a property provided by a UserClass (of which Falcon knows nothing about the internal object manipulation logic), it will just hand down to the right class the request to get that property on an opaque data of the type that the UserClass is expecting to manipulate.
In case of explicit subclass access (i.e. HyperClass.BaseClass.prop), the process is even simpler. If the BaseClass is a CoreClass, we simply ask for the default value of that property (as defined by our standard); if it's a UserClass, we simply hand down the request for that property to the UserClass, feeding it with its own data (Note to me: a flag should indicate if we're taking the default property via the baseclass access).
The HyperClass fits also the interface object picture.
Interfaces are inherited exactly like objects. When forming an HyperClass, if a subclass provides an interface object (i.e. the sequence interface for hooking into for/in, or the array subscript interface), the winning interface gets wrapped into a HyperInterface for the same task, which records that interface and the position in the HyperObject that must be passed to that interface if invoked.
This makes possible to turn all the basic structures into UserClasses, with the ability to be even derived. The special callback hooks used for operator overload can force language-based CoreClasses to push in the hierarcy a standard InterfaceObject, whose action is that of calling the required callback. This allows, for example to override the language-defined action for an Array (declared as a UserClass) providing a __getIndex callback on a language level Array subclass (implemented as a CoreClass at module level, and becoming a HyperClass during the link step).
... cool, huh?
Talking with the people in #falcon IRC channel (at irc.freenode.org), we worked out the concept a bit, and we found out that the best place where to store the interface objects is the Falcon "CoreClass" instance.
The CoreClass has the meaning to describe what Falcon can do with the data you pass. Rather than storing the knowledge of how to handle the object in the CoreObject hierarcy, we can think of the object as a totally opaque entity, and store all the knowledge about it in the CoreClass.
This allows embedders and modules to provide totally opaque data (their data) and separate it from the code that Falcon should use to handle it.
The way to associate CoreClasses with the opaque data they should handle is still under investigation. One option is that of just storing the class and its opaque data in the item, but then we must make sure that a method creation (obj.mth) has a way to refer the class. Shouldn't be too hard in the new model we're developing, as the methods may now have a back-reference to the classes from which they come from.
Another option may that of having a pool of pre-allocated small buffers where the opaque-data / handler-class pair can be stored, and then store that pointer in the items. This has the advantage that we're sure that the opaque data can never be disconnected from the class, but it requires extra allocation, memory management and dereferences.
The current coupling of methods and objects on which they operate solves a problem similar to this, and has been proven one of our most solid assets since first alpha release; so, keeping the pair class/opaque-object or method(class)/opaque-object directly in the item structure seems a very smooth and viable solution.
The CoreClass has the meaning to describe what Falcon can do with the data you pass. Rather than storing the knowledge of how to handle the object in the CoreObject hierarcy, we can think of the object as a totally opaque entity, and store all the knowledge about it in the CoreClass.
This allows embedders and modules to provide totally opaque data (their data) and separate it from the code that Falcon should use to handle it.
The way to associate CoreClasses with the opaque data they should handle is still under investigation. One option is that of just storing the class and its opaque data in the item, but then we must make sure that a method creation (obj.mth) has a way to refer the class. Shouldn't be too hard in the new model we're developing, as the methods may now have a back-reference to the classes from which they come from.
Another option may that of having a pool of pre-allocated small buffers where the opaque-data / handler-class pair can be stored, and then store that pointer in the items. This has the advantage that we're sure that the opaque data can never be disconnected from the class, but it requires extra allocation, memory management and dereferences.
The current coupling of methods and objects on which they operate solves a problem similar to this, and has been proven one of our most solid assets since first alpha release; so, keeping the pair class/opaque-object or method(class)/opaque-object directly in the item structure seems a very smooth and viable solution.
Sunday, October 11, 2009
I've been able to write the first prototype of "The Evolution Game".
It's a competitive programming game where two falcon scripts are fighting for survival in an arena with limited space and resources. The new twist in it is that, except for extremely naive strategies, it's practically necessary for each script to evolve and mutate in order to win.
We plan to build a contest on that, inviting programmers to submit their scripts, and see who is the best evolutionary programmer in the world.
There's also a small secret, that people experts of fractals, high order numerical-math, statistics and evolutionary computation will probably spot very soon. But I will keep silent about this 'till the end of the contest.
For now, I am pretty impressed about the power of the TEG program itself. I thought I could support an arena of 100x100 cells, each being a running Falcon script, called back several times per turn (one to five), at 1 turn per second, after introducing a bit of concurrent calculations. Contrarily to my expectations, the prototype, which is still largely unoptimized and totally single threaded, is running a 100x100 arena at five to eight turns per second. The script used in the prototype are largely trivial, but even making them three times as complex as they are, we'd still have a better performance than I expected after having introduced all the possible optimizations and parallelism.
We're gonna have fun with this toy.
It's a competitive programming game where two falcon scripts are fighting for survival in an arena with limited space and resources. The new twist in it is that, except for extremely naive strategies, it's practically necessary for each script to evolve and mutate in order to win.
We plan to build a contest on that, inviting programmers to submit their scripts, and see who is the best evolutionary programmer in the world.
There's also a small secret, that people experts of fractals, high order numerical-math, statistics and evolutionary computation will probably spot very soon. But I will keep silent about this 'till the end of the contest.
For now, I am pretty impressed about the power of the TEG program itself. I thought I could support an arena of 100x100 cells, each being a running Falcon script, called back several times per turn (one to five), at 1 turn per second, after introducing a bit of concurrent calculations. Contrarily to my expectations, the prototype, which is still largely unoptimized and totally single threaded, is running a 100x100 arena at five to eight turns per second. The script used in the prototype are largely trivial, but even making them three times as complex as they are, we'd still have a better performance than I expected after having introduced all the possible optimizations and parallelism.
We're gonna have fun with this toy.
Saturday, August 08, 2009
There's who says that it's "just another language", and thus, "useless", but I got users pretty loving it.
This is a chat log from our channel:
This is a chat log from our channel:
ago 08 16:01:23 olefowdie also, the package manager I told you I was creating will now probably become the basis for yet another linux distribution... and I am trying to think of more stuff i could write for it in falcon pl...
ago 08 16:01:35 olefowdie since falcon is secretly the code for the whole friggin universe.
ago 08 16:01:36 jonnymind olefowdie: I love you.
ago 08 16:01:57 olefowdie I bet there are aliens secretly stealing wifi access from ISS just to grab copies of Falcon.
ago 08 16:02:43 olefowdie jonnymind: i will as soon more is working
ago 08 16:03:46 olefowdie I wonder... you know if martians exist, they probably have a secret underground shrine to FalconPL
ago 08 16:03:50 jonnymind heheh
ago 08 16:03:57 jonnymind :-))
ago 08 16:04:36 olefowdie ... and I bet that God's rendering engine for the universe is written in FalconPL- proof = duck bill platapus (just as versatile in feature listings)
ago 08 16:05:50 * btiffin (i=brian@CPE000802e558f0-CM0014f8cd183a.cpe.net.cable.rogers.com) รจ entrato in #falcon
ago 08 16:06:48 jonnymind *this one is going on my blog
ago 08 16:07:01 olefowdie lol.
Friday, February 20, 2009
Despite of the fact that I am currently the only active developer, and despite of the fact that other developers joining randomly in the project have been contributing to about 2% of the code, people is coming gathering around the project. We've got Pete taking good care of the documentation (my Italian English couldn't stand for a long time), and several visitors of our site have been starting writing Wikipedia pages for Falcon in various languages. Then, we have the AuroraUX distro (SunOS) electing Falcon as one of the language of choices (directly available in the core packages), and Blastwave (the project porting GNU/open source projects to OpenSolaris). And Kross, bridging Falcon and KDE 4.2.
All this encourages me to push forward the 0.9 release and to seek for more developers willing to join and help in the project. Future of Falcon Programming Language seems bright; thanks to everyone supporting our ideas.
All this encourages me to push forward the 0.9 release and to seek for more developers willing to join and help in the project. Future of Falcon Programming Language seems bright; thanks to everyone supporting our ideas.
Sunday, November 30, 2008
It took about 2 full days just to prepare the packages, the ground for the packages, test them, repackage also the modules that were impacted by the changes and so on, but now it's done.
Well, almost. Still need to make a new release of the SDL module. Will do in a couple of hours.
It seems that open source is all about releases; and in part it's true. OTOH, for a developer there are but a few things as distressing and time consuming as making a release (with downloadable packages). And I still have to do house cleaning stuff as SVN tagging, spreading the news to the packagers in the various distro and the like...
Next time I must repeat the mantra: "Making a release takes a week... making a release takes a week..."
For the curious; if you're asking yourself why this release is called Vulture, it's because it fed on Condor.
Well, almost. Still need to make a new release of the SDL module. Will do in a couple of hours.
It seems that open source is all about releases; and in part it's true. OTOH, for a developer there are but a few things as distressing and time consuming as making a release (with downloadable packages). And I still have to do house cleaning stuff as SVN tagging, spreading the news to the packagers in the various distro and the like...
Next time I must repeat the mantra: "Making a release takes a week... making a release takes a week..."
For the curious; if you're asking yourself why this release is called Vulture, it's because it fed on Condor.
Wednesday, November 26, 2008
Ok, so we'll have a bugfix release to consolidate some very relevant bugfixes we have performed in this couple of weeks.
Also, we'll go for a major number, as one bugfix introduced a binary incompatibility. To be correct, it was not a bugfix; I just had some unclean interface (the relatively obsolete UserData class) that leaded me to write an unclean code in a secondary module (sdl_image binding). I can't resist fixing unclean things, and I thought that if this can lead me to write unclean code, it would certainly drive others in the same errors. Although UserData was the main mean to do reflection before, while it is now just a residual quick-but-not-dirty resource to write a binding quickly, it has still a role, so it will stay in 0.9. If it has to be cleaned, better now than later.
The release is scheduled by the next saturday.
Also, we'll go for a major number, as one bugfix introduced a binary incompatibility. To be correct, it was not a bugfix; I just had some unclean interface (the relatively obsolete UserData class) that leaded me to write an unclean code in a secondary module (sdl_image binding). I can't resist fixing unclean things, and I thought that if this can lead me to write unclean code, it would certainly drive others in the same errors. Although UserData was the main mean to do reflection before, while it is now just a residual quick-but-not-dirty resource to write a binding quickly, it has still a role, so it will stay in 0.9. If it has to be cleaned, better now than later.
The release is scheduled by the next saturday.
Monday, November 24, 2008
It wasn't originally my intention to release a bugfix version before 0.9, which is due for mid-january. The point is that I nailed down a couple of nasty bugs really fast, and I took the occasion to add a couple of interesting things that I was going to add in 0.9 anyhow. Also, if I didn't make any relevant mistake, the code is currently binary-compatible with 0.8.12, and so other modules and applications compiled with it doesn't need re-compilation. So, I'd like to consolidate the bugfixes before changing definitely the inner 10,000 lines composing the core of the system to prepare them for the final round of 0.9->1.0.
I wonder if I should call it 0.8.12.2 or 0.8.14...
I wonder if I should call it 0.8.12.2 or 0.8.14...
Saturday, November 15, 2008
Being hit once by your own decision is part of the life game, but being hit twice is more than I can stand. In the initial development of Falcon, I thought that the protection provided by the special path for load/import directive was enough to distinguish Falcon modules from system modules. I was right, but there was another aspect I didn't take into account: that Falcon modules may need to dynlink those unprotected libraries.
Example: the SDL module. The Falcon module was called sdl.so on POSIX and sdl.dll on Windows; on POSIX, system libraries as the SDL system, require a "lib" prefix, so the dynamic library needed by the Falcon module is "libsdl.so.*". Pitifully, on Windows it's not the same; SDL system is driven by SDL.dll. The Falcon module loader is anyhow able to load the proper falcon module on "load sdl" command, but the OS dynamic linker resolves the request of the module for the system SDL.dll into... a request to load itself!
As a result, Falcon SDL module is temporarily named fsdl. When the same thing happened with ODBC module, I decide that in release 0.9 we'll have a sensible suffix attached to falcon module names, for example "sdl_fm.dll", so that we can "load sdl" without confusing the module loader nor the system dynamic linker.
Example: the SDL module. The Falcon module was called sdl.so on POSIX and sdl.dll on Windows; on POSIX, system libraries as the SDL system, require a "lib" prefix, so the dynamic library needed by the Falcon module is "libsdl.so.*". Pitifully, on Windows it's not the same; SDL system is driven by SDL.dll. The Falcon module loader is anyhow able to load the proper falcon module on "load sdl" command, but the OS dynamic linker resolves the request of the module for the system SDL.dll into... a request to load itself!
As a result, Falcon SDL module is temporarily named fsdl. When the same thing happened with ODBC module, I decide that in release 0.9 we'll have a sensible suffix attached to falcon module names, for example "sdl_fm.dll", so that we can "load sdl" without confusing the module loader nor the system dynamic linker.
Monday, October 27, 2008
Release 0.8.12 is issued. I am working with PPL of various distros (and side by side with Blastwave guys) to bring the official release in the latest distros. Windows and Mac versions are downloadable directly from the site.
Cheers!
Cheers!
Friday, October 03, 2008
I adapted one of the standard LUA samples "life" (a famous early days computer simulation) to see how much space we have to cover in the 0.9 series (the optimizing round).
The first rough result were quite sad, as Falcon run the simulation in about 300% of the time required by LUA. But we have still unbuffered char-by-char output in standard print function, and LUA has already an optimized bytecode compiler, while we have none.
So, I changed the output function and used our stream block write, and performed some very simple hand optimization as i.e. caching the "self" accessors used in wide loops, and used Falcon specific (faster) loop instructions. The result were far more pleasing...
The first rough result were quite sad, as Falcon run the simulation in about 300% of the time required by LUA. But we have still unbuffered char-by-char output in standard print function, and LUA has already an optimized bytecode compiler, while we have none.
So, I changed the output function and used our stream block write, and performed some very simple hand optimization as i.e. caching the "self" accessors used in wide loops, and used Falcon specific (faster) loop instructions. The result were far more pleasing...
[ Read More... ]
Tuesday, September 23, 2008
And with them, full blown tabular programming.
I just need to add some API level method to the Table class; everything at VM level is ready.
With the last round, methods called from dynamic bindings in arrays can access the calling array through the "self" keyword. This lets to build dynamic objects (and classes) not unlike LUA programmers are accustomed to do, but in Falcon there is a significant shift. If an array is part of a table, that is, if it's a row, then properties are taken from the table, and they resolve into an item of the array. So, while bindings stay outside (or besides) the array, table properties lie inside the array itself. Still, a method can access the owning array, and indirectly the owning table, via the "self" item.
I just need to add some API level method to the Table class; everything at VM level is ready.
With the last round, methods called from dynamic bindings in arrays can access the calling array through the "self" keyword. This lets to build dynamic objects (and classes) not unlike LUA programmers are accustomed to do, but in Falcon there is a significant shift. If an array is part of a table, that is, if it's a row, then properties are taken from the table, and they resolve into an item of the array. So, while bindings stay outside (or besides) the array, table properties lie inside the array itself. Still, a method can access the owning array, and indirectly the owning table, via the "self" item.
[ Read More... ]
Sunday, September 14, 2008
It's official. Falcon will be in the next number of Linux Journal; I wrote the article, but LJ editorialists have selected it to be rightfully standing besides the top news: an interview to Guido Van Rossum on Python 3000.
The issue preview.
Falcon gained also some square centimetres on the front cover.
I cannot reproduce the article for obvious reasons; it's just a generic introduction and an exert of the "showdown" document, with some of the most coolest differential features outlined in the last part.
I kept silent up to the moment I received the 3 copies of the magazine at home as an article writer; I didn't want to burn the article in any way. After the issue is out, I will finally set for some serious media strategy by contacting more magazines and PC related media. Things are ready to welcome a first group of wilful developers and users, so it is now high time for the Falcon to fly.
The issue preview.
Falcon gained also some square centimetres on the front cover.
I cannot reproduce the article for obvious reasons; it's just a generic introduction and an exert of the "showdown" document, with some of the most coolest differential features outlined in the last part.
I kept silent up to the moment I received the 3 copies of the magazine at home as an article writer; I didn't want to burn the article in any way. After the issue is out, I will finally set for some serious media strategy by contacting more magazines and PC related media. Things are ready to welcome a first group of wilful developers and users, so it is now high time for the Falcon to fly.
