A draft for a Io Module System²²!! Introduction²²The Io programming language does not come by default with a module system. A module system (or a package systems) aims at allowing to load specific blocks of code into a program during its runtime. The advantage of modules is that they can be shared among different programs.²²Having a module system allow developers to write extensions, and if properly managed, to share them and hence collaboratively build a "standard library". Languages such as Python or Ruby are not only attractive because of their traits, but also (if not mainly) because of their library.²²Although Io was mainly designed to be en embedded language, it has the potential of also being a stand-alone language. In this perspective, a module system is an essential step toward this aim.²²Related pages:²²* IoLanguage²* IoDocumentationSystem²²----²²!! Requirements²²This section presents a list of requirements related to the conception of a modern module system. These requirements are inspired from existing module system implementations (most notably Python, Perl and Java)²²* Loading:²** Load a module at runtime (dynamic loading)²** Reload an already loaded module²** Load modules from a set of defined repositories²** Ensure modules that were have already been loaded are not loaded again.²²* Importing:²** Import part of a module²** Automatic module dependecy/conflict resolution²** Import module with specific options (like optimised version)²²* Installing:²** Global module database²** Automatic installation, update and removal of modules²²* General:²** Get meta-data (author, license, etc) on a module²** Allow modules to be either pure Io or written in C²** Get information on installed or available modules²** Allow modules to be moved to different locations without having to reinstall them²** Provide developers and quality guidelines to preserve consistency among modules²** Provide tools to create, check and validate modules²²* Initialising:²** Modules should not start event loops or perform other complex actions²automatically, but should wait for explicit initialization. This allows an²utilities like document generators and editors to load a module, and analyze²its overall structure and documentation.²** Ensure modules that have been initialized are not initialized again.²²* Security:²** Sandbox a module²** Add authorisation access to modules, so that if b imports a and c import b, c may not be authorised to access a.²²----²²!! Design questions²²!!! 1. Should the module be instanciated when imported ?²²!!!! Details:²²Look at the following code:²² ² IoPackage import "MyModule" as "Module1"² IoPackage import "MyModule" as "Module2"² ² Module1 anObject = "Hello"² Module1 getSlot("anObject")² ==> "Hello"² Module2 getSlot("Hello")² ==> Nil² ²²When modules are instanciated, they do not share the same data, they are in fact cloned.²²!!!! Discussion:²²Advantages:²* Module instanciation prevents modification of modules by other scripts in the VM that would also import it, which is a good point for security.²²Disadvantages:²* Scripts cannot communicate by sharing modules²* There could be some problems if modules that contain stuff like an event loop are imported, initialised and run twice²* Cloning modules may eat up a lot of unnecessary memory.²²References:²* Python instancitates its modules.²²----²²!! Proposition²²!!! Packages and modules²²Module: a module is a collection of object (which can be prototypes) Package: a package²is a collection of package or modules, only.²²[definitions to be detailed]²²!!! Meta-information²²!!! Dependecy and conflict resolution algorithms²²----²²!! Examples²²The following is just a conjecture by sdunlop:²²Creating a module:²² ² MyModule = Module clone("MyModule") document ("""² This module implements ...""")² ² MyModule load = method(² self MyPrototype? = Object clone documentPrototype ( ... )² MyPrototype init = method( ... ) document ( ... )² )² ² MyModule setup = method( ... )² ² MyModule teardown = method( ... )² ² MyModule author = "Scott Dunlop"² MyModule author-email = "sdunlop@midcoast.com"² MyModule author-website = "http://www.midcoast.com/~sdunlop"² MyModule majorVersion = 0² MyModule minorVersion = 1² MyModule stability = ALPHA² export(MyModule)² ²²Importing a Module and a Prototype:²² ² HisModule = importModule("MyModule")² HisPrototype = HisModule import ("MyPrototype")² ²²Alternatively, the classic naive "from module import *" equivalent:²² importAllFromModule("HisModule")²²----²²Here is an alternative proposal, using a IoPackage?? object (I make a difference²between packages and modules - think of packages as nodes and modules as leaves):²² ² #Import all from a module² IoPackage from ("MyModule") importAll² ² #Import protos from a module into the Lobby² IoPackage from ("MyModule") import ("MyProto")² IoPackage from ("MyPackage1.MyPackage2.MyModule") import ("Proto1", "Proto2")² ² #Import protos from a module into the Lobby as another Proto² IoPackage from ("MyModule") import ("MyProto") as ("RenamedProto")² IoPackage from ("MyModule") import ("MyProto", "MyProto2") as ("RenamedProto1","RenamedProto2")² ² #Import only a module/package into the lobby² IoPackage import ("MyModule")² ² #Import only a module/package and rename it² IoPackage import ("MyModule") as ("RenamedModule")² ²²IoPackage? would support resolution of modules given the dot-separated module name. Forinstance "Package1.Module1" would first look for the "Package1" somewhere (localdirectories, tarballs, URLs?) and then when Package1 is found, ask resolution of Module1to Package1.²²I think it is important to provide a IoPackage? object because we could implement²different strategies for importing modules, some that may take into account security, orprovide fancy loading features.²²-- sebastien²²----²² I prefer the verbosity of your method to my own, but only slot names composed²entirely of symbols can be used as operators in Io, therefore, something like:²² IoPackage import "MyModule" as "RenamedModule"²²should be expressed as:²² IoPackage import("MyModule") as ("RenamedModule")²²--sdunlop²²----²Cool :) I fixed this --sebastien²²----²²The major issue, for me, is how to keep the modules from interacting with each other in unconstrained ways. Consider three modules, main.io:²² require("Foo")² require("Bar")²²Foo.io:²² foobar = 42²²and Bar.io:²² write(Foo foobar, "\n")²²Unless this is explicitly forbidden somehow, these modules are legal but ill-formed. Bar depends upon the existence of Foo, but it does not state this requirement. Thus our dependency graph is incomplete and the meaning of Bar changes based upon its context. It might raise an error because "Foo" or "Foo foobar" don't exist, print 42, or do something completely different. You can't really tell what Bar is going to do by looking at its code.²²How then do we restrict access between two modules within the current framework of Io? The only way I can think of doing it is by changing the meaning of Lobby. Instead of Lobby being the object in which all global code is run, Lobby becomes a system-defined module which is implicitly imported into all other modules. Every module is its own namespace.²²Now consider the above example of main, Foo, and Bar again. The 'main' module requires Foo and Bar, which have not yet been loaded. The system tracks down the implementations associated with Foo and Bar and executes them. Foo runs in its own namespace and then added to main's Protos object (to indicate that main has imported it and can now access it). Now Bar runs in its own namespace, which doesn't define Foo. It looks at its proto, which points to the Protos object, which only contains Lobby -- Lobby also does not define Foo. Thus, we have an error, as one might expect from looking at Bar's code. If Bar is changed to:²² require(Foo)² write(Foo foobar, "\n")²²then the system recognized that the Foo module is already loaded and just adds the existing Foo object to Bar Protos, allowing Bar to access Foo.²²Thus, we have the case that if Bar attempts to use Foo's interface, it must either explicitly require Foo or raise an error. This only leaves one hole, whereby modules can interact without explicit requirements between them. If Foo were changed to add foobar to the Lobby instead of its namespace (thisModule), then Bar could access foobar through the Lobby. I think this is unlikely to pose a problem and it may actually be a good thing to allow users to circumvent some of the module system's requirements when they really want to.²²Even though this is a significant change to how namespaces are handled in Io, it will only break programs that attempt to refer to the slots they're adding through the Lobby. Those programs would need to be changed to refer use thisModule, instead.²²As a last note, since this is already a very long addition to the page (sorry!), I've basically removed the notion of explicitly declaring that a file is a module, in favor of having Io automatically treat the first file it loads and every required file after that implicitly as a module. Having a file declare itself a module breaks doFile() and requires the interpreter to keep track of more information.²²I call the module loading facility require() instead of import() because import(), to me, implies that you're making the definitions of a module available inside the current one. I view import() as a separate mechanism:²² require("Foo") # Make the Foo module object available.² Foo import("foobar") # Bring in the foobar object.²²It may be worthwhile to provide another module loader, request(), which defines the module object as Nil if it fails, instead of throwing an exception (as I imagine require() doing). This might be useful in combination with module versioning:²² request("GTK", majorVersion == 2)² if(GTK == Nil) then(² request("GTK", majorVersion < 2)² # Build a compatability layer for older versions of GTK.² )² # code common to all versions of GTK we support.²²or in general:²² request("ReadLine")² require("File") import("standardInput")² ...² # Use full line-editing capabilities, if ReadLine package is available.² expr += if(ReadLine, ReadLine readLine, standardInput readLine)² ...²²-dak²²------------------------------------------------------------------------------------²²References²²* The Non-Pareil module system²* The Fink project²* Python Package Index, http://www.python.org/pypi²* Scheme FAQ, question about module system,²http://www.schemers.org/Documents/FAQ/#id2835125²* IoDocumentationSystem -- A specification for automatically generating documentation²that relies on modules behaving predictably.²²------------------------------------------------------------------------------------²²Comments