A draft for an Io Module System

Introduction

The Io programming language does not come by default with a module system. A module system (or a package systems) aims at allowing to load specific blocks of code into a program during its runtime. The advantage of modules is that they can be shared among different programs.

Having a module system allow developers to write extensions, and if properly managed, to share them and hence collaboratively build a "standard library". Languages such as Python or Ruby are not only attractive because of their traits, but also (if not mainly) because of their library.

Although Io was mainly designed to be en embedded language, it has the potential of also being a stand-alone language. In this perspective, a module system is an essential step toward this aim.

IoLanguage
DocumentationSystem
ModuleSystemDiscussion, which is a forum regarding this document

Requirements

This section presents a list of requirements related to the conception of a modern module system. These requirements are inspired from existing module system implementations (most notably Python, Perl and Java)

Loading:
- Load a module at runtime (dynamic loading)
- Reload an already loaded module
- Load modules from a set of defined repositories
- Ensure modules that have already been loaded are not loaded again.
Importing:
- Import part of a module
- Automatic module dependecy/conflict resolution
- Import module with specific options (like optimised version)
Installing:
- Global module database
- Automatic installation, update and removal of modules
Security:
- Sandbox a module
- Add authorisation access to modules, so that if b imports a and c import b, c may not be authorised to access a.

General:
- Get meta-data (author, license, etc) on a module
- Allow modules to be either pure Io or written in C
- Get information on installed or available modules
- Allow modules to be moved to different locations without having to reinstall them
- Provide developers and quality guidelines to preserve consistency among modules
- Provide tools to create, check and validate modules
Initialising:
- Modules should not start event loops or perform other complex actions automatically, but should wait for explicit initialization. This allows an utilities like document generators and editors to load a module, and analyze its overall structure and documentation.
- Ensure modules that have been initialized are not initialized again.

Proposition

This section presents our proposition for a module system for the Io language. We will introduce the concepts of packages and modules, the notion of module meta-information, concerns regarding security, and eventually a prototype implementation.

Packages and modules

We would like to propose the concepts of package and module as the fundamental concepts of a module system.

Packages and modules are elements that allow to express the structure of Io code:

Packages allow to structure and organise modules
Modules allow to structure and organise Io objects (code).

For instance, considering the application of packages and modules to a filesystem as illustrated below:

  .
  |-- org
  |   `-- typez
  |       |-- io
  |           |-- Pouet.io

We would consider org, typez and io as packages, and then the Pouet.io module.

Meta-information

A module system is not solely a way to structure and organise piece of code : modules have dependencies, authors, licenses, security policies and so on. In this respect, it is important to express all this information on modules. This information is called the meta-information.

Packages, as opposed to modules, do not carry any meta-information. Their only purpose is to organise, but not to express dependencies or other properties. This information is let to modules only.

The meta-information defined by modules is the following:

Version: (Major, Minor)
Dependencies: a list of depending module names
Authorisations: a list of authorised or unauthorised modules

Other meta-information such as authors, license, and so forth are redundant with our DocumentationSystem proposal.

Modules and package naming

Module names should be in MixedCase with only alphanumeric charcters, while package names should be lowercase, also only with alphanumeric characters. This allows to visually identify what is a package and what is a module in a given name

A canonical name name corresponds to a dot-separated list of packages names, with an optional ending module name.

Example of canonical names include myackage, MyModule?, myackage.MyModule?, myackage.myotherpackage.MyModule?, but of course not MyModule?.mypackage, because modules cannot contain packages.

Resolving modules according to their name

Modules could be contained either on a filesystem, or in an archive (such as a tarball or zipfile). In this respect, the resolving of packages and modules should be made by iterating from left to right on the dot-separated list of package and module names that compose a canonical name. Each name should be resolved according to the context in which the last package is located.

For instance, imagine that we have the following layout:

  `
  |-- package
  |   `-- otherpackage.zip

Now, imaging that otherpackage.zip contains the following structure:

  `
   |-- spam
   |   `- PouetPouet?

The, the canonical name package.otherpackage.spam.PouetPouet? will be valid. The resolution of module will first look in a designated filesystem location for a package folder or zip file, then it will look for an otherpackage folder or zip file.

At this point, the resolution of modules changes to support exploration of zip files and not modules, and continues its resolution with spam and PouetPouet?.

Dependecy and conflict resolution algorithms

Security and the module system

API proposal

Prototypes:

Module: the module object prototype
Package: the package object prototype
Resolver: interface for resolving modules from canonical names
LocalResolver ?: resolves packages and modules on the local filesystem
HTTPResolver ?: resolves packages and modules with http requests
ZipResolver ?: resolves packages and modules in zip files
NativeResolver ?: resolves packages and modules with shared objects
Loader: generic interace for loading a module
IoLoader ?: loads a module under the form of a Io script
NativeLoader ?: loads a module under the form of a native shared object

Prototype implementation

References

The Non-Pareil module system
The Fink project
Python Package Index, http://www.python.org/pypi
Python SandBox module [1]
Scheme FAQ, question about module system, http://www.schemers.org/Documents/FAQ/#id2835125
DocumentationSystem -- A draft for automatically generating documentation that relies on modules behaving predictably.

Discussion

Discussion was moved in ModuleSystemDiscussion.....

Io/ModuleSystem