Packages organization and exporting symbols

I’ve started to re-design my main library for evolutionary computation. One of the main things I did for the new version was a complete new organization of the packages (and respective files/modules). Before I had essentially two main packages, the library itself and the examples. Although simple, it became a pain to use this model when I extended it heavily with more algorithms and related utilities. I hope I am not going now in the opposite direction (too complicated) but so far I like the new organization.

In short, there is a package for each main branch of algorithms (e.g., GA, GP) with everything specific that kind, which imports from a core package with the common components. These “sub-packages” are gathered together in a single package (the main library package). This way, it is possible to use in a project everything or simply just the desired component (e.g., if you just want GP). Furthermore, an extra package for the users is also provided to allow REPL experimentation without being on the library main package.

However, while implementing this scheme I realized that I wanted to have all the exported symbols from the packages that compose the library, also exported by the library main package. This way, all the symbols that compose the library are easily seen on the main package. For me this is very useful since it allows exploration of a library, especially if it has many things. Since I have never done something like this before, I went and search for a way to solve this minor problem in an easy way.

The answer is basically use do-external-symbols. With this macro you iterate over the exported symbols of a given package and then export them again on the package you want. Do this inside an eval-when form and when the library is loaded and the main package will contain all the symbols. If *library-sub-packages* is a list with the packages labels that compose your library:

(eval-when (:compile-toplevel :compile-toplevel :execute)
  (dolist (package *library-sub-packages*)
    (do-external-symbols (symbol (find-package package))
      (export symbol (find-package *library-main-package*)))))

Making all the exported symbols of internal packages also exportable by the main package turn out to be an easy thing to do. I don’t recall seing do-external-symbols (or the related macros) but I’m glad such a macro is provided. As always, the HyperSpec is your friend :-)

4 thoughts on “Packages organization and exporting symbols

  1. I find I generally dislike establishing package structure this way. I’d rather see explicit import & export lists in a defpackage form, in a single place. If a user wants to find out what the exports are to e.g. figure out if there might be a conflict, they have to load your code first, or go through N different defpackage forms. This might be easy or hard depending on how scattered the defpackage forms are.

    Creating the export lists and maintaining them can be a hassle, but I prefer to undertake that hassle for the benefit of users (myself included).

    1. Hi Zach,

      That’s a very good point! In fact, my first solution was copying all the symbols to the export list of the main defpackage. However, I thought it could lead me to some mistakes and the hassle. That’s why I thought the described approach was cool. But I see your point and to look at a single file with all the export symbols is really a big plus. As a middle ground, perhaps putting all the defpackage forms in a single file and do the “automatic” export at the end of it and documenting it can be an alternative. That way the exports lists are all together (in spite of not being a single one).

      I need to think better about this… :-)

      1. In fact, what you can do is keep your automatic find-and-re-export process, but use it to actually generate the main package file (or part of it). You can then run your script on a regular basis (or say, after each commit to your repo) to maintain an up-to-date version.

        I think this is a win-win situation because you keep the automation and yet, you also have the explicit file that your users can browse.

      2. Hi Didier,

        I like your suggestion! I will implement it and give it a try. Thanks! :-)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: