Clasp Manual
Table of Contents
- 1. Introduction
- 2. Starting and Stopping
- 3. Lisp
- 3.1. Evaluation and Compilation
- 3.2. Types and Classes
- 3.3. Data and Control Flow
- 3.4. Iteration
- 3.5. Objects
- 3.6. Structures
- 3.7. Conditions
- 3.8. Symbols
- 3.9. Packages
- 3.10. Numbers
- 3.11. Characters
- 3.12. Conses
- 3.13. Arrays
- 3.14. Strings
- 3.15. Sequences
- 3.16. Hash Tables
- 3.17. Filenames
- 3.18. Files
- 3.19. Streams
- 3.20. Printer
- 3.21. Reader
- 3.22. System Construction
- 3.23. Environment
- 4. C++ Interface (clbind)
- 5. Foreign Function Interface
- 6. REPL
- 7. Debug interface
- 8. Multiprocessing
- 9. Introspection
- 10. Sockets
- 11. Serve Event
- 12. Garbage Collection
- 13. POSIX
1. Introduction
Clasp is an implementation of Common Lisp primarily designed for compatibility with C++-language programs and libraries.
Note that Clasp is a work in progress, as is this manual.
1.1. Conformance
Clasp conforms with the requirements of ANSI INCITS 226-1994 (R2004) with some exceptions, listed below in the "Lisp" section. Any deviation from the standard not listed there is a bug, and should be reported (see "Contributing").
1.2. Distribution
At present, the developers do not make Clasp binaries available. Clasp is open source, and available on-line at https://github.com/clasp-developers/clasp.
1.3. Building
See Building.
1.4. Contributing
Check the community page for instructions on contributing to Clasp.
Discussions of Clasp development are mostly carried out on Freenode's #clasp channel.
1.5. Credits
Clasp is the project of Dr. Christian Schafmeister. Additional contributions have been made by Tarn W. Burton, Karsten Poeck, Alex Wood, and many other contributors as seen in the Git history.
Clasp's source code is derived substantially from that of Embeddable Common Lisp. Code from SBCL and SICL has been incorporated as well. Most notably, the compiler is SICL's Cleavir compiler with some minor customizations.
2. Starting and Stopping
The Clasp executable accepts many different command line arguments. You can get the most up to date summary by passing --help
. Here are some important ones:
--noinform
: Skip messages at startup.--noprint
: Start a read-eval loop instead of a read-eval-print loop, and don't prompt. This is intended for scripts (e.g. piping a list of forms to Clasp).--disable-debugger
: Set things up so that if the default debugger would be entered, Clasp quits with a backtrace and nonzero exit status instead.*debugger-hook*
andext:*invoke-debugger-hook*
(below) work as usual, i.e. this setting does not affect them.--non-interactive
: Rather than starting a REPL, quit. This is intended to be used with--eval
and--load
. Implies--disable-debugger
.--feature feature
: Intern "feature" as a keyword and push it to*features*
.--eval form
: Evaluate the given form.--eval
and--load
options are processed in order from left to right.--load filename
:cl:load
the given file. Intended for convenience, since--eval (load ...)
could involve annoying quotation issues.--norc
: Do not load~/.clasprc
(see below).
If the file ~/.clasprc
exists, Clasp will cl:load
it before processing --eval
and --load
options and starting the REPL.
To exit Clasp, the ext:quit
function can be used. If provided an integer argument, it's used as the exit status; the default is zero.
3. Lisp
Clasp is an implementation of ANSI Common Lisp. The following subsections detail intentional deviations from the standard, some extensions to standard functionality, and notes on optimizing Lisp programs to run well within Clasp. (Extensions not closely related to standard systems are explained in their own sections below.)
3.1. Evaluation and Compilation
Clasp expands compiler macros essentially unconditionally.
That is, provided the operator is not declared notinline
,
the compiler will attempt to expand compiler macros.
If compiler macroexpansion signals, this signal will be recorded and reported by the compiler.
If compiler macroexpansion errors out, the compiler will abandon expansion and use the unexpanded form,
while noting the error for display.
ext:with-current-source-form
is a useful macro to allow error messages from macroexpanders and compiler-macro-expanders to have more specific source information. See its docstring for more information.
3.2. Types and Classes
typep
calls can be efficiently compiled only if the second argument, the type specifier, is constant.
Clasp's compiler will process the type specifier and generate code to check for objects of that type
directly, avoiding the usual runtime cost of interpreting the type.
Types defined by deftype
use a "type expander" function analogous to a macro expander function.
A type expander is a function of two arguments, a type specifier and an environment. When called on an appropriate specifier and environment, it computes and returns another type specifier. Type expanders are accessible through the ext:type-expander
accessor.
3.2.1. Disjointness
Unless otherwise specified, types Clasp defines as extensions can be considered to be in a disjointness
relationship with standard and other extension types, as in CLHS 4.2.2 "Type Relationships".
That is, if Clasp defines a type foo
, you can assume that foo
is not a subtype of hash-table
,
or cons
, or so on, and vice versa, unless it is explicitly stated to be.
But just as in 4.2.2, Clasp extension types may be subtypes of structure-object
or standard-object
without this being explicitly noted here.
3.3. Data and Control Flow
defsetf
, define-setf-expander
etc. define a "setf expander" function analogous to a macro expander
function. A setf expander is a function of two arguments, a place and an environment.
When called an appropriate place and environment, the expander computes and returns the values used by
setf
. Setf expanders are accessible through the ext:setf-expander
accessor.
3.4. Iteration
loop
supports iteration over general sequences (see below) through a for-as-sequence subclause.
This is identical to the subclause in SBCL.
The syntax is being {each | the} {element | elements} {of | in}
.
For example, (loop for x being each element in '(1 2 3) do (print x))
.
3.5. Objects
CLOS, as part of Common Lisp, is fully supported.
3.5.1. Metaobject Protocol
The Metaobject Protocol, as described in AMOP, is supported. Undocumented deviations from AMOP are bugs and should be reported, as with the CL standard.
Symbols relating to MOP are exported from the "CLOS" package.
3.5.2. Generic function dispatch efficiency
Clasp uses a new system for generic function dispatch designed by Dr. Robert Strandh. [Paper reference goes here.] Essentially, after a few calls to a generic function, a just-in-time compiler will install a discriminating function for it that can pass control to the correct effective method very efficiently. This means that calls with arguments that all have the same specializers as those of a previous call will in general be more efficient.
For some applications, the specializers a function will be called with are known beforehand,
and the runtime overhead of the just-in-time compilation would be unfortunate.
Clasp defines an interface to take care of most of the compilation early: The clos:satiate
function.
See its docstring for more info.
3.5.3. Miscellany
A consequence of the dispatch method described above is that obsolete instances are updated as soon as they are used as an argument to any generic function call - not just to slot accessors. This is allowed by the standard, but may surprise some programmers.
3.6. Structures
3.7. Conditions
In addition to cl:restart-name
, Clasp provides some readers to introspect about restarts,
for advanced users (e.g. writing your own debugger, or the Dissect library):
ext:restart-function
returns the function called by cl:invoke-restart
, and ext:report-function
,
ext:interactive-function
, and ext:interactive-function
return the corresponding arguments in
cl:restart-bind
. These will always be appropriate functions, so for example ext:report-function
will
always return a function of one stream argument, but if no :report-function
was provided it will report
the restart in Clasp's default way. The identities of these returned functions cannot be relied on,
i.e. they may not be identical to those provided to cl:restart-bind
.
There is also ext:restart-associated-conditions
, which returns a list of conditions associated
(by cl:with-condition-restarts
) with the restart in the current dynamic environment.
3.8. Symbols
3.9. Packages
Clasp supports package-local nicknames, through an interface based on that of SBCL's.
A package-local nickname is a nickname for a package that is only active when some other package is in
place as *package*
. For example, if the package "FOO" has "B" as a package-local nickname for package
"BAR", then while *package*
is the foo
package, the prefix "B:" will be read as if it was "BAR:".
Local nicknames may be specified in defpackage
through the (:local-nicknames (nickname package-name)*)
extended options. nickname
must be a string designator and package-name
a package designator - both
are unevaluated. The functions ext:package-local-nicknames
, ext:add-package-local-nickname
,
ext:remove-package-local-nickname
, and ext:package-locally-nicknamed-by-list
can be used for a more
programmatic interface.
3.10. Numbers
There are two types of floats, single-float
and double-float
. short-float
is synonymous with the
former and long-float
is synonymous with the latter, per the standard's requirements.
single-float
is in the IEEE754 binary32 (single) format, and double-float
in binary64 (double) format.
The representation of a float as bits can be interconverted with a float using the functions
ext:single-float-to-bits
, bits-to-single-float
, double-float-to-bits
, and bits-to-double-float
.
These functions take or return nonnegative integers;
for example (logbitp 31 (ext:single-floats-to-bit float))
returns whether the sign bit is set.
3.11. Characters
Clasp supports Unicode by default. code-char
and char-code
work with Unicode codepoints.
Unicode character names are also supported, e.g. (princ #\GREEK_SMALL_LETTER_LAMDA) -> λ
.
=(defun λ(n)(* 2 n)) (λ 32) -> 64=is also possible.
Type character
includes all characters in Unicode. Type base-char
includes only single byte
characters, i.e. Basic Latin and Latin-1 Supplement.
3.12. Conses
3.13. Arrays
In Clasp, arrays with no fill-pointer, displacement, or express adjustability are simple
(as in simple-array
), and arrays that have any of these are not.
Additionally, Clasp implements multidimensional arrays - even ones that are simple in this sense - as
if they were displaced to an underlying one dimensional array.
As such, it is most efficient to work with one-dimensional simple arrays directly.
3.14. Strings
3.15. Sequences
3.15.1. Extensible Sequences
The extensible sequences protocol developed by Christophe Rhodes is supported. Symbols related to the protocol are external in Clasp's "SEQUENCE" package. This protocol allows programmers to define their own sequence classes that work efficiently with standard Common Lisp functions. It is recommended that programmers consult other resources, such as Dr. Rhodes' paper, for more information on how to use this protocol effectively.
To summarize: Programmers wishing to make a custom sequence class must ensure their class has
cl:sequence
as a superclass. (Note that sequence
is itself abstract, so if a custom class needs to
have e.g. slots, it should also be a subclass of standard-object
or something like it.)
Methods on elt
, (setf elt)
, length
applicable to objects of the class must be defined for any
sequence functions to work; an applicable method on make-sequence-like
must be defined for creation
of this sequence to work; and an applicable method on adjust-sequence
must be defined for destructive
operations to work. Standard sequence functions will then operate correctly with these sequences,
as will make-sequence
and coerce
.
For efficiency, programmers may also define applicable methods on make-sequence-iterator
,
or less efficiently but more simply, on make-simple-sequence-iterator
, iterator-step
, iterator-endp
,
iterator-element
, (setf iterator-element)
, iterator-index
, and iterator-copy
.
Note that because the sequence:
generic function cognates to cl:
sequence functions are defined to
have the same behavior in almost all cases, Clasp takes the view that they need not be called.
For example, a call to cl:find
with a custom sequence object may result in a call to sequence:find
,
but may not. In other words the cognates are considered optional, and only possibly useful for
optimization. This is still in flux. If you think it's a bad idea, contact a maintainer to talk.
As a small extension to the extension, if a custom sequence object does not implement enough of the
protocol for a sequence function to complete, it will signal an error of type
sequence:protocol-unimplemented
. The reader sequence:protocol-unimplemented-operation
can be used to
get the name of the operation that failed from these conditions.
3.16. Hash Tables
make-hash-table
supports additional keyword arguments.
:weakness
can be used to indicate that the garbage collection may collect individual hash table entries
even when the hash table itself is live, in certain circumstances. At present, only weak-key hash tables
are supported: when the weakness argument is :key
, the hash table's reference to the key of a table
entry is weak, and if there are no non-weak references to a key, it is collectable.
See the "Garbage Collection" section below for more information on weak references.
If the weakness parameter is passed as nil
, or not passed, the hash table does not contain weak references.
:thread-safe
can be used to make hash table access safe across multiple threads.
If a thread-safe argument is not passed, or nil
is passed, the hash table cannot safely be written to
or read from multiple threads simultaneously (see "Memory Model", below, for a brief explanation of
terminology). If the thread-safe argument is true, the implementation will ensure that accesses can be
carried out from multiple threads simultaneously safely. This does impose a small performance penalty,
which is why it is not the default.
If a :test
other than a standard equality predicate is passed, :hash-function
must be specified as
well. The hash function should be a designator for a function of one argument that is analogous to
sxhash
, i.e. (funcall test x y)
implies (
(funcall hash-function x) (funcall hash-function y))=
and so on. This will create a "custom" hash table that can be used with the standard hash table functions
like gethash
, with the exception that at the moment, attempting to dump a custom hash table has
undefined consequences.
3.17. Filenames
3.18. Files
ext:rmdir
deletes a directory. EXT:RMTREE
deletes an entire directory tree.
3.19. Streams
3.19.1. Gray streams
The Gray stream interface as described in ANSI committee issue "STREAM-DEFINITION-BY-USER" (readable, e.g., on Kent Pitman's website) is supported. Symbols are exported from package "GRAY". We recommend programmers use a multi-implementation compatibility layer such as trivial-gray-streams rather than use Clasp's implementation directly.
Gray streams allow programmers to define their own stream classes with custom behavior that work with standard Common Lisp functions. It is recommended that programmers consult another resource, such as the trivial-gray-streams documentation, for more information on how to use this interface effectively.
3.20. Printer
When format
's control string argument is constant, the compiler will process it early, so that the
runtime doesn't have to. This improves runtime speed but increases code size.
3.21. Reader
3.22. System Construction
3.23. Environment
3.23.1. Stepper
cl:step
can be used to step through compiled code as partially described in the standard.
Currently, the stepper can only stop on forms that happen to be compiled as calls, though this may be
improved in the future.
Code is steppable if it is compiled with debug 3
optimization settings, or is within the step
macro.
While unsteppable code is being executed, the stepper will not stop.
Clasp further defines some aspects of stepping for the sake of editor/debugger integration.
When the stepper pauses execution, a condition of type cl:step
is passed to the debugger.
This condition will print with the source form for the call.
Similarly to cl:break
, *debugger-hook*
is bound to nil
, but ext:*invoke-debugger-hook*
(described below) can still be used for interception. The following restarts are available from a step
condition:
cl:continue
: Continue without stepping. The stepper will not invoke the debugger again.clasp-debug:step-into
: Continue stepping. If the function to be called is steppable, the stepper will pause within it, and otherwise stepping will proceed after the call.clasp-debug:step-over
: Continue stepping, but not into the call the stepper is on. Stepping will proceed after the call is exited (either by normal return or a non-local exit).
4. C++ Interface (clbind)
5. Foreign Function Interface
Clasp can interact with C programs and libraries through its Foreign Function Interface (FFI). Symbols relating to this interface are external in package "CLASP-FFI". However, it is recommended for most applications that you use a cross-implementation wrapper layer, specifically CFFI. Clasp's own interface has not yet been documented.
6. REPL
Clasp's built in read-eval-print loop supports various commands in addition to evaluating Lisp forms.
These commands consist of lines beginning with a Lisp keyword, followed possibly by additional arguments.
The most up to date documentation for this interface is the on-line help system,
obtainable with the command :help
.
Clasp has a built in debugger, which will be entered by invoke-debugger
by default.
:help
can describe the debugger commands as well. Some basic commands are :b
to print a backtrace,
:rN
to invoke the Nth restart, :v
to print local variables in the frame, and :up
, :down
,
and :go
for navigating frames.
In addition to the standard *debugger-hook*
, Clasp has ext:*invoke-debugger-hook*
.
This is a similar hook function, but it will be tried before *debugger-hook*
, and importantly,
will be called even for break
(which binds *debugger-hook*
to nil
per the standard).
This can be used to set up your own debugger in an IDE.
In the debugger, the function ext:tpl-frame
can be used to return a representation of the current frame
suitable for the programmatic debug interface described below, and ext:tpl-argument
and
ext:tpl-arguments
can be used to retrieve arguments.
In some applications, it's useful for the program to exit rather than exit a debugger.
The functions ext:disable-debugger
and ext:enable-debugger
can be used to set whether the debugger
will be entered. These only affect the built in debugger, and they do not affect *debugger-hook*
or ext:*invoke-debugger-hook*
.
7. Debug interface
For advanced users, such as those developing development tools such as debuggers to use with Clasp,
a programmatic interface to debug information is provided by the CLASP-DEBUG
package.
The with-stack
and call-with-stack
operators allow frame
objects, representing part of the current
control stack, to be interrogated. These frame objects have several readers: frame-function
,
frame-arguments
, frame-locals
, frame-source-position
, and frame-language
.
More specific information about the function can be obtained with frame-function-name
,
frame-function-lambda-list
, frame-function-source-position
, frame-function-form
,
frame-function-documentation
, and disassemble-frame
.
Note that frames necessarily have dynamic extent, because the local variables, arguments, and functions they refer to may be dynamic-extent themselves.
To navigate frames smoothly, a notion of "visibility" exists. Frames can be "invisible" if they aren't of
interest to users. This includes things like internal system code. Of course, the concept of visiblity
can change. Frame visibility is controlled by the *frame-filters*
variable, which holds a list of
function designators: a frame is visible if none of the functions return a true value when given the
frame as an argument. As such, all frames are considered visible if *frame-filters*
is bound to nil
.
up
and down
can be used to navigate visible frames, while frame-up
and frame-down
ignore
visibility. map-stack
, list-stack
, and map-indexed-stack
can be used to perform manipulations on
all frames at once.
with-truncated-stack
and with-capped-stack
can be used as hints to with-stack
(and therefore debuggers) that only a portion of the control stack is of interest. For example, a
function that signals an error can use with-truncated-stack
to ensure that lower debugger frames are
not included in backtraces.
print-backtrace
is provided as a simple way to print a current backtrace, without needing to use
with-stack
and the other more detailed operators.
8. Multiprocessing
Multiprocessing is supported. Symbols relating to multiprocessing are exported from the "MP" package. It is recommended that programmers use the Bordeaux-Threads compatibility library rather than Clasp's interface directly when possible.
8.1. Processes
A process is a Lisp object representing a distinct thread of execution. Each process evaluates a call to
a Lisp function, and exits when that call would finally return values. Processes have names for debugging
purposes. Processes are "nascent" or "not yet started" if they haven't yet begun evaluating, "active"
if they have begun evaluating, "suspended" if that evaluation has been paused by process-suspend
,
and "exited" if they have finished evaluation (normally or by aborting).
Processes have type process
. make-process
creates a new process but does not start it.
process-start
enables a process, and process-run-function
both creates and enables a process.
The name of a process can be retrieved with process-name
. process-active-p
can be used to query
whether a process is active. process-suspend
, process-resume
, interrupt-process
, and
process-kill
interfere with a process's evaluation. process-join
waits until a process until it
completes, and then returns the values its function returned, or signals an error of
type process-join-error
if the process ended abnormally. Within a process, exit-process
can be used
to end the process's evaluation immediately, and abort-process
to do so abnormally; in either case the
dynamic environment is properly unwound. all-processes
gets a list of all enabled processes.
The variable *current-process*
is bound in any process to that process. Consult the docstrings of
these functions for more information.
8.2. Special variables
Bindings of special variables (by let
, progv
, lambda lists, etc.) are thread-local.
That is, executing a binding form for a variable will not affect that variable's value in other threads.
The global value - from symbol-value
- is, in contrast, shared between threads.
8.3. Mutexes
A mutex (short for "MUTual EXclusion"), or lock, can be used to control access to a shared resource by multiple processes.
Mutexes have type mutex
. A mutex is created with make-lock
, or make-recursive-mutex
for a recursive
mutex. get-lock
and giveup-lock
obtain and release exclusion on a mutex, respectively.
mutex-name
retrieves any name of a mutex given at creation.
8.4. Shared Mutexes
Not yet documented.
8.5. Condition Variables
Not yet documented.
8.6. Memory Model
Clasp does not have a formal memory model. Here is a sketch of one: Two accesses of a place are concurrent if they take place in different threads and are not excluded from running simultaneously by locks. Two concurrent accesses conflict if at least one is a write. If a conflicting access is not atomic the program has undefined behavior (e.g. tearing).
Some accesses are atomic but unordered, meaning that there is not necessarily a modification order to the place that is observed by all threads, e.g. one thread may see writes occur in a different order from another thread. Some accesses are sequentially consistent, meaning that there is such a globally observable modification order, and furthermore that all sequentially consistent accesses have a globally observable order.
Places may be complex, indeed completely custom. Clasp defines atomicity of some simple places;
other places, and more complex modification operations, should hopefully be understandable from those.
For example, to setf the second
of a list, one cdr
must be read before a car
is written, and each
of these individual accesses is atomic while the overall access is not.
In general Clasp tries to guarantee unordered atomicity, but does not always succeed, and in some cases it's probably not possible.
Note that in this context "atomic" does not necessarily mean "lock-free", although Clasp attempts to make atomic operations proceed without locks.
Places that can be accessed unorderedly are: car
, cdr
, symbol-value
, symbol-plist
,
symbol-function=/=fdefinition
, compiler-macro-function
, ext:setf-expander
, ext:type-expander
.
Access to the elements of simple one-dimensional arrays should be unordered, except for integer element
types smaller than (unsigned-byte 8)
. Access to standard-object
and structure-object
slots
(of :instance
or :class
allocation) should also be unordered.
8.7. Atomics
Access can be guaranteed atomic by using the atomic
macro. That is, (atomic place)
is a place that
can be accessed atomically, or else an error will be signaled. Clasp defines car
, cdr
, first
,
rest
, symbol-value
, special variables, symbol-plist
, standard-instance-access
, slot-value
,
clos:slot-value-using-class
, and svref
as atomically accessible, as well as the
place or macro
places that expand to these places. Additional atomically accessible places can be defined with the
define-atomic-expansion
macro. See its documentation string for more information. Additionally,
documentation on atomic access may be available with kind atomic
; e.g.
try (documentation 'symbol-value 'mp:atomic)
.
8.8. Compare-and-swap
The mp:cas
macro can be used to execute an atomic compare-and-swap of atomically accessible places.
See its docstring for more information. cas
is used to define higher order atomic read-modify-write
operations provided by Clasp: atomic-update
, atomic-incf
, atomic-decf
, atomic-push
, atomic-pop
,
and atomic-pushnew
. The first is a general operator analogous to what define-modify-macro
operators
do, while the others are analogous to their standard versions. -explicit
variants can be used to
explicitly specify the order of the operation.
8.9. Fences
The mp:fence
function can be used to establish memory fences of a specified order.
9. Introspection
Information about objects is stored and accessible. This is primarily intended for editor integration,
but the functions can be used in any context. It is not recommended that they be used for purposes other
than human understanding, however - it can sometimes be dropped or inaccurate. These mechanisms are
similar to the standard documentation
function.
ext:function-lambda-list
can be used to get the lambda list of a function object, and
ext:compiled-function-name
its name.
ext:source-location
returns a list of source locations for a symbol or object. Source locations are of
type ext:source-location
; they contain a pathname accessible with ext:source-location-pathname
, and a
file offset (as from file-position
) accessible with ext:source-location-offset
.
10. Sockets
A low level networking API based on SBCL's (which is in turn based on BSD sockets) is available in package "SB-BSD-SOCKETS". Clasp's interface has not yet been specifically documented.
11. Serve Event
Not yet documented.
12. Garbage Collection
Symbols related to garbage collection are exported from the "GCTOOLS" package.
The function garbage-collect
forces a garbage collection.
finalize
registers a finalizer function for an object. When the object is collected, the function will
be called with no arguments. Note that this function should not close over the object, because then the
closure will keep that object alive indefinitely.
13. POSIX
13.1. Signal Handling
Handlers for standard POSIX signals can be defined in Clasp using the ext:enable-interrupt
function,
which expects a keyword to identify the type of signal (e.g. :sigpipe
for SIGPIPE
).
If a Lisp function is used as the handler, it must be a function of one argument, the signal number.
ext:enable-interrupt
, or ext:ignore-interrupt
and ext:default-interrupt
, can be used to set the
handler to the ignore-signal handler or the default handler respectively, analogous to SIG_IGN
and
SIG_DFL
. The current handler function, if there is one, can be retrieved with ext:get-signal-handler
.
13.2. Further posix interfaces
ext:stat
and ext:fstat
wrap the corresponding posix-interfaces. Use ext:file-stream-file-descriptor
to get the file-descriptor for a stream.
The environment can be accessed with EXT:SETENV=and =EXT:GETENV
Working directories can be accessed with EXT:GETCWD=and =EXT:CHDIR
.