Simple-Inferiors

About Simple-Inferiors

This is a library to allow easy handling of external processes, and primarily to get their output. It handles proper copying of stdout and stderr of the process simultaneously, both in a sequential and parallel fashion. It also features a lazy directory switching mechanism, to avoid running into parallelism problems when having to chdir.

How To

Load simple-inferiors with ASDF or Quicklisp

(ql:quickload :simple-inferiors)

Run a program!

(simple-inferiors:run "bash" '("-c" "for i in `seq 1 10`; do echo $i; done"))

Not very exciting. By default the output is discarded and you only get the exit code. Let's see what it says:

(simple-inferiors:run "bash" '("-c" "for i in `seq 1 10`; do echo $i; done")
                      :output T)

By default the streams will be copied character by character. This allows the most immediate fetching of the output from the process, at the cost of being very CPU intensive. If you can afford it, you may want to switch to a more efficient method, such as copying line by line:

(simple-inferiors:run "bash" '("-c" "for i in `seq 1 10`; do echo $i; done")
                      :output T :copier :line)

The output actually doesn't change for this tiny test case, but the performance can be radically different for larger outputs. You may also pass a number to specify a custom buffer size, or a function to handle the stream copying yourself.

When the stack is unwound, simple-inferiors tries to terminate the external process. By default it will try to ask the process to terminate with 0.1 second delays and then it will try to kill it if it still hasn't terminated. In order to control this stopping, you must supply a different handler to run.

(simple-inferiors:run "bash" '("-c" "sleep 60") 
                      :handler (lambda (c p oi oo ei eo) 
                                 (simple-inferiors:handle-process-sequential c p oi oo ei eo :stop-attempts 100)))

You may also use handle-process-parallel if you would like to use threads to handle the stdout and stderr of the process instead of attempting to read both simultaneously sequentially, or provide your own handler function entirely.

If you need to handle different directories for your process, you can use with-chdir. Note that with-chdir does not actually perform a chdir and instead rebinds *cwd*. The chdir is only performed (if at all necessary) at the very last stage when a process is run. This avoids clashing if parallelism is involved. with-chdir merges the passed location (resolved by location) with the current *cwd*. If *cwd* is NIL (such as at the very beginning), the cwd is used to merge the path. If you need to absolutely definitely perform a chdir, you may use with-exchdir. Note that it will signal an invalid-location-error if changing directory to an inexistent location is attempted.

Copyright

simple-inferiors is licensed under the Artistic license. © Nicolas Hafner <shinmera@tymoon.eu> . This library can be obtained on https://github.com/Shinmera/simple-inferiors.

Package Index

SIMPLE-INFERIORS (ORG.SHIRAKUMO.SIMPLE-INFERIORS)

special

`CWD`

The variable containing the current directory, virtually.
This variable is only resolved once needed, such as when using WITH-EXCHDIR.

See WITH-CHDIR
See WITH-EXCHDIR

condition

`INFERIOR-PROCESS-FAILED-ERROR`

Error variant of INFERIOR-PROCESS-FAILED-CONDITION

See INFERIOR-PROCESS-FAILED-CONDITION

condition

`INFERIOR-PROCESS-FAILED-WARNING`

Warning variant of INFERIOR-PROCESS-FAILED-CONDITION

See INFERIOR-PROCESS-FAILED-CONDITION

condition

`INVALID-LOCATION-ERROR`

Signalled if an attempt is made to change directory to a location that does not exist.

See LOCATION

accessor (

`FAILED-ARGS`

CONDITION)

Accesses the program arguments passed to RUN that failed to execute properly.

See INFERIOR-PROCESS-FAILED-CONDITION

accessor (

`FAILED-EXIT`

CONDITION)

Accesses the exit code returned from the program that failed to execute properly in RUN.

See INFERIOR-PROCESS-FAILED-CONDITION

accessor (

`FAILED-PROGRAM`

CONDITION)

Accesses the program string passed to RUN that failed to execute properly.

See INFERIOR-PROCESS-FAILED-CONDITION

accessor (

`LOCATION`

THING)

Attempts to resolve the THING to a pathname.

THING can be one of
  NULL     => UIOP:GETCWD
  PATHNAME => THING
  STRING   => UIOP:PARSE-NATIVE-NAMESTRING

This generic function is intended to be extended with methods by the user to allow using objects as locations directly.

function (

`COPY-STREAM`

INPUT OUTPUT &KEY CONSUME-ALL (BUFFER 64))

Copies data from INPUT to OUTPUT using the given BUFFER format.
If CONSUME-ALL is non-NIL, all data is read from INPUT until EOF is reached.

BUFFER can be one of
  (EQL :LINE)      => The stream is copied one line at a time.
  (EQL :CAHRACTER) => The stream is copied one character at a time.
  INTEGER          => A character buffer of size BUFFER is used.

Note that this function tries to be as non-blocking as possible if CONSUME-ALL is NIL.
This means that it will only copy anything if there is something to read, but might
also read more than one line, character, or buffer at a time, if more data is available.

Once nothing more can be copied, FINISH-OUTPUT on OUTPUT is called.

function (

`ENSURE-COPIER`

COPIER-ISH)

Ensures that COPIER-ISH is an actual function usable for copying streams.

COPIER-ISH can be one of
  FUNCTION => The function is used directly.
  INTEGER  => MAKE-COPIER is called with the COPIER-ISH.
  KEYWORD  => MAKE-COPIER is called with the COPIER-ISH.
  SYMBOL   => The function associated with the symbol is used.

The function must accept an INPUT and OUTPUT stream, as well as in the very least a keyword argument called CONSUME-ALL that, when non-NIL, will copy the whole INPUT to OUTPUT in one go until EOF is reached.

See MAKE-COPIER

function (

`HANDLE-PROCESS-PARALLEL`

COPIER PROCESS OUT-IN OUT-OUT ERR-IN ERR-OUT &KEY (STOP-ATTEMPTS 10)
 (STOP-SLEEP 0.1)

)

Handles the PROCESS using COPIER with the OUT-IN, OUT-OUT, ERR-IN, and ERR-OUT streams in parallel. For that, it opens two threads for the respective stream pairs that handle the copying and joins them with the initial thread on unwinding.

As all handlers, this is responsible for copying the data from the IN to the respective OUT streams as well as ensuring that the process is stopped and all remaining data is read on unwinding.

function (

`HANDLE-PROCESS-SEQUENTIAL`

COPIER PROCESS OUT-IN OUT-OUT ERR-IN ERR-OUT &KEY (COOLDOWN 0.05)
 (STOP-ATTEMPTS 10) (STOP-SLEEP 0.1)

)

Handles the PROCESS using COPIER with the OUT-IN, OUT-OUT, ERR-IN, and ERR-OUT streams sequentially.
Between copies, it will sleep for COOLDOWN seconds to make sure no excessive CPU is wasted trying to read repeatedly.

As all handlers, this is responsible for copying the data from the IN to the respective OUT streams as well as ensuring that the process is stopped and all remaining data is read on unwinding.

function (

`MAKE-COPIER`

BUFFER)

Creates a copier function that accepts an input and output stream as well as optional extra arguments using BUFFER.
This simply creates a wrapper lambda around COPY-STREAM.

See COPY-STREAM

function (

`RUN`

PROGRAM ARGS &KEY INPUT OUTPUT ERROR (ON-NON-ZERO-EXIT :RETURN)
 (HANDLER (FUNCTION HANDLE-PROCESS-SEQUENTIAL)) (COPIER :CHARACTER)

)

Runs an inferior process, supplying PROGRAM with ARGS and using INPUT for STDIN, OUTPUT for STDOUT, and ERROR FOR STDERR.

The current *CWD* is resolved to an actual location, checked for validity, and then used as the location to start the process in. Depending on implementation support, this may have to fall back on using a manual chdir for launching the process.

HANDLER must be a function of six arguments:
  COPIER  => The function computed by ENSURE-COPIER on COPIER:
  PROCESS => The process object used by EXTERNAL-PROGRAM.
  OUT-IN  => The receiving STDOUT stream from the process.
  OUT-OUT => The outputting stream computed by WITH-RESOLVED-STREAM on OUTPUT.
  ERR-IN  => The receiving STDERR stream from the process.
  ERR-OUT => The outputting stream computed by WITH-RESOLVED-STREAM on ERROR.
The handler must ensure that the process is stopped and all data has been copied when an unwind takes place. Furthermore it should not return until the process is done.

ON-NON-ZERO-EXIT can be one of
  NIL     => NIL is returned.
  :RETURN => The exit code is returned.
  :ERROR  => A INFERIOR-PROCESS-FAILED-ERROR is signalled.
  :WARN   => A INFERIOR-PROCESS-FAILED-WARNING is signalled.

See *CWD*
See ENSURE-COPIER
See HANDLE-PROCESS-SEQUENTIAL
See HANDLE-PROCESS-PARALLEL

generic (

`VALID-LOCATION-P`

THING)

Checks whether THING is a valid (existing) location.

See LOCATION

macro (

`WITH-CHDIR`

(NEW-PATH) &BODY BODY)

Changes the directory lazily.
This merges the passed NEW-PATH (resolved through LOCATION) with *CWD* (resolved through LOCATION) and binds that to *CWD*.

See LOCATION
See *CWD*

macro (

`WITH-EXCHDIR`

(&OPTIONAL (NEW-PATH)) &BODY BODY)

Changes the directory directly.
If NEW-PATH is not passed, *CWD* is used instead. Either way it is resolved through LOCATION and checked by CHECK-LOCATION before the actual directory change is performed. This will /also/ act like WITH-CHDIR by additionally rebinding *CWD*.

Note that since a binary can only ever be in one directory at once, you should avoid using this unless necessary, or unless you are sure that the system is not paralellised.

See LOCATION
See CHECK-LOCATION
See *CWD*