memory regions
1.1.0Implementation of a memory region abstraction
Table of Contents
About memory-regions
This library implements the concept of a "memory region" along with several useful tools to deal with them. A memory region is simply a pointer with a size, to provider safer access to foreign memory.
How To
Load the memory-regions system, or one of its subsystems if you only need select parts of the library.
Specifically:
memory-regions/region
Provides the base memory region type and protocol.memory-regions/allocator
Provides the allocator protocol and some base implementations.memory-regions/sequence
Provides the ability to coerce a memory region to a sequence.memory-regions/static-vector
Provides a memory region type with static-vector backing storage.memory-regions/stream
Provides the ability to coerce a memory region to a stream.memory-regions/object
Provides the ability to coerce a memory region to an object.memory-regions/pathname
Provides the ability to turn files into memory regions.memory-regions
All of the above.
Memory regions can be created in a couple of ways:
memory-region
Create a memory region directly from a raw pointer and size.
The region will not know anything about memory ownership and it is up to you to ensure that it is not dereferenced after the memory has been freed.to-memory-region
Coerce another object to a memory region. This will only work for objects that are backed by a memory region themselves, or otherwise have memory that remains static and does not move.with-memory-region
Coerce another object to a memory region. This will notably work for more objects thanto-memory-region
, such as standard vectors and arrays, as those can be pinned in place for the duration of the body. This also means that it is a very bad idea to use the provided memory region outside of the body or let it escape in some other way.static-vector-memory-region
Creates a new memory region with a Lisp static-vector as its storage. This allows you to address the region's memory directly as a lisp vector.allocate
Allocate a memory region within the context of anallocator
. Seewith-arena
as well for lexically scoped regions. Using theNIL
allocator is equivalent to manually allocating memory and constructing a region with it.
In the last case you can also make use of several other functions to manipulate memory regions:
deallocate
Free the represented memory again. This invalidates the memory region.reallocate
Change the size of the memory region. This may move the memory region's pointer, though, so beware of using this if you passed the raw pointer elsewhere.
You can also use memory regions as a convenient way to manipulate the memory they back:
clear
Clears the backed memory out with zeroes.fill
Fills the backed memory with the provided byte.replace
Replaces the backed memory contents with data from the provided source. This is notably nice because both the destination and source can be things thatwith-memory-region
can coerce.
Finally, with the above extension systems you can also use memory regions in conjunction with other libraries that usually can't directly interface with foreign memory:
to-stream
Creates a binary-stream that is backed by the memory region. You can transparently write to and read from this, making it especially useful for getting file format libraries to parse from memory directly.to-sequence
Creates a sequence that is backed by the memory region. The nice thing about this wrapper is that it can wrap even complex CFFI types, providing a more lisp-native access to foreign memory.to-object
Creates an object that is backed by the memory region. This allows you to access a foreign structure's slots viaslot-value
.
System Information
Definition Index
-
ORG.SHIRAKUMO.MEMORY-REGIONS
No documentation provided.-
EXTERNAL SPECIAL-VARIABLE *ALLOCATOR*
Holds the default allocator used for allocation operations. See WITH-ARENA See ALLOCATOR (type)
-
EXTERNAL SPECIAL-VARIABLE *STACK-ALLOCATION-SIZE-LIMIT*
No documentation provided. -
EXTERNAL SPECIAL-VARIABLE *STANDARD-ALLOCATOR*
Holds the class name of the default allocator used for new arenas. See WITH-ARENA See ALLOCATOR (type)
-
EXTERNAL CLASS ALLOCATOR
Representation of an allocation manager that can hand out regions. If you plan on implementing your own allocator, you must implement the related functions. You may also use NIL with any of the allocation functions, in which case the C runtime allocator (malloc, etc) is used. When you use T with any of the allocation functions, the current value of the *ALLOCATOR* variable is used. In order to free the allocator itself you should pass it to DEALLOCATE with NIL as the region. See ALLOCATE See DEALLOCATE See REALLOCATE See TOTAL-SIZE See FREE-SPACE See USED-SPACE See MINIMUM-BLOCK-SIZE See MAXIMUM-BLOCK-SIZE See *ALLOCATOR* See WITH-ARENA See CLEAR
-
EXTERNAL CLASS BUMP-ALLOCATOR
Implementation of a simple bump allocator. Allocations using this will be as fast as possible. Deallocations are only regarded if the region is the last region that was allocated. The idea being that you typically don't deallocate individual regions, and instead CLEAR or DEALLOCATE the allocator itself when you're done. See ALLOCATOR (type)
-
EXTERNAL CLASS MEMORY-REGION-ISH
Superclass for all objects that can be coerced to a memory region. See TO-MEMORY-REGION See CALL-WITH-MEMORY-REGION See MEMORY-REGION (type) See START See END See SIZE See CLEAR See FILL See REPLACE See SUBREGION
-
EXTERNAL CLASS MEMORY-REGION-OBJECT
No documentation provided. -
EXTERNAL CLASS MEMORY-REGION-SEQUENCE
No documentation provided. -
EXTERNAL CLASS MEMORY-REGION-STREAM
No documentation provided. -
EXTERNAL CONDITION ALLOCATOR-CONDITION
Supertype for conditions related to allocators. See ALLOCATOR
-
EXTERNAL CONDITION BLOCK-TOO-BIG
Error signalled when a block that is too big for the allocator is requested. See SIZE See ALLOCATOR-CONDITION (type) See ALLOCATOR (type)
-
EXTERNAL CONDITION OUT-OF-MEMORY
Error signalled when an allocation request cannot be fulfilled. See ALLOCATOR-CONDITION (type) See ALLOCATOR (type)
-
EXTERNAL STRUCTURE MEMORY-REGION
Representation of a region of memory. This is a pointer and a size. See MEMORY-REGION-POINTER See MEMORY-REGION-SIZE See MEMORY-REGION-VALID-P See MEMORY-REGION-ISH (type) See TO-MEMORY-REGION See CALL-WITH-MEMORY-REGION See START See END See SIZE See CLEAR See FILL See REPLACE See SUBREGION
-
EXTERNAL STRUCTURE STATIC-VECTOR-MEMORY-REGION
Representation of a memory region using a static-vector backing. These regions will survive an image dump and their underlying memory is accessible as a Lisp vector as well. Note that while they will survive dumping, the address of the region may differ after dump. Calling DEALLOCATE with NIL as the allocator will deallocate the underlying static-vector as well. See STATIC-VECTOR-MEMORY-REGION See MEMORY-REGION-VECTOR
-
EXTERNAL FUNCTION MEMORY-REGION
- POINTER
- SIZE
- &REST
No documentation provided. -
EXTERNAL FUNCTION MEMORY-REGION-POINTER
- INSTANCE
- &REST
Returns the pointer to the memory region. See MEMORY-REGION (type)
-
EXTERNAL FUNCTION (SETF MEMORY-REGION-POINTER)
- VALUE
- INSTANCE
- &REST
No documentation provided. -
EXTERNAL FUNCTION MEMORY-REGION-SIZE
- INSTANCE
- &REST
Returns the number of octets the memory region encompasses. See MEMORY-REGION (type)
-
EXTERNAL FUNCTION (SETF MEMORY-REGION-SIZE)
- VALUE
- INSTANCE
- &REST
No documentation provided. -
EXTERNAL FUNCTION MEMORY-REGION-VALID-P
- REGION
- &REST
Returns true if the memory region is valid and may be accessed. See MEMORY-REGION (type)
-
EXTERNAL FUNCTION MEMORY-REGION-VECTOR
- REGION
- &REST
Returns the underlying static-vector backing the memory-region. See STATIC-VECTOR-MEMORY-REGION (type)
-
EXTERNAL FUNCTION (SETF MEMORY-REGION-VECTOR)
- VECTOR
- REGION
- &REST
No documentation provided. -
EXTERNAL FUNCTION STATIC-VECTOR-MEMORY-REGION
- LENGTH/VEC
- &REST
Creates a new memory region with static-vector backing. The argument may either be the length (in octets) of the storage, or a static-vector. Note that this call cannot check whether the vector you passed is actually a static-vector as allocated by STATIC-VECTORS:MAKE-STATIC-VECTOR and things will break badly if you pass anything else. See STATIC-VECTOR-MEMORY-REGION (type) See STATIC-VECTORS:MAKE-STATIC-VECTOR
-
EXTERNAL GENERIC-FUNCTION ALLOCATE
- ALLOCATOR
- SIZE
- &REST
Allocate a new memory-region. The allocator may provide a region that is bigger than the requested size, but not smaller. If the allocator would have to shrink the region to provide the block of memory, an error of type BLOCK-TOO-BIG must be signalled, instead. If the allocator cannot allocate the requested amount of memory at all, an error of type OUT-OF-MEMORY is signalled, instead. The data pointed to by the returned region is *NOT* guaranteed to have been cleared out and may contain random non-zero data. SIZE must be a positive fixnum. See MEMORY-REGION (type) See ALLOCATOR (type) See BLOCK-TOO-BIG (type) See OUT-OF-MEMORY (type)
-
EXTERNAL GENERIC-FUNCTION ALLOCATOR
- CONDITION
- &REST
Returns the allocator related to the condition. See ALLOCATOR-ERROR
-
EXTERNAL GENERIC-FUNCTION CALL-WITH-MEMORY-REGION
- FUNCTION
- DATA
- &KEY
- START
- OFFSET
- DIRECTION
- SIZE
- &ALLOW-OTHER-KEYS
- &REST
Calls the function with the argument coerced to a memory region. The passed memory region is only valid within the dynamic extent of the function call, and may not be accessed outside of it. You may pass OFFSET to provide a starting offset into the resulting memory region, as well as potentially other arguments depending on the source. If a CFFI:FOREIGN-POINTER is passed as the source, you must also pass a SIZE argument to provide the memory region's extents. See MEMORY-REGION (type) See TO-MEMORY-REGION See WITH-MEMORY-REGION
-
EXTERNAL GENERIC-FUNCTION CLEAR
- REGION
- &REST
Clears the memory region and fills it with zeroes. See MEMORY-REGION (type) See MEMORY-REGION-ISH (type)
-
EXTERNAL GENERIC-FUNCTION DEALLOCATE
- ALLOCATOR
- REGION
- &REST
Free a previously allocated memory-region. The passed memory region must have been provided by a call to ALLOCATE of the same allocator. Passing unrelated region objects will lead to undefined behaviour. The returned memory region will no longer be valid. Passing in an invalid region has no effect. If you pass NIL as the memory region, the allocator itself will be deallocated, after which it must not be used again. See MEMORY-REGION (type) See ALLOCATOR (type) See MEMORY-REGION-VALID-P See ALLOCATE
-
EXTERNAL GENERIC-FUNCTION END
- REGION
- &REST
Returns a pointer to the end of the memory region. See MEMORY-REGION (type) See MEMORY-REGION-ISH (type)
-
EXTERNAL GENERIC-FUNCTION FILL
- DST
- BYTE
- &KEY
- START
- END
- &REST
Fills the memory region with a specific octet. See MEMORY-REGION (type) See MEMORY-REGION-ISH (type)
-
EXTERNAL GENERIC-FUNCTION FREE-SPACE
- ALLOCATOR
- &REST
Returns the remaining number of octets the allocator can distribute to memory-regions. See ALLOCATOR (type)
-
EXTERNAL GENERIC-FUNCTION MAXIMUM-BLOCK-SIZE
- ALLOCATOR
- &REST
Returns the maximal size a memory-region can have under this allocator. See ALLOCATOR (type)
-
EXTERNAL GENERIC-FUNCTION MINIMUM-BLOCK-SIZE
- ALLOCATOR
- &REST
Returns the minimal size of an allocated memory-region. See ALLOCATOR (type)
-
EXTERNAL GENERIC-FUNCTION REALLOCATE
- ALLOCATOR
- REGION
- NEW-SIZE
- &REST
Resizes a previously allocated memory-region. The passed memory region must have been provided by a call to ALLOCATE of the same allocator. Passing unrelated region objects will lead to undefined behaviour. The memory region's pointer and size are modified in-place, if the reallocation is successful, and the data pointed to will be preserved. If the region's size is increased, octets beyond the previous size are *NOT* guaranteed to have been cleared and may contain random non-zero data. The same constraints and error behaviour as for ALLOCATE apply. See MEMORY-REGION (type) See ALLOCATOR (type) See ALLOCATE
-
EXTERNAL GENERIC-FUNCTION REPLACE
- DST
- SRC
- &KEY
- START1
- END1
- START2
- END2
- &REST
Replaces the memory region's contents with that of another. See MEMORY-REGION (type) See MEMORY-REGION-ISH (type)
-
EXTERNAL GENERIC-FUNCTION SIZE
- REGION
- &REST
Returns the number of octets represented by the memory region. See MEMORY-REGION (type) See MEMORY-REGION-ISH (type)
-
EXTERNAL GENERIC-FUNCTION (SETF SIZE)
- NEW-VALUE
- OBJECT
- &REST
No documentation provided. -
EXTERNAL GENERIC-FUNCTION START
- REGION
- &REST
Returns a pointer to the start of the memory region. See MEMORY-REGION (type) See MEMORY-REGION-ISH (type)
-
EXTERNAL GENERIC-FUNCTION (SETF START)
- NEW-VALUE
- OBJECT
- &REST
No documentation provided. -
EXTERNAL GENERIC-FUNCTION SUBREGION
- REGION
- &OPTIONAL
- START
- END
- &REST
Returns a sub-region of the given memory-region. The following restrictions apply: - START and END must both be positive fixnums - START must be less than or equal to END - END must be less than the memory region's SIZE See MEMORY-REGION (type) See MEMORY-REGION-ISH (type)
-
EXTERNAL GENERIC-FUNCTION TO-MEMORY-REGION
- THING
- &REST
Coerces the given argument to a memory region. See MEMORY-REGION (type) See CALL-WITH-MEMORY-REGION See WITH-MEMORY-REGION
-
EXTERNAL GENERIC-FUNCTION TO-OBJECT
- OBJECT
- STRUCTURE-TYPE
- &REST
No documentation provided. -
EXTERNAL GENERIC-FUNCTION TO-SEQUENCE
- SEQUENCE
- &OPTIONAL
- ELEMENT-TYPE
- &REST
Coerces the given thing to a sequence. See MEMORY-REGION-SEQUENCE (type)
-
EXTERNAL GENERIC-FUNCTION TO-STREAM
- STREAM
- &REST
Coerces the given thing to a stream. See MEMORY-REGION-SEQUENCE (type)
-
EXTERNAL GENERIC-FUNCTION TOTAL-SIZE
- ALLOCATOR
- &REST
Returns the total number of octets the allocator can distribute to memory-regions. See ALLOCATOR (type)
-
EXTERNAL GENERIC-FUNCTION USED-SPACE
- ALLOCATOR
- &REST
Returns the number of octets that are currently distributed to memory-regions. See ALLOCATOR (type)
-
EXTERNAL MACRO WITH-ARENA
- SIZE
- &OPTIONAL
- TYPE
- &REST
- ARGS
- &REST
- &BODY
- BODY
- &REST
Executes body with an arena allocator bound. Any memory-regions allocated by the allocator may only be referenced within the dynamic-extent of the BODY. Attempting to do so outside of that dynamic-extent leads to undefined behaviour. As much as possible the arena will be stack-allocated. See *STANDARD-ALLOCATOR* See *ALLOCATOR* See ALLOCATOR (type)
-
EXTERNAL MACRO WITH-MEMORY-REGION
- REGION
- DATA/SIZE
- &REST
- ARGS
- &REST
- &BODY
- BODY
- &REST
Convenience macro to dynamically create a memory region. If the source is a positive fixnum constant, the memory area is allocated directly on the stack. See CALL-WITH-MEMORY-REGION
-
EXTERNAL MACRO WITH-MEMORY-REGIONS
- REGIONS
- &BODY
- BODY
- &REST
Convenience macro to dynamically create multiple memory regions. The bindings are sequential as by LET*. See WITH-MEMORY-REGION
-
EXTERNAL MACRO WITH-POINTER-TO-ARRAY-DATA
- PTR
- DATA
- &KEY
- DIRECTION
- &REST
- &BODY
- BODY
- &REST
Provides a foreign pointer to the array contents. DIRECTION must be one of: :INPUT --- The data is only read :OUTPUT --- The data is only written :IO --- The data is both read and written As much as possible the provided pointer will literally point to the array's in-memory representation. When this is not possible, a foreign memory area is used instead, and the data is transferred as informed by the DIRECTION argument.
-