ref?ref is a native addon for Node.js that aids in doing C programming in JavaScript, by extending
the built-in Buffer class with some fancy additions like:
There is indeed a lot of meat to ref, but it all fits together in one way or another in the end.
For simplicity, ref's API can be broken down into 3 sections:
exportsAll the static versions of ref's functions and default "types" available on the exports returned from require('ref').
The "type" system allows you to define a "type" on any Buffer instance, and then
use generic ref() and deref() functions to reference and dereference values.
Buffer extensionsBuffer.prototype gets extended with some convenience functions. These all just mirror
their static counterpart, using the Buffer's this variable as the buffer variable.
This section documents all the functions exported from require('ref').
A Buffer that references the C NULL pointer. That is, its memory address
points to 0. Its length is 0 because accessing any data from this buffer
would cause a segmentation fault.
console.log(ref.NULL);
<SlowBuffer@0x0 >
NULL_POINTER is a pointer-sized Buffer instance pointing to NULL.
Conceptually, it's equivalent to the following C code:
char *null_pointer;
null_pointer = NULL;
Accepts a Buffer instance and returns the memory address of the buffer
instance.
console.log(ref.address(new Buffer(1)));
4320233616
console.log(ref.address(ref.NULL)));
0
set() function.type set to "type", and (optionally) "value" written to it.Returns a new Buffer instance big enough to hold type,
with the given value written to it.
var intBuf = ref.alloc(ref.types.int)
var int_with_4 = ref.alloc(ref.types.int, 4)
Buffer instance with the specified String wrtten to it, and a trailing NUL byte.Returns a new Buffer instance with the given String written to it with the
given encoding (defaults to 'utf8'). The buffer is 1 byte longer than the
string itself, and is NUL terminated.
var buf = ref.allocCString('hello world');
console.log(buf.toString());
'hello world\u0000'
Coerces a "type" object from a String or an actual "type" object. String values
are looked up from the ref.types Object. So:
"int" gets coerced into ref.types.int."int *" gets translated into ref.refType(ref.types.int)ref.types.int gets translated into ref.types.int (returns itself)Throws an Error if no valid "type" object could be determined. Most ref
functions use this function under the hood, so anywhere a "type" object is
expected, a String may be passed as well, including simply setting the
buffer.type property.
var type = ref.coerceType('int **');
console.log(type.indirection);
3
Accepts a Buffer instance and attempts to "dereference" it.
That is, first it checks the indirection count of buffer's "type", and if
it's greater than 1 then it merely returns another Buffer, but with one
level less indirection.
When buffer's indirection is at 1, then it checks for buffer.type
which should be an Object with its own get() function.
var buf = ref.alloc('int', 6);
var val = ref.deref(buf);
console.log(val);
6
indirection decremented by 1.Returns a new clone of the given "type" object, with its
indirection level decremented by 1.
A string that represents the native endianness of the machine's processor.
The possible values are either "LE" or "BE".
console.log(ref.endianness);
'LE'
getType() on the buffer.Calls the get() function of the Buffer's current "type" (or the
passed in type if present) at the given offset.
This function handles checking the "indirection" level and returning a proper "dereferenced" Bufffer instance when necessary.
Returns the "type" property of the given Buffer. Creates a default type for the buffer when none exists.
Accepts a Buffer instance and returns true if the buffer represents the
NULL pointer, false otherwise.
console.log(ref.isNull(new Buffer(1)));
false
console.log(ref.isNull(ref.NULL));
true
Returns a JavaScript String read from buffer at the given offset. The C String is read until the first NULL byte, which indicates the end of the String.
This function can read beyond the length of a Buffer.
var buf = new Buffer('hello\0world\0');
var str = ref.readCString(buf, 0);
console.log(str);
'hello'
Returns a big-endian signed 64-bit int read from buffer at the given offset.
If the returned value will fit inside a JavaScript Number without losing precision, then a Number is returned, otherwise a String is returned.
var buf = ref.alloc('int64');
ref.writeInt64BE(buf, 0, '9223372036854775807');
var val = ref.readInt64BE(buf, 0)
console.log(val)
'9223372036854775807'
Returns a little-endian signed 64-bit int read from buffer at the given offset.
If the returned value will fit inside a JavaScript Number without losing precision, then a Number is returned, otherwise a String is returned.
var buf = ref.alloc('int64');
ref.writeInt64LE(buf, 0, '9223372036854775807');
var val = ref.readInt64LE(buf, 0)
console.log(val)
'9223372036854775807'
Reads a JavaScript Object that has previously been written to the given buffer at the given offset.
var obj = { foo: 'bar' };
var buf = ref.alloc('Object', obj);
var obj2 = ref.readObject(buf, 0);
console.log(obj === obj2);
true
Reads a Buffer instance from the given buffer at the given offset.
The size parameter specifies the length of the returned Buffer instance,
which defaults to 0.
var buf = new Buffer('hello world');
var pointer = ref.alloc('pointer');
var buf2 = ref.readPointer(pointer, 0, buf.length);
console.log(buf.toString());
'hello world'
Returns a big-endian unsigned 64-bit int read from buffer at the given offset.
If the returned value will fit inside a JavaScript Number without losing precision, then a Number is returned, otherwise a String is returned.
var buf = ref.alloc('uint64');
ref.writeUInt64BE(buf, 0, '18446744073709551615');
var val = ref.readUInt64BE(buf, 0)
console.log(val)
'18446744073709551615'
Returns a little-endian unsigned 64-bit int read from buffer at the given offset.
If the returned value will fit inside a JavaScript Number without losing precision, then a Number is returned, otherwise a String is returned.
var buf = ref.alloc('uint64');
ref.writeUInt64LE(buf, 0, '18446744073709551615');
var val = ref.readUInt64LE(buf, 0)
console.log(val)
'18446744073709551615'
ref() accepts a Buffer instance and returns a new Buffer
instance that is "pointer" sized and has its data pointing to the given
Buffer instance. Essentially the created Buffer is a "reference" to the
original pointer, equivalent to the following C code:
char *buf = buffer;
char **ref = &buf;
indirection incremented by 1.Returns a new clone of the given "type" object, with its
indirection level incremented by 1.
Say you wanted to create a type representing a void *:
var voidPtrType = ref.refType(ref.types.void);
length property of the returned Buffer.Returns a new Buffer instance with the specified size, with the same memory address as buffer.
This function "attaches" buffer to the returned Buffer to prevent it from being garbage collected.
NULL bytes are required to terminate the buffer.length that is terminated by size NUL bytes.Accepts a Buffer instance and a number of NULL bytes to read from the
pointer. This function will scan past the boundary of the Buffer's length
until it finds size number of aligned NULL bytes.
This is useful for finding the end of NUL-termintated array or C string. For
example, the readCString() function could be implemented like:
function readCString (buf) {
return ref.reinterpretUntilZeros(buf, 1).toString('utf8')
}
This function "attaches" buffer to the returned Buffer to prevent it from being garbage collected.
getType() on the buffer.Calls the set() function of the Buffer's current "type" (or the
passed in type if present) at the given offset.
This function handles checking the "indirection" level writing a pointer rather
than calling the set() function if the indirection is greater than 1.
Writes the given string as a C String (NULL terminated) to the given buffer at the given offset. "encoding" is optional and defaults to 'utf8'.
Unlike readCString(), this function requires the buffer to actually have the
proper length.
Writes the input Number or String as a big-endian signed 64-bit int into buffer at the given offset.
var buf = ref.alloc('int64');
ref.writeInt64BE(buf, 0, '9223372036854775807');
Writes the input Number or String as a little-endian signed 64-bit int into buffer at the given offset.
var buf = ref.alloc('int64');
ref.writeInt64LE(buf, 0, '9223372036854775807');
Writes a pointer to object into buffer at the specified _offset.
This function "attaches" object to buffer to prevent it from being garbage collected.
var buf = ref.alloc('Object');
ref.writeObject(buf, 0, { foo: 'bar' });
Writes the memory address of pointer to buffer at the specified offset.
This function "attaches" object to buffer to prevent it from being garbage collected.
var someBuffer = new Buffer('whatever');
var buf = ref.alloc('pointer');
ref.writePointer(buf, 0, someBuffer);
Writes the input Number or String as a big-endian unsigned 64-bit int into buffer at the given offset.
var buf = ref.alloc('uint64');
ref.writeUInt64BE(buf, 0, '18446744073709551615');
Writes the input Number or String as a little-endian unsigned 64-bit int into buffer at the given offset.
var buf = ref.alloc('uint64');
ref.writeUInt64LE(buf, 0, '18446744073709551615');
Attaches object to buffer such that it prevents object from being garbage collected until buffer does.
length property of the returned Buffer.Same as ref.reinterpret(), except that this version does not attach
buffer to the returned Buffer, which is potentially unsafe if the
garbage collector runs.
NULL bytes that are required to terminate the buffer.length that is terminated by size NUL bytes.Same as ref.reinterpretUntilZeros(), except that this version does not
attach buffer to the returned Buffer, which is potentially unsafe if the
garbage collector runs.
Same as ref.writeObject(), except that this version does not attach the
Object to the Buffer, which is potentially unsafe if the garbage collector
runs.
Same as ref.writePointer(), except that this version does not attach
pointer to buffer, which is potentially unsafe if the garbage collector
runs.
A "type" in ref is simply an plain 'ol JavaScript Object, with a set
of expected properties attached that implement the logic for getting
& setting values on a given Buffer instance.
To attach a "type" to a Buffer instance, you simply attach the "type"
object to the Buffer's type property. ref comes with a set of commonly used types which are described in this
section.
It's trivial to create your own "type" that reads and writes your
own custom datatype/class to and from Buffer instances using ref's unified API.
To create your own "type", simply create a JavaScript Object with
the following properties defined:
| Name | Data Type | Description |
|---|---|---|
size | Number | The size in bytes required to hold this datatype. |
indirection | Number | The current level of indirection of the buffer. When defining
your own "types", just set this value to 1. |
get | Function | The function to invoke when ref.get() is invoked on a buffer of this type. |
set | Function | The function to invoke when ref.set() is invoked on a buffer of this type. |
name | String | (Optional) The name to use during debugging for this datatype. |
alignment | Number | (Optional) The alignment of this datatype when placed inside a struct.
Defaults to the type's size. |
Here is the list of ref's built-in "type" Objects. All these built-in "types" can be found
on the ref.types export Object. All the built-in types use "native endianness" when
multi-byte datatypes are involved.
The CString (a.k.a "string") type.
CStrings are a kind of weird thing. We say it's sizeof(char *), and
indirection level of 1, which means that we have to return a Buffer that
is pointer sized, and points to a some utf8 string data, so we have to create
a 2nd "in-between" buffer.
The bool type.
Wrapper type around types.uint8 that accepts/returns true or
false Boolean JavaScript values.
Buffer.prototype gets extended with some convenience functions that you can use in
your modules and/or applications.
Shorthand for ref.address(this, …).
Accepts a Buffer instance and returns the memory address of the buffer
instance.
console.log(ref.address(new Buffer(1)));
4320233616
console.log(ref.address(ref.NULL)));
0
Shorthand for ref.deref(this, …).
Accepts a Buffer instance and attempts to "dereference" it.
That is, first it checks the indirection count of buffer's "type", and if
it's greater than 1 then it merely returns another Buffer, but with one
level less indirection.
When buffer's indirection is at 1, then it checks for buffer.type
which should be an Object with its own get() function.
var buf = ref.alloc('int', 6);
var val = ref.deref(buf);
console.log(val);
6
ref overwrites the default Buffer#inspect() function to include the
hex-encoded memory address of the Buffer instance when invoked.
This is simply a nice-to-have.
Before:
console.log(new Buffer('ref'));
<Buffer 72 65 66>
After:
console.log(new Buffer('ref'));
<Buffer@0x103015490 72 65 66>
Shorthand for ref.isNull(this, …).
Accepts a Buffer instance and returns true if the buffer represents the
NULL pointer, false otherwise.
console.log(ref.isNull(new Buffer(1)));
false
console.log(ref.isNull(ref.NULL));
true
Shorthand for ref.readCString(this, …).
Returns a JavaScript String read from buffer at the given offset. The C String is read until the first NULL byte, which indicates the end of the String.
This function can read beyond the length of a Buffer.
var buf = new Buffer('hello\0world\0');
var str = ref.readCString(buf, 0);
console.log(str);
'hello'
Shorthand for ref.readInt64BE(this, …).
Returns a big-endian signed 64-bit int read from buffer at the given offset.
If the returned value will fit inside a JavaScript Number without losing precision, then a Number is returned, otherwise a String is returned.
var buf = ref.alloc('int64');
ref.writeInt64BE(buf, 0, '9223372036854775807');
var val = ref.readInt64BE(buf, 0)
console.log(val)
'9223372036854775807'
Shorthand for ref.readInt64LE(this, …).
Returns a little-endian signed 64-bit int read from buffer at the given offset.
If the returned value will fit inside a JavaScript Number without losing precision, then a Number is returned, otherwise a String is returned.
var buf = ref.alloc('int64');
ref.writeInt64LE(buf, 0, '9223372036854775807');
var val = ref.readInt64LE(buf, 0)
console.log(val)
'9223372036854775807'
Shorthand for ref.readObject(this, …).
Reads a JavaScript Object that has previously been written to the given buffer at the given offset.
var obj = { foo: 'bar' };
var buf = ref.alloc('Object', obj);
var obj2 = ref.readObject(buf, 0);
console.log(obj === obj2);
true
Shorthand for ref.readPointer(this, …).
Reads a Buffer instance from the given buffer at the given offset.
The size parameter specifies the length of the returned Buffer instance,
which defaults to 0.
var buf = new Buffer('hello world');
var pointer = ref.alloc('pointer');
var buf2 = ref.readPointer(pointer, 0, buf.length);
console.log(buf.toString());
'hello world'
Shorthand for ref.readUInt64BE(this, …).
Returns a big-endian unsigned 64-bit int read from buffer at the given offset.
If the returned value will fit inside a JavaScript Number without losing precision, then a Number is returned, otherwise a String is returned.
var buf = ref.alloc('uint64');
ref.writeUInt64BE(buf, 0, '18446744073709551615');
var val = ref.readUInt64BE(buf, 0)
console.log(val)
'18446744073709551615'
Shorthand for ref.readUInt64LE(this, …).
Returns a little-endian unsigned 64-bit int read from buffer at the given offset.
If the returned value will fit inside a JavaScript Number without losing precision, then a Number is returned, otherwise a String is returned.
var buf = ref.alloc('uint64');
ref.writeUInt64LE(buf, 0, '18446744073709551615');
var val = ref.readUInt64LE(buf, 0)
console.log(val)
'18446744073709551615'
Shorthand for ref.ref(this, …).
ref() accepts a Buffer instance and returns a new Buffer
instance that is "pointer" sized and has its data pointing to the given
Buffer instance. Essentially the created Buffer is a "reference" to the
original pointer, equivalent to the following C code:
char *buf = buffer;
char **ref = &buf;
Shorthand for ref.reinterpret(this, …).
Returns a new Buffer instance with the specified size, with the same memory address as buffer.
This function "attaches" buffer to the returned Buffer to prevent it from being garbage collected.
Shorthand for ref.reinterpretUntilZeros(this, …).
Accepts a Buffer instance and a number of NULL bytes to read from the
pointer. This function will scan past the boundary of the Buffer's length
until it finds size number of aligned NULL bytes.
This is useful for finding the end of NUL-termintated array or C string. For
example, the readCString() function could be implemented like:
function readCString (buf) {
return ref.reinterpretUntilZeros(buf, 1).toString('utf8')
}
This function "attaches" buffer to the returned Buffer to prevent it from being garbage collected.
Shorthand for ref.writeCString(this, …).
Writes the given string as a C String (NULL terminated) to the given buffer at the given offset. "encoding" is optional and defaults to 'utf8'.
Unlike readCString(), this function requires the buffer to actually have the
proper length.
Shorthand for ref.writeInt64BE(this, …).
Writes the input Number or String as a big-endian signed 64-bit int into buffer at the given offset.
var buf = ref.alloc('int64');
ref.writeInt64BE(buf, 0, '9223372036854775807');
Shorthand for ref.writeInt64LE(this, …).
Writes the input Number or String as a little-endian signed 64-bit int into buffer at the given offset.
var buf = ref.alloc('int64');
ref.writeInt64LE(buf, 0, '9223372036854775807');
Shorthand for ref.writeObject(this, …).
Writes a pointer to object into buffer at the specified _offset.
This function "attaches" object to buffer to prevent it from being garbage collected.
var buf = ref.alloc('Object');
ref.writeObject(buf, 0, { foo: 'bar' });
Shorthand for ref.writePointer(this, …).
Writes the memory address of pointer to buffer at the specified offset.
This function "attaches" object to buffer to prevent it from being garbage collected.
var someBuffer = new Buffer('whatever');
var buf = ref.alloc('pointer');
ref.writePointer(buf, 0, someBuffer);
Shorthand for ref.writeUInt64BE(this, …).
Writes the input Number or String as a big-endian unsigned 64-bit int into buffer at the given offset.
var buf = ref.alloc('uint64');
ref.writeUInt64BE(buf, 0, '18446744073709551615');
Shorthand for ref.writeUInt64LE(this, …).
Writes the input Number or String as a little-endian unsigned 64-bit int into buffer at the given offset.
var buf = ref.alloc('uint64');
ref.writeUInt64LE(buf, 0, '18446744073709551615');