mirror of
https://github.com/HeyPuter/puter.git
synced 2026-10-13 07:10:05 +00:00
fix(kv): SDK list rejections, boot-cache read-after-write, byte-accurate size checks (PUT-1899) (#4108)
- puter.kv.list() no longer throws synchronously for invalid options; it returns a rejected promise, or for stream: true an iterator whose first next() rejects. Error codes unchanged. - Writes through puter.kv (set, del, incr, decr, add, remove, update, expire, expireAt, flush) stop the boot-key batch from serving the keys they touch, so the next get() reads the store. Untouched boot keys are still served from the batch. - Client-side key/value size checks measure UTF-8 bytes, and values are measured as their JSON encoding, matching the store. Same codes and messages.
This commit is contained in:
1 parent
4a54beac6b
commit
fde323842e
20 files changed
+295
-57
No files matched your search
@@ -4,7 +4,7 @@ description: Returns the maximum key size (in bytes) for the key-value store.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
A property of the `puter.kv` object that returns the maximum key size (in bytes) for the key-value store.
|
||||
A property of the `puter.kv` object that returns the maximum key size (in bytes) for the key-value store. A key is measured in UTF-8 bytes, so a key with non-ASCII characters fits fewer characters.
|
||||
|
||||
## Syntax
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Returns the maximum value size (in bytes) for the key-value store.
|
||||
platforms: [websites, apps, nodejs, workers]
|
||||
---
|
||||
|
||||
A property of the `puter.kv` object that returns the maximum value size (in bytes) for the key-value store.
|
||||
A property of the `puter.kv` object that returns the maximum value size (in bytes) for the key-value store. A value is measured as the UTF-8 bytes of `JSON.stringify(value)`, so a string's quotes and escapes count.
|
||||
|
||||
## Syntax
|
||||
|
||||
|
||||
@@ -56,6 +56,8 @@ If the user has no keys, the array will be empty.
|
||||
|
||||
When paginating, iterate until the result has no `cursor` — a page may hold fewer than `limit` items while more pages still exist.
|
||||
|
||||
Invalid options, such as a non-boolean `reverse` or `offset` combined with `stream`, reject the returned promise with an `invalid_request` error; `list()` never throws synchronously. With `stream: true`, the iterator's first `next()` rejects instead.
|
||||
|
||||
Full (non-paginated) listings keep resolving to a plain array, so existing code is unaffected — under the hood the SDK now fetches them page by page. They still read the entire store, though: every page is metered, so on large stores a bare `list()` gets slow and costly (the SDK logs a one-time console warning when a full listing spans multiple pages). Prefer `stream: true` or explicit `limit`/`cursor` pages, and narrow the scan with a `pattern`.
|
||||
|
||||
With `stream: true`, the method returns an async iterator of [`KVListPage`](/Objects/kvlistpage) objects instead:
|
||||
|
||||
@@ -22,11 +22,11 @@ puter.kv.set({ items: [ { key, value, expireAt }, ... ] })
|
||||
|
||||
#### `key` (String) (required)
|
||||
|
||||
A string containing the name of the key you want to create/update. The maximum allowed `key` size is **1 KB**.
|
||||
A string containing the name of the key you want to create/update. The maximum allowed `key` size is **1 KB** (1,024 bytes of UTF-8); a longer key rejects with `key_too_large`.
|
||||
|
||||
#### `value` (String | Number | Boolean | Object | Array)
|
||||
|
||||
The value you want to give the key you are creating/updating. Objects and arrays are stored as-is and come back the same way. The maximum allowed `value` size is **400 KB**.
|
||||
The value you want to give the key you are creating/updating. Objects and arrays are stored as-is and come back the same way. The maximum allowed `value` size is **400 KB**, measured as the UTF-8 bytes of the value's JSON encoding (a string's quotes and escapes count); a larger value rejects with `value_too_large`.
|
||||
|
||||
Numbers are stored with the precision JavaScript itself keeps: every number in the value — including one nested inside an object or array — must be within **±9,007,199,254,740,991** (`Number.MAX_SAFE_INTEGER`). A number past that is stored clamped to the bound rather than rejected, and `NaN` is stored as `null`. Store an id or a total that has to stay exact past that point as a string.
|
||||
|
||||
|
||||
@@ -60,5 +60,6 @@ export async function add (keyOrOptions, valueOrMap, optConfig) {
|
||||
|
||||
assertKeyPresent(options.key);
|
||||
assertKeySize(options.key);
|
||||
this.guiCache.invalidate(options.key);
|
||||
return await utils.makeDriverMethod({ iface: 'puter-kvstore', method: 'add', argNames: ['key'], puter: this.puter })(options);
|
||||
}
|
||||
@@ -50,5 +50,6 @@ export async function decr (keyOrOptions, amountOrMap, optConfig) {
|
||||
const options = parseCounterArgs(keyOrOptions, amountOrMap, optConfig);
|
||||
assertKeyPresent(options.key);
|
||||
assertKeySize(options.key);
|
||||
this.guiCache.invalidate(options.key);
|
||||
return await utils.makeDriverMethod({ iface: 'puter-kvstore', method: 'decr', argNames: ['key'], puter: this.puter })(options);
|
||||
}
|
||||
@@ -4,15 +4,16 @@ import { assertKeyPresent, assertKeySize } from './lib/validate.js';
|
||||
|
||||
/** @typedef {import('./types.js').KVOptConfig} KVOptConfig */
|
||||
|
||||
const delDriverCall = (puter, args) =>
|
||||
const delDriverCall = (/** @type {import('./index.js').KVModule} */ kv, args) =>
|
||||
utils.makeDriverMethod({
|
||||
iface: 'puter-kvstore',
|
||||
method: 'del',
|
||||
argNames: ['key'],
|
||||
puter,
|
||||
puter: kv.puter,
|
||||
preprocess: (driverArgs) => {
|
||||
assertKeyPresent(driverArgs.key);
|
||||
assertKeySize(driverArgs.key);
|
||||
kv.guiCache.invalidate(driverArgs.key);
|
||||
return driverArgs;
|
||||
},
|
||||
})(args);
|
||||
@@ -35,12 +36,10 @@ const delDriverCall = (puter, args) =>
|
||||
* @returns {Promise<boolean>}
|
||||
*/
|
||||
export async function del (keyOrObject, ...rest) {
|
||||
const { puter } = this;
|
||||
|
||||
if ( isObject(keyOrObject) && rest.length === 0 ) {
|
||||
return await delDriverCall(puter, keyOrObject);
|
||||
return await delDriverCall(this, keyOrObject);
|
||||
}
|
||||
|
||||
const { optConfig, success, error } = parseOptConfigThenCallbacks(rest);
|
||||
return await delDriverCall(puter, { key: keyOrObject, optConfig, success, error });
|
||||
return await delDriverCall(this, { key: keyOrObject, optConfig, success, error });
|
||||
}
|
||||
@@ -27,5 +27,6 @@ import { assertKeyPresent, assertKeySize } from './lib/validate.js';
|
||||
export async function expire (key, ttl, optConfig) {
|
||||
assertKeyPresent(key);
|
||||
assertKeySize(key);
|
||||
this.guiCache.invalidate(key);
|
||||
return await utils.makeDriverMethod({ iface: 'puter-kvstore', method: 'expire', argNames: ['key', 'ttl'], puter: this.puter })({ key, ttl, optConfig });
|
||||
}
|
||||
@@ -29,5 +29,6 @@ import { assertKeyPresent, assertKeySize } from './lib/validate.js';
|
||||
export async function expireAt (key, timestamp, optConfig) {
|
||||
assertKeyPresent(key);
|
||||
assertKeySize(key);
|
||||
this.guiCache.invalidate(key);
|
||||
return await utils.makeDriverMethod({ iface: 'puter-kvstore', method: 'expireAt', argNames: ['key', 'timestamp'], puter: this.puter })({ key, timestamp, optConfig });
|
||||
}
|
||||
@@ -26,6 +26,7 @@ const flushDriverCall = (puter, args) =>
|
||||
*/
|
||||
export async function flush (optConfigOrCallback, ...rest) {
|
||||
const { puter } = this;
|
||||
this.guiCache.invalidateAll();
|
||||
|
||||
if ( isObject(optConfigOrCallback) && rest.length === 0 ) {
|
||||
const input = optConfigOrCallback;
|
||||
|
||||
@@ -50,5 +50,6 @@ export async function incr (keyOrOptions, amountOrMap, optConfig) {
|
||||
const options = parseCounterArgs(keyOrOptions, amountOrMap, optConfig);
|
||||
assertKeyPresent(options.key);
|
||||
assertKeySize(options.key);
|
||||
this.guiCache.invalidate(options.key);
|
||||
return await utils.makeDriverMethod({ iface: 'puter-kvstore', method: 'incr', argNames: ['key'], puter: this.puter })(options);
|
||||
}
|
||||
@@ -30,14 +30,15 @@ export class KVModule extends PuterModule {
|
||||
guiCache;
|
||||
|
||||
/**
|
||||
* The maximum allowed key size, in bytes (`1 KB`).
|
||||
* The maximum allowed key size, in UTF-8 bytes (`1 KB`).
|
||||
*
|
||||
* @readonly
|
||||
*/
|
||||
MAX_KEY_SIZE = MAX_KEY_SIZE;
|
||||
|
||||
/**
|
||||
* The maximum allowed value size, in bytes (`400 KB`).
|
||||
* The maximum allowed value size, in UTF-8 bytes of the value's JSON
|
||||
* encoding (`400 KB`).
|
||||
*
|
||||
* @readonly
|
||||
*/
|
||||
|
||||
@@ -223,6 +223,41 @@ describe('kv.set driver payloads', () => {
|
||||
expect(FakeXHR.requests).toHaveLength(0);
|
||||
});
|
||||
|
||||
// The store measures UTF-8 bytes of the value's JSON, so a string's
|
||||
// quotes count and `.length` undercounts everything but ASCII strings.
|
||||
it.each([
|
||||
['an ASCII string one byte over once quoted', 'v'.repeat(399 * 1024 - 1)],
|
||||
['a multibyte string', '\u20ac'.repeat(140 * 1024)],
|
||||
['an object', { blob: 'x'.repeat(399 * 1024) }],
|
||||
['a long array', Array(300_000).fill(0)],
|
||||
])('rejects %s over the byte limit without a request', async (_label, value) => {
|
||||
await expect(kv.set('k', value)).rejects.toEqual({
|
||||
message: `Value size cannot be larger than ${399 * 1024}`,
|
||||
code: 'value_too_large',
|
||||
});
|
||||
expect(FakeXHR.requests).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('accepts a value whose JSON is exactly at the byte limit', async () => {
|
||||
await expect(kv.set('k', 'v'.repeat(399 * 1024 - 2))).resolves.toBe(true);
|
||||
expect(FakeXHR.requests).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('rejects a batch item over the byte limit without a request', async () => {
|
||||
await expect(kv.set([
|
||||
{ key: 'a', value: 1 },
|
||||
{ key: 'b', value: { blob: 'x'.repeat(399 * 1024) } },
|
||||
])).rejects.toMatchObject({ code: 'value_too_large' });
|
||||
expect(FakeXHR.requests).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('rejects a multibyte key over the byte limit without a request', async () => {
|
||||
// 513 characters, 1026 bytes.
|
||||
await expect(kv.set('\u00e9'.repeat(513), 'v')).rejects.toMatchObject({ code: 'key_too_large' });
|
||||
await expect(kv.set([{ key: '\u00e9'.repeat(513), value: 'v' }])).rejects.toMatchObject({ code: 'key_too_large' });
|
||||
expect(FakeXHR.requests).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('rejects an empty batch', async () => {
|
||||
await expect(kv.set([])).rejects.toMatchObject({ code: 'items_required' });
|
||||
});
|
||||
@@ -253,6 +288,7 @@ describe('kv.get driver payloads', () => {
|
||||
|
||||
it('rejects an oversized key without a request', async () => {
|
||||
await expect(kv.get('k'.repeat(1025))).rejects.toMatchObject({ code: 'key_too_large' });
|
||||
await expect(kv.get('\u00e9'.repeat(513))).rejects.toMatchObject({ code: 'key_too_large' });
|
||||
expect(FakeXHR.requests).toHaveLength(0);
|
||||
});
|
||||
|
||||
@@ -307,6 +343,74 @@ describe('kv.get GUI boot cache', () => {
|
||||
await kv.get('ordinary-key');
|
||||
expect(lastBody().args).toEqual({ key: 'ordinary-key' });
|
||||
});
|
||||
|
||||
describe('after a write', () => {
|
||||
// The batch answers 'stale' for every boot key; a single-key read
|
||||
// answers 'fresh', and every other call succeeds.
|
||||
beforeEach(() => {
|
||||
FakeXHR.respondWith = (body) => {
|
||||
if ( body.method === 'get' ) {
|
||||
return Array.isArray(body.args.key)
|
||||
? { success: true, result: body.args.key.map(() => 'stale') }
|
||||
: { success: true, result: 'fresh' };
|
||||
}
|
||||
return { success: true, result: true };
|
||||
};
|
||||
});
|
||||
|
||||
it.each([
|
||||
['set(key, value)', () => kv.set('sidebar_items', 'v')],
|
||||
['set({ key, value })', () => kv.set({ key: 'sidebar_items', value: 'v' })],
|
||||
['set([items])', () => kv.set([{ key: 'sidebar_items', value: 'v' }])],
|
||||
['set({ items })', () => kv.set({ items: [{ key: 'sidebar_items', value: 'v' }] })],
|
||||
['del(key)', () => kv.del('sidebar_items')],
|
||||
['del({ key })', () => kv.del({ key: 'sidebar_items' })],
|
||||
['incr(key)', () => kv.incr('sidebar_items')],
|
||||
['decr(key)', () => kv.decr('sidebar_items')],
|
||||
['add(key, value)', () => kv.add('sidebar_items', 'v')],
|
||||
['remove(key, path)', () => kv.remove('sidebar_items', 'a')],
|
||||
['update(key, map)', () => kv.update('sidebar_items', { a: 1 })],
|
||||
['update({ key, pathAndValueMap })', () => kv.update({ key: 'sidebar_items', pathAndValueMap: { a: 1 } })],
|
||||
['expire(key, ttl)', () => kv.expire('sidebar_items', 60)],
|
||||
['expireAt(key, timestamp)', () => kv.expireAt('sidebar_items', 1)],
|
||||
['flush()', () => kv.flush()],
|
||||
])('%s makes the next get(key) read the store', async (_label, write) => {
|
||||
await expect(kv.get('menubar_style')).resolves.toBe('stale');
|
||||
await write();
|
||||
await expect(kv.get('sidebar_items')).resolves.toBe('fresh');
|
||||
expect(lastBody()).toMatchObject({ method: 'get', args: { key: 'sidebar_items' } });
|
||||
});
|
||||
|
||||
it('keeps serving the boot keys that were not written', async () => {
|
||||
await kv.get('menubar_style');
|
||||
await kv.set('sidebar_items', 'v');
|
||||
const requests = FakeXHR.requests.length;
|
||||
await expect(kv.get('menubar_style')).resolves.toBe('stale');
|
||||
expect(FakeXHR.requests).toHaveLength(requests);
|
||||
});
|
||||
|
||||
it('a write before the batch is fetched still bypasses it for that key', async () => {
|
||||
await kv.set('sidebar_items', 'v');
|
||||
await expect(kv.get('sidebar_items')).resolves.toBe('fresh');
|
||||
await expect(kv.get('menubar_style')).resolves.toBe('stale');
|
||||
});
|
||||
|
||||
it('a write issued while the batch is in flight bypasses it', async () => {
|
||||
const pending = kv.get('sidebar_items');
|
||||
const write = kv.set('sidebar_items', 'v');
|
||||
await expect(pending).resolves.toBe('fresh');
|
||||
await write;
|
||||
});
|
||||
|
||||
it('a rejected write still bypasses the batch', async () => {
|
||||
await kv.get('menubar_style');
|
||||
FakeXHR.respondWith = (body) => body.method === 'set'
|
||||
? { success: false, error: { code: 'insufficient_funds' } }
|
||||
: { success: true, result: 'fresh' };
|
||||
await expect(kv.set('sidebar_items', 'v')).rejects.toBeTruthy();
|
||||
await expect(kv.get('sidebar_items')).resolves.toBe('fresh');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('kv.incr / kv.decr driver payloads', () => {
|
||||
@@ -760,14 +864,22 @@ describe('kv.list driver payloads', () => {
|
||||
expect(lastBody().args).toEqual({ as: 'keys', limit: 1000, fetchUntilFull: true, cursor: 'c9' });
|
||||
});
|
||||
|
||||
it('list({ stream: true, offset }) rejects client-side', () => {
|
||||
let err;
|
||||
try {
|
||||
kv.list({ stream: true, offset: 1 });
|
||||
} catch (e) {
|
||||
err = e;
|
||||
}
|
||||
expect(err).toMatchObject({ code: 'invalid_request' });
|
||||
it('list({ stream: true, offset }) rejects on the first next() without a request', async () => {
|
||||
const pages = kv.list({ stream: true, offset: 1 });
|
||||
await expect(pages.next()).rejects.toMatchObject({ code: 'invalid_request' });
|
||||
expect(FakeXHR.requests).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('list({ reverse: nonBoolean }) rejects instead of throwing', async () => {
|
||||
const result = kv.list({ reverse: 'yes' });
|
||||
expect(result).toBeInstanceOf(Promise);
|
||||
await expect(result).rejects.toEqual({ message: 'reverse must be a boolean', code: 'invalid_request' });
|
||||
expect(FakeXHR.requests).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('list({ stream: true, reverse: nonBoolean }) rejects on the first next()', async () => {
|
||||
const pages = kv.list({ stream: true, reverse: 1 });
|
||||
await expect(pages.next()).rejects.toMatchObject({ code: 'invalid_request' });
|
||||
expect(FakeXHR.requests).toHaveLength(0);
|
||||
});
|
||||
|
||||
|
||||
@@ -39,12 +39,15 @@ const createDeferred = () => {
|
||||
/**
|
||||
* Boot-time read cache for the GUI's well-known keys. Lazy: nothing is
|
||||
* fetched until the first `lookup()` resolves the init deferred; every read
|
||||
* within the lifetime window is then served from the one batched response.
|
||||
* within the lifetime window is then served from the one batched response,
|
||||
* except for keys written through the module since it was created.
|
||||
*/
|
||||
export class GuiBootCache {
|
||||
/** @param {import('../../../index.js').Puter} puter */
|
||||
constructor (puter) {
|
||||
this.puter = puter;
|
||||
/** @type {Set<string>} */
|
||||
this.written = new Set();
|
||||
this.batch = createDeferred();
|
||||
this.init = createDeferred();
|
||||
(async () => {
|
||||
@@ -85,19 +88,40 @@ export class GuiBootCache {
|
||||
|
||||
/** True when `key` is a boot key this cache can still serve. */
|
||||
serves (key) {
|
||||
return typeof key === 'string' && GUI_CACHE_KEYS.includes(key) && this.batch !== null;
|
||||
return typeof key === 'string' && GUI_CACHE_KEYS.includes(key) && !this.written.has(key) && this.batch !== null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Stops serving `keys` from the batch, so a read after a write goes to the
|
||||
* store. Called when a write is issued, before it lands.
|
||||
*
|
||||
* @param {...unknown} keys
|
||||
*/
|
||||
invalidate (...keys) {
|
||||
if ( this.batch === null ) return;
|
||||
for ( const key of keys ) {
|
||||
if ( typeof key === 'string' && GUI_CACHE_KEYS.includes(key) ) {
|
||||
this.written.add(key);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
invalidateAll () {
|
||||
this.invalidate(...GUI_CACHE_KEYS);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {string} key
|
||||
* @returns {Promise<{ hit: true, value: unknown } | { hit: false }>}
|
||||
* `hit: false` when the batch failed or came back malformed — the
|
||||
* caller should fall through to its own driver call.
|
||||
* `hit: false` when the batch failed, came back malformed, or the key
|
||||
* was written meanwhile — the caller should fall through to its own
|
||||
* driver call.
|
||||
*/
|
||||
async lookup (key) {
|
||||
this.init && this.init.resolve();
|
||||
const cache = await this.batch.promise;
|
||||
if ( cache === null ) return { hit: false };
|
||||
// Written while this read waited: the batch may predate the write.
|
||||
if ( cache === null || this.written.has(key) ) return { hit: false };
|
||||
// `null`, not `undefined`, to match a normal miss from `kv.get()`.
|
||||
return { hit: true, value: cache[key] ?? null };
|
||||
}
|
||||
|
||||
@@ -6,6 +6,11 @@ export const MAX_VALUE_SIZE = 399 * 1024;
|
||||
// Validators throw the stable `{ message, code }` error objects the SDK
|
||||
// documents; the codes are API surface and must not change.
|
||||
|
||||
const encoder = new TextEncoder();
|
||||
|
||||
/** @param {string} text */
|
||||
const utf8Length = (text) => encoder.encode(text).length;
|
||||
|
||||
/** @param {unknown} key */
|
||||
export const assertKeyPresent = (key) => {
|
||||
if ( key === undefined || key === null ) {
|
||||
@@ -13,16 +18,35 @@ export const assertKeyPresent = (key) => {
|
||||
}
|
||||
};
|
||||
|
||||
/** @param {unknown} key */
|
||||
/**
|
||||
* Measured like the store does, in UTF-8 bytes. An array (a batched read) is
|
||||
* checked key by key; non-string keys are coerced and checked by the store.
|
||||
*
|
||||
* @param {unknown} key
|
||||
*/
|
||||
export const assertKeySize = (key) => {
|
||||
if ( /** @type {{ length?: number }} */ (key).length > MAX_KEY_SIZE ) {
|
||||
throw { message: `Key size cannot be larger than ${MAX_KEY_SIZE}`, code: 'key_too_large' };
|
||||
for ( const k of Array.isArray(key) ? key : [key] ) {
|
||||
if ( typeof k === 'string' && utf8Length(k) > MAX_KEY_SIZE ) {
|
||||
throw { message: `Key size cannot be larger than ${MAX_KEY_SIZE}`, code: 'key_too_large' };
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
/** @param {unknown} value */
|
||||
/**
|
||||
* Measured like the store does: UTF-8 bytes of the value's JSON encoding.
|
||||
*
|
||||
* @param {unknown} value
|
||||
*/
|
||||
export const assertValueSize = (value) => {
|
||||
if ( value && /** @type {{ length?: number }} */ (value).length > MAX_VALUE_SIZE ) {
|
||||
let json;
|
||||
try {
|
||||
json = JSON.stringify(value ?? null);
|
||||
} catch {
|
||||
// Not JSON-encodable (a cycle, a BigInt); the request fails to encode
|
||||
// the same way, so there is no size to check.
|
||||
return;
|
||||
}
|
||||
if ( typeof json === 'string' && utf8Length(json) > MAX_VALUE_SIZE ) {
|
||||
throw { message: `Value size cannot be larger than ${MAX_VALUE_SIZE}`, code: 'value_too_large' };
|
||||
}
|
||||
};
|
||||
@@ -54,6 +54,11 @@ const normalizeListPattern = (pattern) => {
|
||||
return trimmed;
|
||||
};
|
||||
|
||||
// Argument errors surface where network errors do: a rejected promise, or for
|
||||
// `stream` an iterator whose first `next()` rejects.
|
||||
const failList = (stream, error) =>
|
||||
stream ? (async function* () { throw error; })() : Promise.reject(error);
|
||||
|
||||
/**
|
||||
* @overload
|
||||
* @param {string} [pattern]
|
||||
@@ -159,9 +164,10 @@ export function list (patternOrOptions, returnValuesOrOptConfig, maybeOptConfig)
|
||||
|
||||
if ( isOptionsObject ) {
|
||||
const input = patternOrOptions;
|
||||
stream = input.stream === true;
|
||||
if ( input.reverse !== undefined ) {
|
||||
if ( typeof input.reverse !== 'boolean' ) {
|
||||
throw { message: 'reverse must be a boolean', code: 'invalid_request' };
|
||||
return failList(stream, { message: 'reverse must be a boolean', code: 'invalid_request' });
|
||||
}
|
||||
options.reverse = input.reverse;
|
||||
}
|
||||
@@ -169,7 +175,6 @@ export function list (patternOrOptions, returnValuesOrOptConfig, maybeOptConfig)
|
||||
pattern = input.pattern;
|
||||
}
|
||||
returnValues = !!input.returnValues;
|
||||
stream = input.stream === true;
|
||||
if ( isObject(input.optConfig) ) {
|
||||
options.optConfig = input.optConfig;
|
||||
} else if ( isOptConfigShorthand(input) ) {
|
||||
@@ -224,7 +229,7 @@ export function list (patternOrOptions, returnValuesOrOptConfig, maybeOptConfig)
|
||||
|
||||
if ( stream ) {
|
||||
if ( options.offset !== undefined ) {
|
||||
throw { message: '`offset` cannot be combined with `stream`; pass `cursor` to resume from a position.', code: 'invalid_request' };
|
||||
return failList(stream, { message: '`offset` cannot be combined with `stream`; pass `cursor` to resume from a position.', code: 'invalid_request' });
|
||||
}
|
||||
const base = { ...options };
|
||||
delete base.cursor;
|
||||
|
||||
@@ -42,6 +42,7 @@ export async function remove (key, ...pathsAndOptConfig) {
|
||||
if ( paths.some((path) => typeof path !== 'string') ) {
|
||||
throw { message: 'All paths must be strings', code: 'paths_invalid' };
|
||||
}
|
||||
this.guiCache.invalidate(key);
|
||||
|
||||
return await utils.makeDriverMethod({ iface: 'puter-kvstore', method: 'remove', argNames: ['key', 'paths'], puter: this.puter })({ key, paths, optConfig });
|
||||
}
|
||||
@@ -17,26 +17,29 @@ import { assertKeyPresent, assertKeySize, assertValueSize } from './lib/validate
|
||||
* @typedef {import('./types.js').KVSetObject<T>} KVSetObject
|
||||
*/
|
||||
|
||||
const setSingle = (puter, args) =>
|
||||
/** @typedef {import('./index.js').KVModule} KVModule */
|
||||
|
||||
const setSingle = (/** @type {KVModule} */ kv, args) =>
|
||||
utils.makeDriverMethod({
|
||||
iface: 'puter-kvstore',
|
||||
method: 'set',
|
||||
argNames: ['key', 'value', 'expireAt'],
|
||||
puter,
|
||||
puter: kv.puter,
|
||||
preprocess: (driverArgs) => {
|
||||
assertKeyPresent(driverArgs.key);
|
||||
assertKeySize(driverArgs.key);
|
||||
assertValueSize(driverArgs.value);
|
||||
kv.guiCache.invalidate(driverArgs.key);
|
||||
return driverArgs;
|
||||
},
|
||||
})(args);
|
||||
|
||||
const setBatch = (puter, args) =>
|
||||
const setBatch = (/** @type {KVModule} */ kv, args) =>
|
||||
utils.makeDriverMethod({
|
||||
iface: 'puter-kvstore',
|
||||
method: 'batchPut',
|
||||
argNames: ['items'],
|
||||
puter,
|
||||
puter: kv.puter,
|
||||
preprocess: (driverArgs) => {
|
||||
if ( !Array.isArray(driverArgs.items) || driverArgs.items.length === 0 ) {
|
||||
throw { message: 'Items are required', code: 'items_required' };
|
||||
@@ -60,6 +63,7 @@ const setBatch = (puter, args) =>
|
||||
...(item.expireAt !== undefined ? { expireAt: item.expireAt } : {}),
|
||||
};
|
||||
});
|
||||
kv.guiCache.invalidate(...items.map((item) => item.key));
|
||||
|
||||
return {
|
||||
...driverArgs,
|
||||
@@ -120,19 +124,17 @@ const setBatch = (puter, args) =>
|
||||
* @returns {Promise<boolean>}
|
||||
*/
|
||||
export async function set (keyOrItems, value, ...rest) {
|
||||
const { puter } = this;
|
||||
|
||||
if ( Array.isArray(keyOrItems) ) {
|
||||
const trailing = [value, ...rest];
|
||||
const { optConfig, success, error } = parseTrailingArgs(trailing);
|
||||
return await setBatch(puter, { items: keyOrItems, optConfig, success, error });
|
||||
return await setBatch(this, { items: keyOrItems, optConfig, success, error });
|
||||
}
|
||||
|
||||
if ( isObject(keyOrItems) && value === undefined && rest.length === 0 ) {
|
||||
if ( Array.isArray(keyOrItems.items) ) {
|
||||
return await setBatch(puter, keyOrItems);
|
||||
return await setBatch(this, keyOrItems);
|
||||
}
|
||||
return await setSingle(puter, keyOrItems);
|
||||
return await setSingle(this, keyOrItems);
|
||||
}
|
||||
|
||||
let expireAt;
|
||||
@@ -142,5 +144,5 @@ export async function set (keyOrItems, value, ...rest) {
|
||||
expireAt = rest.shift();
|
||||
}
|
||||
const { optConfig, success, error } = parseTrailingArgs(rest);
|
||||
return await setSingle(puter, { key: keyOrItems, value, expireAt, optConfig, success, error });
|
||||
return await setSingle(this, { key: keyOrItems, value, expireAt, optConfig, success, error });
|
||||
}
|
||||
@@ -7,12 +7,12 @@ import { assertKeyPresent, assertKeySize } from './lib/validate.js';
|
||||
/** @typedef {import('./types.js').KVUpdatePath} KVUpdatePath */
|
||||
/** @typedef {import('./types.js').KVValue} KVValue */
|
||||
|
||||
const updateDriverCall = (puter, args) =>
|
||||
const updateDriverCall = (/** @type {import('./index.js').KVModule} */ kv, args) =>
|
||||
utils.makeDriverMethod({
|
||||
iface: 'puter-kvstore',
|
||||
method: 'update',
|
||||
argNames: ['key', 'pathAndValueMap', 'ttl'],
|
||||
puter,
|
||||
puter: kv.puter,
|
||||
preprocess: (driverArgs) => {
|
||||
assertKeyPresent(driverArgs.key);
|
||||
assertKeySize(driverArgs.key);
|
||||
@@ -32,6 +32,7 @@ const updateDriverCall = (puter, args) =>
|
||||
}
|
||||
driverArgs.ttl = ttl;
|
||||
}
|
||||
kv.guiCache.invalidate(driverArgs.key);
|
||||
return driverArgs;
|
||||
},
|
||||
})(args);
|
||||
@@ -71,10 +72,8 @@ const updateDriverCall = (puter, args) =>
|
||||
* @returns {Promise<KVValue>}
|
||||
*/
|
||||
export async function update (keyOrObject, pathAndValueMap, ...rest) {
|
||||
const { puter } = this;
|
||||
|
||||
if ( isObject(keyOrObject) && pathAndValueMap === undefined && rest.length === 0 ) {
|
||||
return await updateDriverCall(puter, keyOrObject);
|
||||
return await updateDriverCall(this, keyOrObject);
|
||||
}
|
||||
|
||||
let ttl;
|
||||
@@ -84,5 +83,5 @@ export async function update (keyOrObject, pathAndValueMap, ...rest) {
|
||||
ttl = rest.shift();
|
||||
}
|
||||
const { optConfig, success, error } = parseTrailingArgs(rest);
|
||||
return await updateDriverCall(puter, { key: keyOrObject, pathAndValueMap, ttl, optConfig, success, error });
|
||||
return await updateDriverCall(this, { key: keyOrObject, pathAndValueMap, ttl, optConfig, success, error });
|
||||
}
|
||||
@@ -395,14 +395,32 @@ export default suite('kv', {
|
||||
t.assert.deepEqual(seen, ['kv-suite-stream-nolimit']);
|
||||
},
|
||||
|
||||
'list with stream rejects offset client-side': async (t) => {
|
||||
let err: { code?: string } | undefined;
|
||||
try {
|
||||
t.puter.kv.list({ stream: true, offset: 1 } as never);
|
||||
} catch (e) {
|
||||
err = e as { code?: string };
|
||||
}
|
||||
t.assert.equal(err?.code, 'invalid_request');
|
||||
'list with stream rejects offset on the first page': async (t) => {
|
||||
const pages = t.puter.kv.list({
|
||||
stream: true,
|
||||
offset: 1,
|
||||
} as never) as AsyncIterableIterator<unknown>;
|
||||
const err = await t.assert.rejects(
|
||||
() => pages.next(),
|
||||
'offset with stream should be rejected',
|
||||
);
|
||||
t.assert.equal((err as { code?: string })?.code, 'invalid_request');
|
||||
},
|
||||
|
||||
'list with a non-boolean reverse returns a rejected promise': async (t) => {
|
||||
const result = t.puter.kv.list({
|
||||
reverse: 'yes',
|
||||
} as never) as Promise<unknown>;
|
||||
t.assert.equal(
|
||||
typeof result.then,
|
||||
'function',
|
||||
'list should return a promise',
|
||||
);
|
||||
const err = await t.assert.rejects(
|
||||
() => result,
|
||||
'a non-boolean reverse should be rejected',
|
||||
);
|
||||
t.assert.equal((err as { code?: string })?.code, 'invalid_request');
|
||||
},
|
||||
|
||||
// -- argument shapes --
|
||||
@@ -519,6 +537,40 @@ export default suite('kv', {
|
||||
);
|
||||
},
|
||||
|
||||
'set measures value size in bytes of the JSON value': async (t) => {
|
||||
const max = t.puter.kv.MAX_VALUE_SIZE;
|
||||
for (const value of [
|
||||
'\u20ac'.repeat(Math.ceil(max / 3)),
|
||||
{ blob: 'x'.repeat(max) },
|
||||
Array(max).fill(0),
|
||||
]) {
|
||||
const err = await t.assert.rejects(
|
||||
() => t.puter.kv.set('kv-suite-oversized-json', value),
|
||||
'a value over the byte cap should be rejected',
|
||||
);
|
||||
t.assert.equal((err as { code?: string })?.code, 'value_too_large');
|
||||
}
|
||||
// Quotes included, this string is exactly at the cap.
|
||||
const atCap = 'v'.repeat(max - 2);
|
||||
t.assert.equal(
|
||||
await t.puter.kv.set('kv-suite-value-at-cap', atCap),
|
||||
true,
|
||||
);
|
||||
t.assert.equal(await t.puter.kv.get('kv-suite-value-at-cap'), atCap);
|
||||
},
|
||||
|
||||
'set rejects a multibyte key over the byte cap': async (t) => {
|
||||
const err = await t.assert.rejects(
|
||||
() =>
|
||||
t.puter.kv.set(
|
||||
'\u00e9'.repeat(t.puter.kv.MAX_KEY_SIZE / 2 + 1),
|
||||
'x',
|
||||
),
|
||||
'a key over the byte cap should be rejected',
|
||||
);
|
||||
t.assert.equal((err as { code?: string })?.code, 'key_too_large');
|
||||
},
|
||||
|
||||
'set rejects an oversized key with key_too_large': async (t) => {
|
||||
const err = await t.assert.rejects(
|
||||
() => t.puter.kv.set('k'.repeat(t.puter.kv.MAX_KEY_SIZE + 1), 'x'),
|
||||
@@ -839,6 +891,17 @@ export default suite('kv', {
|
||||
);
|
||||
},
|
||||
|
||||
'writes to a boot-cached key are visible to the next get': async (t) => {
|
||||
// The first get() of one of these keys batch-reads all of them, and
|
||||
// the batch serves reads for a few seconds afterwards.
|
||||
await t.puter.kv.set('desktop_icons_hidden', 'true');
|
||||
t.assert.equal(await t.puter.kv.get('taskbar_position'), null);
|
||||
await t.puter.kv.set('taskbar_position', 'left');
|
||||
t.assert.equal(await t.puter.kv.get('taskbar_position'), 'left');
|
||||
await t.puter.kv.del('desktop_icons_hidden');
|
||||
t.assert.equal(await t.puter.kv.get('desktop_icons_hidden'), null);
|
||||
},
|
||||
|
||||
'clear is an alias of flush and empties the store': async (t) => {
|
||||
await t.puter.kv.set('kv-suite-clear-a', 1);
|
||||
await t.puter.kv.clear();
|
||||
|
||||
Reference in new issue
Block a user